[FFmpeg-devel] [PATCH 0/2] Support long file names on Windows

[FFmpeg-devel] [PATCH 0/2] Support long file names on Windows

[ffmpegagent]

This patchset adds support for long file and directory paths on Windows. The
implementation follows the same logic that .NET is using internally, with
the only exception that it doesn't expand short path components in 8.3
format. .NET does this as the same function is also used for other purposes,
but in our case, that's not required. Short (8.3) paths are working as well
with the extended path prefix, even when longer than 260.

Successfully tested:

 * Regular paths wth drive letter
 * Regular UNC paths
 * Long paths wth drive letter
 * Long paths wth drive letter and forward slashes
 * Long UNC paths
 * Prefixed paths wth drive letter
 * Prefixed UNC paths

I have kept the individual functions separate on purpose, to make it easy to
compare with the .NET impl. (compilers should inlinie those anyway)

softworkz (2):
  avutil/wchar_filename,file_open: Support long file names on Windows
  avformat/os_support: Support long file names on Windows

 libavformat/os_support.h   |   8 +--
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 123 +++++++++++++++++++++++++++++++++++++
 3 files changed, 128 insertions(+), 5 deletions(-)


base-commit: d2d8b9b972ba2df6b2a2ebe29f5307cbb7a69c33
Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-28%2Fsoftworkz%2Fsubmit_long_filenames-v1
Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v1
Pull-Request: https://github.com/ffstaging/FFmpeg/pull/28
-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 123 +++++++++++++++++++++++++++++++++++++
 2 files changed, 124 insertions(+), 1 deletion(-)

diff --git a/libavutil/file_open.c b/libavutil/file_open.c
index cc302f2f76..57c5e78d51 100644
--- a/libavutil/file_open.c
+++ b/libavutil/file_open.c
@@ -45,7 +45,7 @@ static int win32_open(const char *filename_utf8, int oflag, int pmode)
     wchar_t *filename_w;
 
     /* convert UTF-8 to wide chars */
-    if (utf8towchar(filename_utf8, &filename_w))
+    if (get_extended_win32_path(filename_utf8, &filename_w))
         return -1;
     if (!filename_w)
         goto fallback;
diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h
index 90f082452c..94b4087de0 100644
--- a/libavutil/wchar_filename.h
+++ b/libavutil/wchar_filename.h
@@ -40,6 +40,129 @@ static inline int utf8towchar(const char *filename_utf8, wchar_t **filename_w)
     MultiByteToWideChar(CP_UTF8, 0, filename_utf8, -1, *filename_w, num_chars);
     return 0;
 }
+
+static inline int path_is_extended(const wchar_t *path)
+{
+    // see .NET6: PathInternal.IsExtended()
+    size_t len = wcslen(path);
+    if (len >= 4  && path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\')
+        return 1;
+
+    return 0;
+}
+
+static inline int get_full_path_name(wchar_t **ppath_w)
+{
+    int num_chars;
+    wchar_t *temp_w;
+
+    // see .NET6: PathHelper.GetFullPathName()
+    num_chars = GetFullPathNameW(*ppath_w, 0, NULL, NULL);
+    if (num_chars <= 0) {
+        errno = EINVAL;
+        return -1;
+    }
+
+    temp_w = (wchar_t *)av_calloc(num_chars, sizeof(wchar_t));
+    if (!temp_w) {
+        errno = ENOMEM;
+        return -1;
+    }
+
+    num_chars = GetFullPathNameW(*ppath_w, num_chars, temp_w, NULL);
+    if (num_chars <= 0) {
+        errno = EINVAL;
+        return -1;
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+static inline int path_normalize(wchar_t **ppath_w)
+{
+    int ret;
+
+    // see .NET6: PathHelper.Normalize()
+    if ((ret = get_full_path_name(ppath_w)) < 0)
+        return ret;
+
+    /* What .NET does at this point is to call PathHelper.TryExpandShortFileName()
+       in case the path contains a '~' character.
+       We don't need to do this as we don't need to normalize the file name
+       for presentation, and the extended path prefix works with 8.3 path
+       components as well */
+    return 0;
+}
+
+static inline int add_extended_prefix(wchar_t **ppath_w)
+{
+    const wchar_t *unc_prefix           = L"\\\\?\\UNC\\";
+    const wchar_t *extended_path_prefix = L"\\\\?\\";
+    const wchar_t *path_w               = *ppath_w;
+    const size_t len                    = wcslen(path_w);
+    wchar_t *temp_w;
+
+    if (len < 2)
+        return 0;
+
+    // see .NET6: PathInternal.EnsureExtendedPrefix()
+    if (path_w[0] == L'\\' && path_w[1] == L'\\') {
+        temp_w = (wchar_t *)av_calloc(len + 6 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, unc_prefix);
+        wcscat(temp_w, path_w + 2);
+    } else {
+        temp_w = (wchar_t *)av_calloc(len + 4 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, extended_path_prefix);
+        wcscat(temp_w, path_w);
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+static inline int get_extended_win32_path(const char *path, wchar_t **ppath_w)
+{
+    int ret;
+    size_t len;
+
+    // see .NET6: Path.GetFullPath() and Path.GetFullPathInternal()
+    if ((ret = utf8towchar(path, ppath_w)) < 0)
+        return ret;
+
+    if (path_is_extended(*ppath_w)) {
+        /* \\?\ paths are considered normalized by definition. Windows doesn't normalize \\?\
+           paths and neither should we. Even if we wanted to, GetFullPathName does not work
+           properly with device paths. If one wants to pass a \\?\ path through normalization
+           one can chop off the prefix, pass it to GetFullPath and add it again. */
+        return 0;
+    }
+
+    if ((ret = path_normalize(ppath_w)) < 0)
+        return ret;
+
+    // see .NET6: PathInternal.EnsureExtendedPrefixIfNeeded()
+    len = wcslen(*ppath_w);
+    if (len >= 260 || (*ppath_w)[len - 1] == L' ' || (*ppath_w)[len - 1] == L'.') {
+        if ((ret = add_extended_prefix(ppath_w)) < 0)
+            return ret;
+    }
+
+    return 0;
+}
+
 #endif
 
 #endif /* AVUTIL_WCHAR_FILENAME_H */
-- 
ffmpeg-codebot

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/2] avformat/os_support: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavformat/os_support.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/libavformat/os_support.h b/libavformat/os_support.h
index 5e6b32d2dc..6c1e6c3851 100644
--- a/libavformat/os_support.h
+++ b/libavformat/os_support.h
@@ -153,7 +153,7 @@ static inline int win32_##name(const char *filename_utf8) \
     wchar_t *filename_w;                                  \
     int ret;                                              \
                                                           \
-    if (utf8towchar(filename_utf8, &filename_w))          \
+    if (get_extended_win32_path(filename_utf8, &filename_w)) \
         return -1;                                        \
     if (!filename_w)                                      \
         goto fallback;                                    \
@@ -177,7 +177,7 @@ static inline int win32_##name(const char *filename_utf8, partype par) \
     wchar_t *filename_w;                                  \
     int ret;                                              \
                                                           \
-    if (utf8towchar(filename_utf8, &filename_w))          \
+    if (get_extended_win32_path(filename_utf8, &filename_w)) \
         return -1;                                        \
     if (!filename_w)                                      \
         goto fallback;                                    \
@@ -199,9 +199,9 @@ static inline int win32_rename(const char *src_utf8, const char *dest_utf8)
     wchar_t *src_w, *dest_w;
     int ret;
 
-    if (utf8towchar(src_utf8, &src_w))
+    if (get_extended_win32_path(src_utf8, &src_w))
         return -1;
-    if (utf8towchar(dest_utf8, &dest_w)) {
+    if (get_extended_win32_path(dest_utf8, &dest_w)) {
         av_free(src_w);
         return -1;
     }
-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/2] Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpegagent <ffmpegagent@gmail.com>
> Sent: Friday, May 13, 2022 11:53 AM
> To: ffmpeg-devel@ffmpeg.org
> Cc: softworkz <softworkz@hotmail.com>
> Subject: [PATCH 0/2] Support long file names on Windows
> 
> This patchset adds support for long file and directory paths on
> Windows. The
> implementation follows the same logic that .NET is using internally,
> with
> the only exception that it doesn't expand short path components in 8.3
> format. .NET does this as the same function is also used for other
> purposes,
> but in our case, that's not required. Short (8.3) paths are working as
> well
> with the extended path prefix, even when longer than 260.
> 
> Successfully tested:
> 
>  * Regular paths wth drive letter
>  * Regular UNC paths
>  * Long paths wth drive letter
>  * Long paths wth drive letter and forward slashes
>  * Long UNC paths
>  * Prefixed paths wth drive letter
>  * Prefixed UNC paths
> 
> I have kept the individual functions separate on purpose, to make it
> easy to
> compare with the .NET impl. (compilers should inlinie those anyway)

Forgot to mention that I tested this with builds from both, MSVC and 
MinGW/MSYS2 (gcc 10.3).

sw


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[nil-admirari]

> diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h
> ...
> +static inline int path_is_extended(const wchar_t *path)
> ...

Why path handling functions ended up in wchar_filename.h?
Isn't it better to move them to file_open or os_support?

> +    num_chars = GetFullPathNameW(*ppath_w, num_chars, temp_w, NULL);

Turns out that GetFullPathNameW handles long path names without the manifest
or a prefix \\?\. Other WinAPI functions, require either a prefix or a manifest,
which is why I thought that path normalisation must be done by hand.

> +static inline int path_normalize(wchar_t **ppath_w)
> +{
> +    int ret;
> +
> +    // see .NET6: PathHelper.Normalize()
> +    if ((ret = get_full_path_name(ppath_w)) < 0)
> +        return ret;
> +
> +    /* What .NET does at this point is to call PathHelper.TryExpandShortFileName()
> +       in case the path contains a '~' character.
> +       We don't need to do this as we don't need to normalize the file name
> +       for presentation, and the extended path prefix works with 8.3 path
> +       components as well */
> +    return 0;
> +}

This function simply forwards the return code of get_full_path_name().
The only non-trivial part of it is a comment.

> +static inline int path_is_extended(const wchar_t *path)
> +{
> +    // see .NET6: PathInternal.IsExtended()
> +    size_t len = wcslen(path);
> +    if (len >= 4  && path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\')
> +        return 1;
> +
> +    return 0;
> +}
>
> +static inline int add_extended_prefix(wchar_t **ppath_w)
> +{
> +    const wchar_t *unc_prefix           = L"\\\\?\\UNC\\";
> ...
> +    // see .NET6: PathInternal.EnsureExtendedPrefix()
> +    if (path_w[0] == L'\\' && path_w[1] == L'\\') {
> ...
> +        wcscpy(temp_w, unc_prefix);
> +        wcscat(temp_w, path_w + 2);
> 
> +static inline int get_extended_win32_path(const char *path, wchar_t **ppath_w)
> +{
> ...
> +    if (path_is_extended(*ppath_w)) {
> +        ...
> +        return 0;
> +    }
> ...
> +        if ((ret = add_extended_prefix(ppath_w)) < 0)

Actual PathInternal.EnsureExtendedPrefix
(https://github.com/dotnet/runtime/blob/main/src/libraries/Common/src/System/IO/PathInternal.Windows.cs)
checks for

            if (IsPartiallyQualified(path.AsSpan()) || IsDevice(path.AsSpan()))
                return path;

where IsDevice handles \\.\, which you do not handle. If I'm not mistaken,
the code paths presented above will turn such paths into \\?\UNC\\.\,
which is an error.

> +static inline int add_extended_prefix(wchar_t **ppath_w)
> +{
> +    const wchar_t *unc_prefix           = L"\\\\?\\UNC\\";
> ...
> +        temp_w = (wchar_t *)av_calloc(len + 6 + 1, sizeof(wchar_t));

Wouldn't it be better to use sizeof unc_prefix instead of magic numbers?



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/2] avformat/os_support: Support long file names on Windows

[nil-admirari]

> diff --git a/libavformat/os_support.h b/libavformat/os_support.h

In addition to what you've already added, this file defines stat as:

#ifdef _WIN32
...
#  ifdef stat
#   undef stat
#  endif
#  define stat _stati64
...

which is
1. not wide-char aware (_wstati64 does exist)
2. not long path aware.

Also there is a function:

static inline int is_dos_path(const char *path)
{
#if HAVE_DOS_PATHS
    if (path[0] && path[1] == ':')
        return 1;
#endif
    return 0;
}

Now, DOS paths C:... end up being \\?\C:.... Are you sure it won't break something down the line?



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/2] Support long file names on Windows

[nil-admirari]

> I have kept the individual functions separate on purpose, to make it easy to
> compare with the .NET impl. (compilers should inlinie those anyway)

Calling add_extended_prefix without pre-validation results in error,
since it does check for \\?\, \\.\, or \??\; only it's wrapper get_extended_win32_path does.
And it's not private, it's in the header.

path_normalize is a do nothing function.

Keeping the comments about where the code originated from may be useful.
Copying the structure of .NET results in problems.



These patches are very difficult to review. E.g. stat is not covered:
https://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/296448.html.

Parts of FFmpeg still use fopen instead of av_fopen_utf8. Some of these uses are
in examples or tests or inside #ifdef DEBUG blocks.
Others aren't: dvdsubdec.c:620 uses fopen and it is not covered by your
previous patchset https://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/296189.html.

So please check that all uses of CRT functions that do I/O:
https://docs.microsoft.com/en-us/cpp/c-runtime-library/stream-i-o?view=msvc-170
https://docs.microsoft.com/en-us/cpp/c-runtime-library/low-level-i-o?view=msvc-170
https://docs.microsoft.com/en-us/cpp/c-runtime-library/directory-control?view=msvc-170
https://docs.microsoft.com/en-us/cpp/c-runtime-library/file-handling?view=msvc-170
are actually covered (plus WinAPI functions as well).



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Sunday, May 15, 2022 9:03 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH 1/2] avutil/wchar_filename,
> file_open: Support long file names on Windows
> 
> > diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h
> > ...
> > +static inline int path_is_extended(const wchar_t *path)
> > ...
> 
> Why path handling functions ended up in wchar_filename.h?
> Isn't it better to move them to file_open or os_support?

The functions are needed in both. file_open.c cannot be included
in libavformat/os_support.h and neither the other way round, 
so they need to be in a 3rd place. How about renaming
wchar_filename.h to windows_filename.h ?


> > +    num_chars = GetFullPathNameW(*ppath_w, num_chars, temp_w,
> NULL);
> 
> Turns out that GetFullPathNameW handles long path names without the
> manifest
> or a prefix \\?\. Other WinAPI functions, require either a prefix or a
> manifest,
> which is why I thought that path normalisation must be done by hand.

Yea, that's where we are lucky.


> > +static inline int path_normalize(wchar_t **ppath_w)
> > +{
> > +    int ret;
> > +
> > +    // see .NET6: PathHelper.Normalize()
> > +    if ((ret = get_full_path_name(ppath_w)) < 0)
> > +        return ret;
> > +
> > +    /* What .NET does at this point is to call
> PathHelper.TryExpandShortFileName()
> > +       in case the path contains a '~' character.
> > +       We don't need to do this as we don't need to normalize the
> file name
> > +       for presentation, and the extended path prefix works with
> 8.3 path
> > +       components as well */
> > +    return 0;
> > +}
> 
> This function simply forwards the return code of get_full_path_name().
> The only non-trivial part of it is a comment.

I wanted those functions to resemble the handling path from
.NET. 

I had already started the implementation of TryExpandShortFilePath()
as well, but then I figured that this isn't necessary. For this,
I had tested a long path in the form with 8.3 path components (where
even the 8.3 form is longer than 260) and even those paths work 
with the extended prefix. That's why I skipped this part as in our
case it will only be used internally while in .NET it is done
because the GetFullPathMethod() is also used for other purposes
where expansion is desirable.

Of course we could merge some of those functions together, but the
compiler will do the inlining anyway, that's why I chose to keep
the functions separate for better clarity.

 
> > +static inline int path_is_extended(const wchar_t *path)
> > +{
> > +    // see .NET6: PathInternal.IsExtended()
> > +    size_t len = wcslen(path);
> > +    if (len >= 4  && path[0] == L'\\' && (path[1] == L'\\' ||
> path[1] == L'?') && path[2] == L'?' && path[3] == L'\\')
> > +        return 1;
> > +
> > +    return 0;
> > +}
> >
> > +static inline int add_extended_prefix(wchar_t **ppath_w)
> > +{
> > +    const wchar_t *unc_prefix           = L"\\\\?\\UNC\\";
> > ...
> > +    // see .NET6: PathInternal.EnsureExtendedPrefix()
> > +    if (path_w[0] == L'\\' && path_w[1] == L'\\') {
> > ...
> > +        wcscpy(temp_w, unc_prefix);
> > +        wcscat(temp_w, path_w + 2);
> >
> > +static inline int get_extended_win32_path(const char *path, wchar_t
> **ppath_w)
> > +{
> > ...
> > +    if (path_is_extended(*ppath_w)) {
> > +        ...
> > +        return 0;
> > +    }
> > ...
> > +        if ((ret = add_extended_prefix(ppath_w)) < 0)
> 
> Actual PathInternal.EnsureExtendedPrefix
> (https://github.com/dotnet/runtime/blob/main/src/libraries/Common/src/
> System/IO/PathInternal.Windows.cs)
> checks for
> 
>             if (IsPartiallyQualified(path.AsSpan()) ||
> IsDevice(path.AsSpan()))
>                 return path;


> where IsDevice handles \\.\, which you do not handle. If I'm not
> mistaken,
> the code paths presented above will turn such paths into \\?\UNC\\.\,
> which is an error.

I have skipped those checks because we won't have partially qualified
paths at this point (due to having called GetFullPathNameW) and
device paths are not allowed to be longer than 260, so this it might
happen that the UNC prefix gets added, but only when it's a long
path which doesn't work anyway (I've tested those cases).

> > +static inline int add_extended_prefix(wchar_t **pp
ath_w)
> > +{
> > +    const wchar_t *unc_prefix           = L"\\\\?\\UNC\\";
> > ...
> > +        temp_w = (wchar_t *)av_calloc(len + 6 + 1,
> sizeof(wchar_t));
> 
> Wouldn't it be better to use sizeof unc_prefix instead of magic
> numbers?

Then we would need to subtract the terminating zeros and divide
by two => ugly.
Or do another wcslen() call => unnecessary. 

I have added comments right above now, explaining those numbers.

Thanks for your review,
softworkz
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/2] Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Sunday, May 15, 2022 9:36 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH 0/2] Support long file names on
> Windows
> 
> > I have kept the individual functions separate on purpose, to make it
> easy to
> > compare with the .NET impl. (compilers should inlinie those anyway)
> 
> Calling add_extended_prefix without pre-validation results in error,
> since it does check for \\?\, \\.\, or \??\; only it's wrapper
> get_extended_win32_path does.
> And it's not private, it's in the header.


> path_normalize is a do nothing function.
> 
> Keeping the comments about where the code originated from may be
> useful.
> Copying the structure of .NET results in problems.

I can squash them together if that would be a common desire.


> These patches are very difficult to review. E.g. stat is not covered:
> https://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/296448.html.

I'll look into stat in a moment.

> 
> Parts of FFmpeg still use fopen instead of av_fopen_utf8. Some of
> these uses are
> in examples or tests or inside #ifdef DEBUG blocks.
> Others aren't: dvdsubdec.c:620 uses fopen and it is not covered by
> your
> previous patchset https://ffmpeg.org/pipermail/ffmpeg-devel/2022-
> May/296189.html.

I have left those out by intention because they are pending removal
and are only for debugging.


Thanks again,
softworkz
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/2] avformat/os_support: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Sunday, May 15, 2022 9:13 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH 2/2] avformat/os_support: Support
> long file names on Windows
> 
> > diff --git a/libavformat/os_support.h b/libavformat/os_support.h
> 
> In addition to what you've already added, this file defines stat as:
> 
> #ifdef _WIN32
> ...
> #  ifdef stat
> #   undef stat
> #  endif
> #  define stat _stati64
> ...
> 
> which is
> 1. not wide-char aware (_wstati64 does exist)
> 2. not long path aware.

That's a good point, even though plain stat is used in 2 cases only.
We already have win32_stat, but what's a bit tricky is that the 
struct that this function takes as a parameter is named the same 
as the function itself.

Now I have repeated the definition of 'struct stat' as 
'struct win32_stat', but maybe someone can come up with a better way.
Would you have any idea perhaps?


> Also there is a function:
> 
> static inline int is_dos_path(const char *path)
> {
> #if HAVE_DOS_PATHS
>     if (path[0] && path[1] == ':')
>         return 1;
> #endif
>     return 0;
> }
> 
> Now, DOS paths C:... end up being \\?\C:.... Are you sure it won't
> break something down the line?

The prefixing of paths is always done right before calling any of
the Windows APIs. Original strings are never modified like this.
You can be sure, because all those operations are done on
WCHAR strings only.

You could ask the question only about the case where a user might 
supply a path that is prefixed already, but when we look at the 
only(!) usage of this function then it is about handling URL and 
checking URL paths and a path starting with \\?\ wouldn't be
valid in a URL anyway.
So I don't think that this is of any concern, but it was a good
point to verify this.

Thanks for the review!

softworkz

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 0/2] Support long file names on Windows

[ffmpegagent]

This patchset adds support for long file and directory paths on Windows. The
implementation follows the same logic that .NET is using internally, with
the only exception that it doesn't expand short path components in 8.3
format. .NET does this as the same function is also used for other purposes,
but in our case, that's not required. Short (8.3) paths are working as well
with the extended path prefix, even when longer than 260.

Successfully tested:

 * Regular paths wth drive letter
 * Regular UNC paths
 * Long paths wth drive letter
 * Long paths wth drive letter and forward slashes
 * Long UNC paths
 * Prefixed paths wth drive letter
 * Prefixed UNC paths

I have kept the individual functions separate on purpose, to make it easy to
compare with the .NET impl. (compilers should inlinie those anyway)

v2

 * wchar_filename: Improve comments and function documentation
 * os_support: adjust defines to use win32_stat

softworkz (2):
  avutil/wchar_filename,file_open: Support long file names on Windows
  avformat/os_support: Support long file names on Windows

 libavformat/os_support.h   |  26 +++++--
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 156 +++++++++++++++++++++++++++++++++++++
 3 files changed, 178 insertions(+), 6 deletions(-)


base-commit: d2d8b9b972ba2df6b2a2ebe29f5307cbb7a69c33
Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-28%2Fsoftworkz%2Fsubmit_long_filenames-v2
Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v2
Pull-Request: https://github.com/ffstaging/FFmpeg/pull/28

Range-diff vs v1:

 1:  26c579e4ee ! 1:  b66dbdf40c avutil/wchar_filename,file_open: Support long file names on Windows
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
           return 0;
       }
      +
     ++/**
     ++ * Checks for extended path prefixes for which normalization needs to be skipped.
     ++ * see .NET6: PathInternal.IsExtended()
     ++ */
      +static inline int path_is_extended(const wchar_t *path)
      +{
     -+    // see .NET6: PathInternal.IsExtended()
      +    size_t len = wcslen(path);
      +    if (len >= 4  && path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\')
      +        return 1;
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      +    return 0;
      +}
      +
     ++/**
     ++ * Performs path normalization by calling GetFullPathNameW().
     ++ * see .NET6: PathHelper.GetFullPathName()
     ++ */
      +static inline int get_full_path_name(wchar_t **ppath_w)
      +{
      +    int num_chars;
      +    wchar_t *temp_w;
      +
     -+    // see .NET6: PathHelper.GetFullPathName()
      +    num_chars = GetFullPathNameW(*ppath_w, 0, NULL, NULL);
      +    if (num_chars <= 0) {
      +        errno = EINVAL;
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      +    return 0;
      +}
      +
     ++/**
     ++ * Normalizes a Windows file or folder path.
     ++ * Expansion of short paths (with 8.3 path components) is currently omitted
     ++ * as it is not required for accessing long paths.
     ++ * see .NET6: PathHelper.Normalize().
     ++ */
      +static inline int path_normalize(wchar_t **ppath_w)
      +{
      +    int ret;
      +
     -+    // see .NET6: PathHelper.Normalize()
      +    if ((ret = get_full_path_name(ppath_w)) < 0)
      +        return ret;
      +
      +    /* What .NET does at this point is to call PathHelper.TryExpandShortFileName()
     -+       in case the path contains a '~' character.
     -+       We don't need to do this as we don't need to normalize the file name
     -+       for presentation, and the extended path prefix works with 8.3 path
     -+       components as well */
     ++     * in case the path contains a '~' character.
     ++     * We don't need to do this as we don't need to normalize the file name
     ++     * for presentation, and the extended path prefix works with 8.3 path
     ++     * components as well
     ++     */
      +    return 0;
      +}
      +
     ++/**
     ++ * Adds an extended path or UNC prefix to longs paths or paths ending
     ++ * with a space or a dot. (' ' or '.').
     ++ * This function expects that the path has been normalized before by
     ++ * calling path_normalize().
     ++ * see .NET6: PathInternal.EnsureExtendedPrefix() *
     ++ */
      +static inline int add_extended_prefix(wchar_t **ppath_w)
      +{
      +    const wchar_t *unc_prefix           = L"\\\\?\\UNC\\";
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      +    if (len < 2)
      +        return 0;
      +
     -+    // see .NET6: PathInternal.EnsureExtendedPrefix()
     ++    /* We're skipping the check IsPartiallyQualified() because
     ++     * we know we have called GetFullPathNameW() already, also
     ++     * we don't check IsDevice() because device paths are not
     ++     * allowed to be long paths and we're calling this only
     ++     * for long paths.
     ++     */
      +    if (path_w[0] == L'\\' && path_w[1] == L'\\') {
     ++        // The length of unc_prefix is 6 plus 1 for terminating zeros
      +        temp_w = (wchar_t *)av_calloc(len + 6 + 1, sizeof(wchar_t));
      +        if (!temp_w) {
      +            errno = ENOMEM;
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      +        wcscpy(temp_w, unc_prefix);
      +        wcscat(temp_w, path_w + 2);
      +    } else {
     ++        // The length of extended_path_prefix is 4 plus 1 for terminating zeros
      +        temp_w = (wchar_t *)av_calloc(len + 4 + 1, sizeof(wchar_t));
      +        if (!temp_w) {
      +            errno = ENOMEM;
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      +    return 0;
      +}
      +
     ++/**
     ++ * Converts a file or folder path to wchar_t for use with Windows file
     ++ * APIs. Paths with extended path prefix (either '\\?\' or \??\') are
     ++ * left unchanged.
     ++ * All other paths are normalized and converted to absolute paths.
     ++ * Longs paths (>= 260) are prefixed with the extended path or extended
     ++ * UNC path prefix.
     ++ * see .NET6: Path.GetFullPath() and Path.GetFullPathInternal()
     ++ */
      +static inline int get_extended_win32_path(const char *path, wchar_t **ppath_w)
      +{
      +    int ret;
      +    size_t len;
      +
     -+    // see .NET6: Path.GetFullPath() and Path.GetFullPathInternal()
      +    if ((ret = utf8towchar(path, ppath_w)) < 0)
      +        return ret;
      +
      +    if (path_is_extended(*ppath_w)) {
     -+        /* \\?\ paths are considered normalized by definition. Windows doesn't normalize \\?\
     -+           paths and neither should we. Even if we wanted to, GetFullPathName does not work
     -+           properly with device paths. If one wants to pass a \\?\ path through normalization
     -+           one can chop off the prefix, pass it to GetFullPath and add it again. */
     ++        /* Paths prefixed with '\\?\' or \??\' are considered normalized by definition.
     ++         * Windows doesn't normalize those paths and neither should we.
     ++         */
      +        return 0;
      +    }
      +
 2:  acd81c61c3 ! 2:  8ecbafe2b7 avformat/os_support: Support long file names on Windows
     @@ Commit message
          Signed-off-by: softworkz <softworkz@hotmail.com>
      
       ## libavformat/os_support.h ##
     +@@
     + #  ifdef stat
     + #   undef stat
     + #  endif
     +-#  define stat _stati64
     ++#  define stat win32_stat
     ++
     ++    struct win32_stat
     ++    {
     ++        _dev_t         st_dev;
     ++        _ino_t         st_ino;
     ++        unsigned short st_mode;
     ++        short          st_nlink;
     ++        short          st_uid;
     ++        short          st_gid;
     ++        _dev_t         st_rdev;
     ++        __int64        st_size;
     ++        __time64_t     st_atime;
     ++        __time64_t     st_mtime;
     ++        __time64_t     st_ctime;
     ++    };
     ++
     + #  ifdef fstat
     + #   undef fstat
     + #  endif
      @@ libavformat/os_support.h: static inline int win32_##name(const char *filename_utf8) \
           wchar_t *filename_w;                                  \
           int ret;                                              \

-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 156 +++++++++++++++++++++++++++++++++++++
 2 files changed, 157 insertions(+), 1 deletion(-)

diff --git a/libavutil/file_open.c b/libavutil/file_open.c
index cc302f2f76..57c5e78d51 100644
--- a/libavutil/file_open.c
+++ b/libavutil/file_open.c
@@ -45,7 +45,7 @@ static int win32_open(const char *filename_utf8, int oflag, int pmode)
     wchar_t *filename_w;
 
     /* convert UTF-8 to wide chars */
-    if (utf8towchar(filename_utf8, &filename_w))
+    if (get_extended_win32_path(filename_utf8, &filename_w))
         return -1;
     if (!filename_w)
         goto fallback;
diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h
index 90f082452c..78e7dd47d7 100644
--- a/libavutil/wchar_filename.h
+++ b/libavutil/wchar_filename.h
@@ -40,6 +40,162 @@ static inline int utf8towchar(const char *filename_utf8, wchar_t **filename_w)
     MultiByteToWideChar(CP_UTF8, 0, filename_utf8, -1, *filename_w, num_chars);
     return 0;
 }
+
+/**
+ * Checks for extended path prefixes for which normalization needs to be skipped.
+ * see .NET6: PathInternal.IsExtended()
+ */
+static inline int path_is_extended(const wchar_t *path)
+{
+    size_t len = wcslen(path);
+    if (len >= 4  && path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\')
+        return 1;
+
+    return 0;
+}
+
+/**
+ * Performs path normalization by calling GetFullPathNameW().
+ * see .NET6: PathHelper.GetFullPathName()
+ */
+static inline int get_full_path_name(wchar_t **ppath_w)
+{
+    int num_chars;
+    wchar_t *temp_w;
+
+    num_chars = GetFullPathNameW(*ppath_w, 0, NULL, NULL);
+    if (num_chars <= 0) {
+        errno = EINVAL;
+        return -1;
+    }
+
+    temp_w = (wchar_t *)av_calloc(num_chars, sizeof(wchar_t));
+    if (!temp_w) {
+        errno = ENOMEM;
+        return -1;
+    }
+
+    num_chars = GetFullPathNameW(*ppath_w, num_chars, temp_w, NULL);
+    if (num_chars <= 0) {
+        errno = EINVAL;
+        return -1;
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+/**
+ * Normalizes a Windows file or folder path.
+ * Expansion of short paths (with 8.3 path components) is currently omitted
+ * as it is not required for accessing long paths.
+ * see .NET6: PathHelper.Normalize().
+ */
+static inline int path_normalize(wchar_t **ppath_w)
+{
+    int ret;
+
+    if ((ret = get_full_path_name(ppath_w)) < 0)
+        return ret;
+
+    /* What .NET does at this point is to call PathHelper.TryExpandShortFileName()
+     * in case the path contains a '~' character.
+     * We don't need to do this as we don't need to normalize the file name
+     * for presentation, and the extended path prefix works with 8.3 path
+     * components as well
+     */
+    return 0;
+}
+
+/**
+ * Adds an extended path or UNC prefix to longs paths or paths ending
+ * with a space or a dot. (' ' or '.').
+ * This function expects that the path has been normalized before by
+ * calling path_normalize().
+ * see .NET6: PathInternal.EnsureExtendedPrefix() *
+ */
+static inline int add_extended_prefix(wchar_t **ppath_w)
+{
+    const wchar_t *unc_prefix           = L"\\\\?\\UNC\\";
+    const wchar_t *extended_path_prefix = L"\\\\?\\";
+    const wchar_t *path_w               = *ppath_w;
+    const size_t len                    = wcslen(path_w);
+    wchar_t *temp_w;
+
+    if (len < 2)
+        return 0;
+
+    /* We're skipping the check IsPartiallyQualified() because
+     * we know we have called GetFullPathNameW() already, also
+     * we don't check IsDevice() because device paths are not
+     * allowed to be long paths and we're calling this only
+     * for long paths.
+     */
+    if (path_w[0] == L'\\' && path_w[1] == L'\\') {
+        // The length of unc_prefix is 6 plus 1 for terminating zeros
+        temp_w = (wchar_t *)av_calloc(len + 6 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, unc_prefix);
+        wcscat(temp_w, path_w + 2);
+    } else {
+        // The length of extended_path_prefix is 4 plus 1 for terminating zeros
+        temp_w = (wchar_t *)av_calloc(len + 4 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, extended_path_prefix);
+        wcscat(temp_w, path_w);
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+/**
+ * Converts a file or folder path to wchar_t for use with Windows file
+ * APIs. Paths with extended path prefix (either '\\?\' or \??\') are
+ * left unchanged.
+ * All other paths are normalized and converted to absolute paths.
+ * Longs paths (>= 260) are prefixed with the extended path or extended
+ * UNC path prefix.
+ * see .NET6: Path.GetFullPath() and Path.GetFullPathInternal()
+ */
+static inline int get_extended_win32_path(const char *path, wchar_t **ppath_w)
+{
+    int ret;
+    size_t len;
+
+    if ((ret = utf8towchar(path, ppath_w)) < 0)
+        return ret;
+
+    if (path_is_extended(*ppath_w)) {
+        /* Paths prefixed with '\\?\' or \??\' are considered normalized by definition.
+         * Windows doesn't normalize those paths and neither should we.
+         */
+        return 0;
+    }
+
+    if ((ret = path_normalize(ppath_w)) < 0)
+        return ret;
+
+    // see .NET6: PathInternal.EnsureExtendedPrefixIfNeeded()
+    len = wcslen(*ppath_w);
+    if (len >= 260 || (*ppath_w)[len - 1] == L' ' || (*ppath_w)[len - 1] == L'.') {
+        if ((ret = add_extended_prefix(ppath_w)) < 0)
+            return ret;
+    }
+
+    return 0;
+}
+
 #endif
 
 #endif /* AVUTIL_WCHAR_FILENAME_H */
-- 
ffmpeg-codebot

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 2/2] avformat/os_support: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavformat/os_support.h | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)

diff --git a/libavformat/os_support.h b/libavformat/os_support.h
index 5e6b32d2dc..bd8c89568f 100644
--- a/libavformat/os_support.h
+++ b/libavformat/os_support.h
@@ -49,7 +49,23 @@
 #  ifdef stat
 #   undef stat
 #  endif
-#  define stat _stati64
+#  define stat win32_stat
+
+    struct win32_stat
+    {
+        _dev_t         st_dev;
+        _ino_t         st_ino;
+        unsigned short st_mode;
+        short          st_nlink;
+        short          st_uid;
+        short          st_gid;
+        _dev_t         st_rdev;
+        __int64        st_size;
+        __time64_t     st_atime;
+        __time64_t     st_mtime;
+        __time64_t     st_ctime;
+    };
+
 #  ifdef fstat
 #   undef fstat
 #  endif
@@ -153,7 +169,7 @@ static inline int win32_##name(const char *filename_utf8) \
     wchar_t *filename_w;                                  \
     int ret;                                              \
                                                           \
-    if (utf8towchar(filename_utf8, &filename_w))          \
+    if (get_extended_win32_path(filename_utf8, &filename_w)) \
         return -1;                                        \
     if (!filename_w)                                      \
         goto fallback;                                    \
@@ -177,7 +193,7 @@ static inline int win32_##name(const char *filename_utf8, partype par) \
     wchar_t *filename_w;                                  \
     int ret;                                              \
                                                           \
-    if (utf8towchar(filename_utf8, &filename_w))          \
+    if (get_extended_win32_path(filename_utf8, &filename_w)) \
         return -1;                                        \
     if (!filename_w)                                      \
         goto fallback;                                    \
@@ -199,9 +215,9 @@ static inline int win32_rename(const char *src_utf8, const char *dest_utf8)
     wchar_t *src_w, *dest_w;
     int ret;
 
-    if (utf8towchar(src_utf8, &src_w))
+    if (get_extended_win32_path(src_utf8, &src_w))
         return -1;
-    if (utf8towchar(dest_utf8, &dest_w)) {
+    if (get_extended_win32_path(dest_utf8, &dest_w)) {
         av_free(src_w);
         return -1;
     }
-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[nil-admirari]

> +static inline int path_is_extended(const wchar_t *path)
> +{
> + size_t len = wcslen(path);
> + if (len >= 4 && path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\')

Length check is probably unnecessary: comparisons will reject '\0'
and further comparisons won't run due to short-circuiting.

> + // The length of unc_prefix is 6 plus 1 for terminating zeros
> + temp_w = (wchar_t *)av_calloc(len + 6 + 1, sizeof(wchar_t));

Not really true. The length of unc_prefix is 8.
2 is subtracted because UNC path already has \\ at the beginning.

> + if (len >= 260 || (*ppath_w)[len - 1] == L' ' || (*ppath_w)[len - 1] == L'.') {

1. Please change 260 to MAX_PATH.
2. GetFullPathName removes trailing spaces and dots, so the second part is always false.



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/2] avformat/os_support: Support long file names on Windows

[nil-admirari]

> We already have win32_stat, but what's a bit tricky is that the 
> struct that this function takes as a parameter is named the same 
> as the function itself.

Sorry, I thought is was a definition of a function, not a struct.
Since stat function is already defined as win32_stat,
It's better to revert the changes.



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[nil-admirari]

> The functions are needed in both. file_open.c cannot be included
> in libavformat/os_support.h and neither the other way round, 
> so they need to be in a 3rd place. How about renaming
> wchar_filename.h to windows_filename.h ?

Probably it's better to rename.

> I have skipped those checks because we won't have partially qualified
> paths at this point (due to having called GetFullPathNameW) and
> device paths are not allowed to be longer than 260, so this it might
> happen that the UNC prefix gets added, but only when it's a long
> path which doesn't work anyway (I've tested those cases).

I think it's better to test for \\.\ explicitly in path_is_extended:
1. It's not obvious that \\.\ aren't allowed to be long.
2. Probably FFmpeg is not going to have a longPathAware manifest,
   but it can be linked with an EXE with such a manifest.
   Would MAX_PATH restriction still apply?

You have the checks inside of get_extended_win32_path and none
inside of add_extended_prefix. Yet add_extended_prefix can be called
by anyone: it's not private. Thus add_extended_prefix either should be inlined,
or it should have the necessary checks in place. Otherwise you end up with
an API that's easy to use incorrectly and hard to use correctly, and it should be
the other way around.



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/2] Support long file names on Windows

[nil-admirari]

> I have left those out by intention because they are pending removal
> and are only for debugging.

Is dvdsubdec.c parse_ifo_palette pending removal? What about
- vf_pnsr.c init()
- vf_vidstabdetect.c config_input()
- vf_vidstabtransform.c config_input()?



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Monday, May 16, 2022 10:12 AM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename,
> file_open: Support long file names on Windows
> 
> > +static inline int path_is_extended(const wchar_t *path)
> > +{
> > + size_t len = wcslen(path);
> > + if (len >= 4 && path[0] == L'\\' && (path[1] == L'\\' || path[1]
> == L'?') && path[2] == L'?' && path[3] == L'\\')
> 
> Length check is probably unnecessary: comparisons will reject '\0'
> and further comparisons won't run due to short-circuiting.

Yup, I think you're right, even though it would appear "unsafe" at
first sight. I've removed the length check.


> > + // The length of unc_prefix is 6 plus 1 for terminating zeros
> > + temp_w = (wchar_t *)av_calloc(len + 6 + 1, sizeof(wchar_t));
> 
> Not really true. The length of unc_prefix is 8.
> 2 is subtracted because UNC path already has \\ at the beginning.

Correct. It actually needs to be "len - 2 + 8 + 1".
I've updated the comment and the calculation.


> > + if (len >= 260 || (*ppath_w)[len - 1] == L' ' || (*ppath_w)[len -
> 1] == L'.') {
> 
> 1. Please change 260 to MAX_PATH.

Done.


> 2. GetFullPathName removes trailing spaces and dots, so the second
> part is always false.

Yea, when someone would want to handle such (weird) kind of path,
one would need to specify an extended path directly.
Removed the second part.


> > We already have win32_stat, but what's a bit tricky is that the
> > struct that this function takes as a parameter is named the same
> > as the function itself.
> 
> Sorry, I thought is was a definition of a function, not a struct.
> Since stat function is already defined as win32_stat,
> It's better to revert the changes.

stat wasn't already defined as win32_stat.
win32_stat was already defined but not mapped. That's what my change
does. 


> > The functions are needed in both. file_open.c cannot be included
> > in libavformat/os_support.h and neither the other way round,
> > so they need to be in a 3rd place. How about renaming
> > wchar_filename.h to windows_filename.h ?
> 
> Probably it's better to rename.

OK, but let's do this in subsequent patch shortly after.


> > I have skipped those checks because we won't have partially
> qualified
> > paths at this point (due to having called GetFullPathNameW) and
> > device paths are not allowed to be longer than 260, so this it might
> > happen that the UNC prefix gets added, but only when it's a long
> > path which doesn't work anyway (I've tested those cases).
> 
> I think it's better to test for \\.\ explicitly in path_is_extended:
> 1. It's not obvious that \\.\ aren't allowed to be long.
> 2. Probably FFmpeg is not going to have a longPathAware manifest,
>    but it can be linked with an EXE with such a manifest.
>    Would MAX_PATH restriction still apply?

That's a good question, but we need to be clear that device paths are
actually intended for accessing devices, e.g. like \\.\COM1
The fact that the prefix also works with a drive-letter path
is more due to some heritage and nobody would normally want to 
use such kind of paths to access files.

That being said, I've added a check (path_is_device_path()) for 
this now in the same way as .NET is doing it - which means
inside add_extended_prefix().


> You have the checks inside of get_extended_win32_path and none
> inside of add_extended_prefix. Yet add_extended_prefix can be called
> by anyone: it's not private. Thus add_extended_prefix either should be
> inlined,
> or it should have the necessary checks in place. Otherwise you end up
> with
> an API that's easy to use incorrectly and hard to use correctly, and
> it should be the other way around.

I have added checks there now for both device path and extended path
prefix. I have also clarified the function doc even further, so I 
hope it's sufficiently clear now. ;-)


> > And what's the point about this?
> 
> Point is obvious: extended paths are difficult to handle correctly.
> get_extended_win32_path cannot be used on its own, only as a final
> step before getting FILE* or a file descriptor.

Yes, that's how it's meant to be used, so the whole application can
be kept free from dealing with it.


Thanks again for the review, these were some good improvements.

Kind regards,
softworkz




_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v3 0/2] Support long file names on Windows

[ffmpegagent]

This patchset adds support for long file and directory paths on Windows. The
implementation follows the same logic that .NET is using internally, with
the only exception that it doesn't expand short path components in 8.3
format. .NET does this as the same function is also used for other purposes,
but in our case, that's not required. Short (8.3) paths are working as well
with the extended path prefix, even when longer than 260.

Successfully tested:

 * Regular paths wth drive letter
 * Regular UNC paths
 * Long paths wth drive letter
 * Long paths wth drive letter and forward slashes
 * Long UNC paths
 * Prefixed paths wth drive letter
 * Prefixed UNC paths

I have kept the individual functions separate on purpose, to make it easy to
compare with the .NET impl. (compilers should inlinie those anyway)

v2

 * wchar_filename: Improve comments and function documentation
 * os_support: adjust defines to use win32_stat

v3

 * removed length check in path_is_extended()
 * added path_is_device_path() check in add_extended_prefix()
 * add_extended_prefix(): clarified doc and add checks
 * clarified string allocation length calculation
 * replaced 260 with MAX_PATH
 * removed redundant checks after normalization

softworkz (2):
  avutil/wchar_filename,file_open: Support long file names on Windows
  avformat/os_support: Support long file names on Windows

 libavformat/os_support.h   |  26 ++++--
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 166 +++++++++++++++++++++++++++++++++++++
 3 files changed, 188 insertions(+), 6 deletions(-)


base-commit: e3580f60775c897c3b13b178c57ab191ecc4a031
Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-28%2Fsoftworkz%2Fsubmit_long_filenames-v3
Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v3
Pull-Request: https://github.com/ffstaging/FFmpeg/pull/28

Range-diff vs v2:

 1:  b66dbdf40c ! 1:  ce70f7b021 avutil/wchar_filename,file_open: Support long file names on Windows
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      + */
      +static inline int path_is_extended(const wchar_t *path)
      +{
     -+    size_t len = wcslen(path);
     -+    if (len >= 4  && path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\')
     ++    if (path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\')
     ++        return 1;
     ++
     ++    return 0;
     ++}
     ++
     ++/**
     ++ * Checks for a device path prefix.
     ++ * see .NET6: PathInternal.IsDevicePath()
     ++ */
     ++static inline int path_is_device_path(const wchar_t *path)
     ++{
     ++    if (path[0] == L'\\' && path[1] == L'\\' && path[2] == L'.' && path[3] == L'\\')
      +        return 1;
      +
      +    return 0;
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      + * Adds an extended path or UNC prefix to longs paths or paths ending
      + * with a space or a dot. (' ' or '.').
      + * This function expects that the path has been normalized before by
     -+ * calling path_normalize().
     ++ * calling path_normalize() and it doesn't check whether the path is
     ++ * actually long (> MAX_PATH).
      + * see .NET6: PathInternal.EnsureExtendedPrefix() *
      + */
      +static inline int add_extended_prefix(wchar_t **ppath_w)
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      +    const size_t len                    = wcslen(path_w);
      +    wchar_t *temp_w;
      +
     -+    if (len < 2)
     ++    /* We're skipping the check IsPartiallyQualified() because
     ++     * we expect to have called GetFullPathNameW() already. */
     ++    if (len < 2 || path_is_extended(*ppath_w) || path_is_device_path(*ppath_w)) {
      +        return 0;
     ++    }
      +
     -+    /* We're skipping the check IsPartiallyQualified() because
     -+     * we know we have called GetFullPathNameW() already, also
     -+     * we don't check IsDevice() because device paths are not
     -+     * allowed to be long paths and we're calling this only
     -+     * for long paths.
     -+     */
      +    if (path_w[0] == L'\\' && path_w[1] == L'\\') {
     -+        // The length of unc_prefix is 6 plus 1 for terminating zeros
     -+        temp_w = (wchar_t *)av_calloc(len + 6 + 1, sizeof(wchar_t));
     ++        /* unc_prefix length is 8 plus 1 for terminating zeros,
     ++         * we subtract 2 for the leading '\\' of the original path */
     ++        temp_w = (wchar_t *)av_calloc(len - 2 + 8 + 1, sizeof(wchar_t));
      +        if (!temp_w) {
      +            errno = ENOMEM;
      +            return -1;
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      + * APIs. Paths with extended path prefix (either '\\?\' or \??\') are
      + * left unchanged.
      + * All other paths are normalized and converted to absolute paths.
     -+ * Longs paths (>= 260) are prefixed with the extended path or extended
     ++ * Longs paths (>= MAX_PATH) are prefixed with the extended path or extended
      + * UNC path prefix.
      + * see .NET6: Path.GetFullPath() and Path.GetFullPathInternal()
      + */
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      +
      +    // see .NET6: PathInternal.EnsureExtendedPrefixIfNeeded()
      +    len = wcslen(*ppath_w);
     -+    if (len >= 260 || (*ppath_w)[len - 1] == L' ' || (*ppath_w)[len - 1] == L'.') {
     ++    if (len >= MAX_PATH) {
      +        if ((ret = add_extended_prefix(ppath_w)) < 0)
      +            return ret;
      +    }
 2:  8ecbafe2b7 = 2:  a5268800a4 avformat/os_support: Support long file names on Windows

-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v3 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 166 +++++++++++++++++++++++++++++++++++++
 2 files changed, 167 insertions(+), 1 deletion(-)

diff --git a/libavutil/file_open.c b/libavutil/file_open.c
index cc302f2f76..57c5e78d51 100644
--- a/libavutil/file_open.c
+++ b/libavutil/file_open.c
@@ -45,7 +45,7 @@ static int win32_open(const char *filename_utf8, int oflag, int pmode)
     wchar_t *filename_w;
 
     /* convert UTF-8 to wide chars */
-    if (utf8towchar(filename_utf8, &filename_w))
+    if (get_extended_win32_path(filename_utf8, &filename_w))
         return -1;
     if (!filename_w)
         goto fallback;
diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h
index 90f082452c..94f8ce54b5 100644
--- a/libavutil/wchar_filename.h
+++ b/libavutil/wchar_filename.h
@@ -40,6 +40,172 @@ static inline int utf8towchar(const char *filename_utf8, wchar_t **filename_w)
     MultiByteToWideChar(CP_UTF8, 0, filename_utf8, -1, *filename_w, num_chars);
     return 0;
 }
+
+/**
+ * Checks for extended path prefixes for which normalization needs to be skipped.
+ * see .NET6: PathInternal.IsExtended()
+ */
+static inline int path_is_extended(const wchar_t *path)
+{
+    if (path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\')
+        return 1;
+
+    return 0;
+}
+
+/**
+ * Checks for a device path prefix.
+ * see .NET6: PathInternal.IsDevicePath()
+ */
+static inline int path_is_device_path(const wchar_t *path)
+{
+    if (path[0] == L'\\' && path[1] == L'\\' && path[2] == L'.' && path[3] == L'\\')
+        return 1;
+
+    return 0;
+}
+
+/**
+ * Performs path normalization by calling GetFullPathNameW().
+ * see .NET6: PathHelper.GetFullPathName()
+ */
+static inline int get_full_path_name(wchar_t **ppath_w)
+{
+    int num_chars;
+    wchar_t *temp_w;
+
+    num_chars = GetFullPathNameW(*ppath_w, 0, NULL, NULL);
+    if (num_chars <= 0) {
+        errno = EINVAL;
+        return -1;
+    }
+
+    temp_w = (wchar_t *)av_calloc(num_chars, sizeof(wchar_t));
+    if (!temp_w) {
+        errno = ENOMEM;
+        return -1;
+    }
+
+    num_chars = GetFullPathNameW(*ppath_w, num_chars, temp_w, NULL);
+    if (num_chars <= 0) {
+        errno = EINVAL;
+        return -1;
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+/**
+ * Normalizes a Windows file or folder path.
+ * Expansion of short paths (with 8.3 path components) is currently omitted
+ * as it is not required for accessing long paths.
+ * see .NET6: PathHelper.Normalize().
+ */
+static inline int path_normalize(wchar_t **ppath_w)
+{
+    int ret;
+
+    if ((ret = get_full_path_name(ppath_w)) < 0)
+        return ret;
+
+    /* What .NET does at this point is to call PathHelper.TryExpandShortFileName()
+     * in case the path contains a '~' character.
+     * We don't need to do this as we don't need to normalize the file name
+     * for presentation, and the extended path prefix works with 8.3 path
+     * components as well
+     */
+    return 0;
+}
+
+/**
+ * Adds an extended path or UNC prefix to longs paths or paths ending
+ * with a space or a dot. (' ' or '.').
+ * This function expects that the path has been normalized before by
+ * calling path_normalize() and it doesn't check whether the path is
+ * actually long (> MAX_PATH).
+ * see .NET6: PathInternal.EnsureExtendedPrefix() *
+ */
+static inline int add_extended_prefix(wchar_t **ppath_w)
+{
+    const wchar_t *unc_prefix           = L"\\\\?\\UNC\\";
+    const wchar_t *extended_path_prefix = L"\\\\?\\";
+    const wchar_t *path_w               = *ppath_w;
+    const size_t len                    = wcslen(path_w);
+    wchar_t *temp_w;
+
+    /* We're skipping the check IsPartiallyQualified() because
+     * we expect to have called GetFullPathNameW() already. */
+    if (len < 2 || path_is_extended(*ppath_w) || path_is_device_path(*ppath_w)) {
+        return 0;
+    }
+
+    if (path_w[0] == L'\\' && path_w[1] == L'\\') {
+        /* unc_prefix length is 8 plus 1 for terminating zeros,
+         * we subtract 2 for the leading '\\' of the original path */
+        temp_w = (wchar_t *)av_calloc(len - 2 + 8 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, unc_prefix);
+        wcscat(temp_w, path_w + 2);
+    } else {
+        // The length of extended_path_prefix is 4 plus 1 for terminating zeros
+        temp_w = (wchar_t *)av_calloc(len + 4 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, extended_path_prefix);
+        wcscat(temp_w, path_w);
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+/**
+ * Converts a file or folder path to wchar_t for use with Windows file
+ * APIs. Paths with extended path prefix (either '\\?\' or \??\') are
+ * left unchanged.
+ * All other paths are normalized and converted to absolute paths.
+ * Longs paths (>= MAX_PATH) are prefixed with the extended path or extended
+ * UNC path prefix.
+ * see .NET6: Path.GetFullPath() and Path.GetFullPathInternal()
+ */
+static inline int get_extended_win32_path(const char *path, wchar_t **ppath_w)
+{
+    int ret;
+    size_t len;
+
+    if ((ret = utf8towchar(path, ppath_w)) < 0)
+        return ret;
+
+    if (path_is_extended(*ppath_w)) {
+        /* Paths prefixed with '\\?\' or \??\' are considered normalized by definition.
+         * Windows doesn't normalize those paths and neither should we.
+         */
+        return 0;
+    }
+
+    if ((ret = path_normalize(ppath_w)) < 0)
+        return ret;
+
+    // see .NET6: PathInternal.EnsureExtendedPrefixIfNeeded()
+    len = wcslen(*ppath_w);
+    if (len >= MAX_PATH) {
+        if ((ret = add_extended_prefix(ppath_w)) < 0)
+            return ret;
+    }
+
+    return 0;
+}
+
 #endif
 
 #endif /* AVUTIL_WCHAR_FILENAME_H */
-- 
ffmpeg-codebot

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v3 2/2] avformat/os_support: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavformat/os_support.h | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)

diff --git a/libavformat/os_support.h b/libavformat/os_support.h
index 5e6b32d2dc..bd8c89568f 100644
--- a/libavformat/os_support.h
+++ b/libavformat/os_support.h
@@ -49,7 +49,23 @@
 #  ifdef stat
 #   undef stat
 #  endif
-#  define stat _stati64
+#  define stat win32_stat
+
+    struct win32_stat
+    {
+        _dev_t         st_dev;
+        _ino_t         st_ino;
+        unsigned short st_mode;
+        short          st_nlink;
+        short          st_uid;
+        short          st_gid;
+        _dev_t         st_rdev;
+        __int64        st_size;
+        __time64_t     st_atime;
+        __time64_t     st_mtime;
+        __time64_t     st_ctime;
+    };
+
 #  ifdef fstat
 #   undef fstat
 #  endif
@@ -153,7 +169,7 @@ static inline int win32_##name(const char *filename_utf8) \
     wchar_t *filename_w;                                  \
     int ret;                                              \
                                                           \
-    if (utf8towchar(filename_utf8, &filename_w))          \
+    if (get_extended_win32_path(filename_utf8, &filename_w)) \
         return -1;                                        \
     if (!filename_w)                                      \
         goto fallback;                                    \
@@ -177,7 +193,7 @@ static inline int win32_##name(const char *filename_utf8, partype par) \
     wchar_t *filename_w;                                  \
     int ret;                                              \
                                                           \
-    if (utf8towchar(filename_utf8, &filename_w))          \
+    if (get_extended_win32_path(filename_utf8, &filename_w)) \
         return -1;                                        \
     if (!filename_w)                                      \
         goto fallback;                                    \
@@ -199,9 +215,9 @@ static inline int win32_rename(const char *src_utf8, const char *dest_utf8)
     wchar_t *src_w, *dest_w;
     int ret;
 
-    if (utf8towchar(src_utf8, &src_w))
+    if (get_extended_win32_path(src_utf8, &src_w))
         return -1;
-    if (utf8towchar(dest_utf8, &dest_w)) {
+    if (get_extended_win32_path(dest_utf8, &dest_w)) {
         av_free(src_w);
         return -1;
     }
-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/2] Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Monday, May 16, 2022 10:45 AM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH 0/2] Support long file names on
> Windows
> 
> > I have left those out by intention because they are pending removal
> > and are only for debugging.
> 
> Is dvdsubdec.c parse_ifo_palette pending removal?

Just the debug part. Seems I missed the other one.

> - vf_pnsr.c init()
> - vf_vidstabdetect.c config_input()
> - vf_vidstabtransform.c config_input()?

I don't have the latter two included due to missing
lib. No idea how I have missed the other two.

Thanks for pointing out. Update will follow.

Kind regards,
softworkz
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[nil-admirari]

> stat wasn't already defined as win32_stat.
> win32_stat was already defined but not mapped. That's what my change
> does. 

There are two defines in os_support.h:

#  ifdef stat
#   undef stat
#  endif
#  define stat _stati64

and

DEF_FS_FUNCTION2(stat, _wstati64, _stati64, struct stat*)

which defines win32_stat (not stat). This function takes struct stat*, which due to previous define
expands into struct _stati64*.

_stati64 and _wstati64 both take struct _stati64*, which is named identically to the first function.
struct _stati64 expands into different structs depending on the value of _USE_32BIT_TIME_T,
which your explicit structure definition does not capture, see:
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/stat-functions?view=msvc-170.
If someone defines_USE_32BIT_TIME_T, your code will fail to compile.

C allows functions and structs to have identical names, preprocessor does not;
therefore win32_stat must be used explicitly where stat is required as in file.c:160

    struct stat st; // expands into struct _stati64 on Windows.
#   ifndef _WIN32
    ret = stat(filename, &st);
#   else
    ret = win32_stat(filename, &st);
#   endif

However, no everyone follows: img2dec.c:504 and ipfsgateway.c:104 use plain stat.

if (stat(filename, &img_stat)) {
stat_ret = stat(ipfs_full_data_folder, &st);

In these files, on Windows, both the struct and the function call expand into _stati64,
and this time the function call bypasses the UTF-8 to wchar conversion.

Apparently yet another macro is necessary:

#ifdef _WIN32
#define ff_stat win32_stat
#else
#define ff_stat stat
#endif



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Tuesday, May 17, 2022 5:07 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename,
> file_open: Support long file names on Windows
> 
> > stat wasn't already defined as win32_stat.
> > win32_stat was already defined but not mapped. That's what my change
> > does.
> 
> There are two defines in os_support.h:
> 
> #  ifdef stat
> #   undef stat
> #  endif
> #  define stat _stati64
> 
> and
> 
> DEF_FS_FUNCTION2(stat, _wstati64, _stati64, struct stat*)
> 
> which defines win32_stat (not stat). This function takes struct stat*,
> which due to previous define
> expands into struct _stati64*.
> 
> _stati64 and _wstati64 both take struct _stati64*, which is named
> identically to the first function.
> struct _stati64 expands into different structs depending on the value
> of _USE_32BIT_TIME_T,
> which your explicit structure definition does not capture, see:
> https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/stat-
> functions?view=msvc-170.
> If someone defines_USE_32BIT_TIME_T, your code will fail to compile.

Yes, that's true. But there are hundreds of other things someone could
define which makes compilation fail. 
We don't need to accommodate for every single possibility, and it's not
that _USE_32BIT_TIME_T would be required or the default for 32bit 
compilation.


> C allows functions and structs to have identical names, preprocessor
> does not;
> therefore win32_stat must be used explicitly where stat is required as
> in file.c:160

Except when you define a compatible struct with the same name as the
function - like I did.


>     struct stat st; // expands into struct _stati64 on Windows.
> #   ifndef _WIN32
>     ret = stat(filename, &st);
> #   else
>     ret = win32_stat(filename, &st);
> #   endif

This could be removed after the patch.


> However, no everyone follows: img2dec.c:504 and ipfsgateway.c:104 use
> plain stat.
> 
> if (stat(filename, &img_stat)) {
> stat_ret = stat(ipfs_full_data_folder, &st);
> 
> In these files, on Windows, both the struct and the function call
> expand into _stati64,
> and this time the function call bypasses the UTF-8 to wchar
> conversion.
> 
> Apparently yet another macro is necessary:
> 
> #ifdef _WIN32
> #define ff_stat win32_stat
> #else
> #define ff_stat stat
> #endif

Probably you didn't spot it. It's already there:

#  define stat win32_stat

Kind regards,
softworkz

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Soft
> Works
> Sent: Tuesday, May 17, 2022 5:28 PM
> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename,
> file_open: Support long file names on Windows
> 
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> nil-
> > admirari@mailo.com
> > Sent: Tuesday, May 17, 2022 5:07 PM
> > To: ffmpeg-devel@ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename,
> > file_open: Support long file names on Windows
> >
> > > stat wasn't already defined as win32_stat.
> > > win32_stat was already defined but not mapped. That's what my
> change
> > > does.
> >
> > There are two defines in os_support.h:
> >
> > #  ifdef stat
> > #   undef stat
> > #  endif
> > #  define stat _stati64
> >
> > and
> >
> > DEF_FS_FUNCTION2(stat, _wstati64, _stati64, struct stat*)
> >
> > which defines win32_stat (not stat). This function takes struct
> stat*,
> > which due to previous define
> > expands into struct _stati64*.
> >
> > _stati64 and _wstati64 both take struct _stati64*, which is named
> > identically to the first function.
> > struct _stati64 expands into different structs depending on the
> value
> > of _USE_32BIT_TIME_T,
> > which your explicit structure definition does not capture, see:
> > https://docs.microsoft.com/en-us/cpp/c-runtime-
> library/reference/stat-
> > functions?view=msvc-170.
> > If someone defines_USE_32BIT_TIME_T, your code will fail to compile.
> 
> Yes, that's true. But there are hundreds of other things someone could
> define which makes compilation fail.
> We don't need to accommodate for every single possibility, and it's
> not
> that _USE_32BIT_TIME_T would be required or the default for 32bit
> compilation.
> 
> 
> > C allows functions and structs to have identical names, preprocessor
> > does not;
> > therefore win32_stat must be used explicitly where stat is required
> as
> > in file.c:160
> 
> Except when you define a compatible struct with the same name as the
> function - like I did.

I don't want to say that I'd consider this to be a great solution.
But the problem is that the function and struct names are identical
and when we want to re-define/map the function, we also need to
provide a struct of that name because the macro-replacement can't
work selectively.

Maybe there is some cool trick to handle this, yet I haven't had
a better idea so far.

Thanks,
softworkz


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[nil-admirari]

> Yes, that's true. But there are hundreds of other things someone could
> define which makes compilation fail.

Doesn't mean that yet another such thing should be added by incorrectly
redefining structs already defined correctly by system headers.

> Probably you didn't spot it. It's already there:
> # define stat win32_stat

I'm actually wondering how does it even compile. All stat structs in code
become struct win32_stat, and all calls to stat become calls to win32_stat,
which in turn wraps _wstati64. But _wstati64 does not accept struct win32_stat*,
it accepts struct _stati64*. Content of these structs is probably identical, but
it should not matter: C is typed nominally, not structurally.

> I don't want to say that I'd consider this to be a great solution.
> But the problem is that the function and struct names are identical
> and when we want to re-define/map the function, we also need to
> provide a struct of that name because the macro-replacement can't
> work selectively.

Doesn't mean that the should be named identically in FFmpeg code.
Naming a struct stat and a function avpriv_stat is a reasonable choice.
You can even define avpriv_stat with parameters the way fstat is defined:

#ifdef _WIN32
#define avpriv_stat(f,s) win32_stat((f), (s))
#else
#define avpriv_stat(f,s) stat((f), (s))
#endif



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Friday, May 20, 2022 7:52 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename,
> file_open: Support long file names on Windows
> 
> > Yes, that's true. But there are hundreds of other things someone could
> > define which makes compilation fail.
> 
> Doesn't mean that yet another such thing should be added by incorrectly
> redefining structs already defined correctly by system headers.
> 
> > Probably you didn't spot it. It's already there:
> > # define stat win32_stat
> 
> I'm actually wondering how does it even compile. All stat structs in code
> become struct win32_stat, and all calls to stat become calls to
> win32_stat,
> which in turn wraps _wstati64. But _wstati64 does not accept struct
> win32_stat*,
> it accepts struct _stati64*. Content of these structs is probably
> identical, but
> it should not matter: C is typed nominally, not structurally.
> 
> > I don't want to say that I'd consider this to be a great solution.
> > But the problem is that the function and struct names are identical
> > and when we want to re-define/map the function, we also need to
> > provide a struct of that name because the macro-replacement can't
> > work selectively.
> 
> Doesn't mean that the should be named identically in FFmpeg code.
> Naming a struct stat and a function avpriv_stat is a reasonable choice.
> You can even define avpriv_stat with parameters the way fstat is defined:
> 
> #ifdef _WIN32
> #define avpriv_stat(f,s) win32_stat((f), (s))
> #else
> #define avpriv_stat(f,s) stat((f), (s))
> #endif

I thought the purpose of all those re-mappings would be that plain
Posix functions can still be used..?
It's already the Posix declaration where the function name 
is the same as the structure name (stat).

I'm unbiased. avpriv_stat() would be ok from my POV as well, but 
I'm not sure whether all others would agree when plain stat could
no longer be used (without breaking long filename support)?

Thanks,
softworkz


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[nil-admirari]

> I thought the purpose of all those re-mappings would be that plain
> Posix functions can still be used..?
> It's already the Posix declaration where the function name 
> is the same as the structure name (stat).

Not possible for stat precisely because of function and struct sharing a name.
That's why stat is used as

# ifndef _WIN32
ret = stat(filename, &st);
# else
ret = win32_stat(filename, &st);
# endif

Such blocks are either to be copied across the application,
or hidden behind yet another macro.



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Saturday, May 21, 2022 1:08 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename,
> file_open: Support long file names on Windows
> 
> > I thought the purpose of all those re-mappings would be that plain
> > Posix functions can still be used..?
> > It's already the Posix declaration where the function name
> > is the same as the structure name (stat).
> 
> Not possible for stat precisely because of function and struct sharing a
> name.

That's exactly what is said above and before :-)

> That's why stat is used as
> 
> # ifndef _WIN32
> ret = stat(filename, &st);
> # else
> ret = win32_stat(filename, &st);
> # endif
> 
> Such blocks are either to be copied across the application,
> or hidden behind yet another macro.

The latter is what my patchset already does.


Kind regards,
sw
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 0/2] Support long file names on Windows

[ffmpegagent]

This patchset adds support for long file and directory paths on Windows. The
implementation follows the same logic that .NET is using internally, with
the only exception that it doesn't expand short path components in 8.3
format. .NET does this as the same function is also used for other purposes,
but in our case, that's not required. Short (8.3) paths are working as well
with the extended path prefix, even when longer than 260.

Successfully tested:

 * Regular paths wth drive letter
 * Regular UNC paths
 * Long paths wth drive letter
 * Long paths wth drive letter and forward slashes
 * Long UNC paths
 * Prefixed paths wth drive letter
 * Prefixed UNC paths

I have kept the individual functions separate on purpose, to make it easy to
compare with the .NET impl. (compilers should inlinie those anyway)

v2

 * wchar_filename: Improve comments and function documentation
 * os_support: adjust defines to use win32_stat

v3

 * removed length check in path_is_extended()
 * added path_is_device_path() check in add_extended_prefix()
 * add_extended_prefix(): clarified doc and add checks
 * clarified string allocation length calculation
 * replaced 260 with MAX_PATH
 * removed redundant checks after normalization

v4

 * rebased. no changes

softworkz (2):
  avutil/wchar_filename,file_open: Support long file names on Windows
  avformat/os_support: Support long file names on Windows

 libavformat/os_support.h   |  26 ++++--
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 166 +++++++++++++++++++++++++++++++++++++
 3 files changed, 188 insertions(+), 6 deletions(-)


base-commit: 6076dbcb55d0c9b6693d1acad12a63f7268301aa
Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-28%2Fsoftworkz%2Fsubmit_long_filenames-v4
Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v4
Pull-Request: https://github.com/ffstaging/FFmpeg/pull/28

Range-diff vs v3:

 1:  ce70f7b021 = 1:  13118dc1fa avutil/wchar_filename,file_open: Support long file names on Windows
 2:  a5268800a4 = 2:  252ed89499 avformat/os_support: Support long file names on Windows

-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 166 +++++++++++++++++++++++++++++++++++++
 2 files changed, 167 insertions(+), 1 deletion(-)

diff --git a/libavutil/file_open.c b/libavutil/file_open.c
index fb64c2e4ee..58a6073353 100644
--- a/libavutil/file_open.c
+++ b/libavutil/file_open.c
@@ -45,7 +45,7 @@ static int win32_open(const char *filename_utf8, int oflag, int pmode)
     wchar_t *filename_w;
 
     /* convert UTF-8 to wide chars */
-    if (utf8towchar(filename_utf8, &filename_w))
+    if (get_extended_win32_path(filename_utf8, &filename_w))
         return -1;
     if (!filename_w)
         goto fallback;
diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h
index 90f082452c..94f8ce54b5 100644
--- a/libavutil/wchar_filename.h
+++ b/libavutil/wchar_filename.h
@@ -40,6 +40,172 @@ static inline int utf8towchar(const char *filename_utf8, wchar_t **filename_w)
     MultiByteToWideChar(CP_UTF8, 0, filename_utf8, -1, *filename_w, num_chars);
     return 0;
 }
+
+/**
+ * Checks for extended path prefixes for which normalization needs to be skipped.
+ * see .NET6: PathInternal.IsExtended()
+ */
+static inline int path_is_extended(const wchar_t *path)
+{
+    if (path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\')
+        return 1;
+
+    return 0;
+}
+
+/**
+ * Checks for a device path prefix.
+ * see .NET6: PathInternal.IsDevicePath()
+ */
+static inline int path_is_device_path(const wchar_t *path)
+{
+    if (path[0] == L'\\' && path[1] == L'\\' && path[2] == L'.' && path[3] == L'\\')
+        return 1;
+
+    return 0;
+}
+
+/**
+ * Performs path normalization by calling GetFullPathNameW().
+ * see .NET6: PathHelper.GetFullPathName()
+ */
+static inline int get_full_path_name(wchar_t **ppath_w)
+{
+    int num_chars;
+    wchar_t *temp_w;
+
+    num_chars = GetFullPathNameW(*ppath_w, 0, NULL, NULL);
+    if (num_chars <= 0) {
+        errno = EINVAL;
+        return -1;
+    }
+
+    temp_w = (wchar_t *)av_calloc(num_chars, sizeof(wchar_t));
+    if (!temp_w) {
+        errno = ENOMEM;
+        return -1;
+    }
+
+    num_chars = GetFullPathNameW(*ppath_w, num_chars, temp_w, NULL);
+    if (num_chars <= 0) {
+        errno = EINVAL;
+        return -1;
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+/**
+ * Normalizes a Windows file or folder path.
+ * Expansion of short paths (with 8.3 path components) is currently omitted
+ * as it is not required for accessing long paths.
+ * see .NET6: PathHelper.Normalize().
+ */
+static inline int path_normalize(wchar_t **ppath_w)
+{
+    int ret;
+
+    if ((ret = get_full_path_name(ppath_w)) < 0)
+        return ret;
+
+    /* What .NET does at this point is to call PathHelper.TryExpandShortFileName()
+     * in case the path contains a '~' character.
+     * We don't need to do this as we don't need to normalize the file name
+     * for presentation, and the extended path prefix works with 8.3 path
+     * components as well
+     */
+    return 0;
+}
+
+/**
+ * Adds an extended path or UNC prefix to longs paths or paths ending
+ * with a space or a dot. (' ' or '.').
+ * This function expects that the path has been normalized before by
+ * calling path_normalize() and it doesn't check whether the path is
+ * actually long (> MAX_PATH).
+ * see .NET6: PathInternal.EnsureExtendedPrefix() *
+ */
+static inline int add_extended_prefix(wchar_t **ppath_w)
+{
+    const wchar_t *unc_prefix           = L"\\\\?\\UNC\\";
+    const wchar_t *extended_path_prefix = L"\\\\?\\";
+    const wchar_t *path_w               = *ppath_w;
+    const size_t len                    = wcslen(path_w);
+    wchar_t *temp_w;
+
+    /* We're skipping the check IsPartiallyQualified() because
+     * we expect to have called GetFullPathNameW() already. */
+    if (len < 2 || path_is_extended(*ppath_w) || path_is_device_path(*ppath_w)) {
+        return 0;
+    }
+
+    if (path_w[0] == L'\\' && path_w[1] == L'\\') {
+        /* unc_prefix length is 8 plus 1 for terminating zeros,
+         * we subtract 2 for the leading '\\' of the original path */
+        temp_w = (wchar_t *)av_calloc(len - 2 + 8 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, unc_prefix);
+        wcscat(temp_w, path_w + 2);
+    } else {
+        // The length of extended_path_prefix is 4 plus 1 for terminating zeros
+        temp_w = (wchar_t *)av_calloc(len + 4 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, extended_path_prefix);
+        wcscat(temp_w, path_w);
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+/**
+ * Converts a file or folder path to wchar_t for use with Windows file
+ * APIs. Paths with extended path prefix (either '\\?\' or \??\') are
+ * left unchanged.
+ * All other paths are normalized and converted to absolute paths.
+ * Longs paths (>= MAX_PATH) are prefixed with the extended path or extended
+ * UNC path prefix.
+ * see .NET6: Path.GetFullPath() and Path.GetFullPathInternal()
+ */
+static inline int get_extended_win32_path(const char *path, wchar_t **ppath_w)
+{
+    int ret;
+    size_t len;
+
+    if ((ret = utf8towchar(path, ppath_w)) < 0)
+        return ret;
+
+    if (path_is_extended(*ppath_w)) {
+        /* Paths prefixed with '\\?\' or \??\' are considered normalized by definition.
+         * Windows doesn't normalize those paths and neither should we.
+         */
+        return 0;
+    }
+
+    if ((ret = path_normalize(ppath_w)) < 0)
+        return ret;
+
+    // see .NET6: PathInternal.EnsureExtendedPrefixIfNeeded()
+    len = wcslen(*ppath_w);
+    if (len >= MAX_PATH) {
+        if ((ret = add_extended_prefix(ppath_w)) < 0)
+            return ret;
+    }
+
+    return 0;
+}
+
 #endif
 
 #endif /* AVUTIL_WCHAR_FILENAME_H */
-- 
ffmpeg-codebot

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 2/2] avformat/os_support: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavformat/os_support.h | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)

diff --git a/libavformat/os_support.h b/libavformat/os_support.h
index 5e6b32d2dc..bd8c89568f 100644
--- a/libavformat/os_support.h
+++ b/libavformat/os_support.h
@@ -49,7 +49,23 @@
 #  ifdef stat
 #   undef stat
 #  endif
-#  define stat _stati64
+#  define stat win32_stat
+
+    struct win32_stat
+    {
+        _dev_t         st_dev;
+        _ino_t         st_ino;
+        unsigned short st_mode;
+        short          st_nlink;
+        short          st_uid;
+        short          st_gid;
+        _dev_t         st_rdev;
+        __int64        st_size;
+        __time64_t     st_atime;
+        __time64_t     st_mtime;
+        __time64_t     st_ctime;
+    };
+
 #  ifdef fstat
 #   undef fstat
 #  endif
@@ -153,7 +169,7 @@ static inline int win32_##name(const char *filename_utf8) \
     wchar_t *filename_w;                                  \
     int ret;                                              \
                                                           \
-    if (utf8towchar(filename_utf8, &filename_w))          \
+    if (get_extended_win32_path(filename_utf8, &filename_w)) \
         return -1;                                        \
     if (!filename_w)                                      \
         goto fallback;                                    \
@@ -177,7 +193,7 @@ static inline int win32_##name(const char *filename_utf8, partype par) \
     wchar_t *filename_w;                                  \
     int ret;                                              \
                                                           \
-    if (utf8towchar(filename_utf8, &filename_w))          \
+    if (get_extended_win32_path(filename_utf8, &filename_w)) \
         return -1;                                        \
     if (!filename_w)                                      \
         goto fallback;                                    \
@@ -199,9 +215,9 @@ static inline int win32_rename(const char *src_utf8, const char *dest_utf8)
     wchar_t *src_w, *dest_w;
     int ret;
 
-    if (utf8towchar(src_utf8, &src_w))
+    if (get_extended_win32_path(src_utf8, &src_w))
         return -1;
-    if (utf8towchar(dest_utf8, &dest_w)) {
+    if (get_extended_win32_path(dest_utf8, &dest_w)) {
         av_free(src_w);
         return -1;
     }
-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[nil-admirari]

>> Not possible for stat precisely because of function and struct sharing a
>> name.

> That's exactly what is said above and before :-)

My previous question was not answered, so I had to look up the answer myself.

> I'm actually wondering how does it even compile. All stat structs in code
> become struct win32_stat, and all calls to stat become calls to win32_stat,
> which in turn wraps _wstati64. But _wstati64 does not accept struct win32_stat*,
> it accepts struct _stati64*. Content of these structs is probably identical, but
> it should not matter: C is typed nominally, not structurally.

Turns out C actually has a concept of compatible types:
https://en.cppreference.com/w/c/language/type.

The problems is:
> they are both structure/union/enumeration types, and
> - (c99) if one is declared with a tag, the other must also be declared with the same tag.
> ...
> If two declarations refer to the same object or function and do not use compatible types,
> the behavior of the program is undefined.

Your structure does not have the same tag as the CRT's one.
Are you sure you want to rely on undefined behaviour?

I haven't compiled your code, but the following simple example:

struct A { int a, b, c; };
struct B { int a, b, c; };
void printA(struct A *a);

struct B b = { 1, 2, 3 };
printA(&b);

generates a

warning: passing argument 1 of ‘printA’ from incompatible pointer type [-Wincompatible-pointer-types]
    |     printA(&b);
    |            ^~
    |            |
    |            struct B *
note: expected ‘struct A *’ but argument is of type ‘struct B *’
    | void printA(struct A *a)

Are you sure you wanna add a couple of similar warnings to the project?

Needless to repeat, _USE_32BIT_TIME_T is not supported.



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Monday, May 23, 2022 5:36 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename,
> file_open: Support long file names on Windows
> 
> >> Not possible for stat precisely because of function and struct sharing
> a
> >> name.
> 
> > That's exactly what is said above and before :-)
> 
> My previous question was not answered, so I had to look up the answer
> myself.
> 
> > I'm actually wondering how does it even compile. All stat structs in
> code
> > become struct win32_stat, and all calls to stat become calls to
> win32_stat,
> > which in turn wraps _wstati64. But _wstati64 does not accept struct
> win32_stat*,
> > it accepts struct _stati64*. Content of these structs is probably
> identical, but
> > it should not matter: C is typed nominally, not structurally.
> 
> Turns out C actually has a concept of compatible types:
> https://en.cppreference.com/w/c/language/type.
> 
> The problems is:
> > they are both structure/union/enumeration types, and
> > - (c99) if one is declared with a tag, the other must also be declared
> with the same tag.
> > ...
> > If two declarations refer to the same object or function and do not use
> compatible types,
> > the behavior of the program is undefined.
> 
> Your structure does not have the same tag as the CRT's one.
> Are you sure you want to rely on undefined behaviour?
> 
> I haven't compiled your code, but the following simple example:
> 
> struct A { int a, b, c; };
> struct B { int a, b, c; };
> void printA(struct A *a);
> 
> struct B b = { 1, 2, 3 };
> printA(&b);
> 
> generates a
> 
> warning: passing argument 1 of ‘printA’ from incompatible pointer type [-
> Wincompatible-pointer-types]
>     |     printA(&b);
>     |            ^~
>     |            |
>     |            struct B *
> note: expected ‘struct A *’ but argument is of type ‘struct B *’
>     | void printA(struct A *a)
> 
> Are you sure you wanna add a couple of similar warnings to the project?

This is not what's happening. No warnings, not even from clang diagnostics
with -Weverything.


> Needless to repeat, _USE_32BIT_TIME_T is not supported.

I don't think it ever was. Have you been compiling and using ffmpeg 
successfully with this?

Kind regards,
sw
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[Hendrik Leppkes]

On Mon, May 23, 2022 at 5:48 PM Soft Works <softworkz@hotmail.com> wrote:
>
>
>
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> > admirari@mailo.com
> > Sent: Monday, May 23, 2022 5:36 PM
> > To: ffmpeg-devel@ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename,
> > file_open: Support long file names on Windows
> >
> > >> Not possible for stat precisely because of function and struct sharing
> > a
> > >> name.
> >
> > > That's exactly what is said above and before :-)
> >
> > My previous question was not answered, so I had to look up the answer
> > myself.
> >
> > > I'm actually wondering how does it even compile. All stat structs in
> > code
> > > become struct win32_stat, and all calls to stat become calls to
> > win32_stat,
> > > which in turn wraps _wstati64. But _wstati64 does not accept struct
> > win32_stat*,
> > > it accepts struct _stati64*. Content of these structs is probably
> > identical, but
> > > it should not matter: C is typed nominally, not structurally.
> >
> > Turns out C actually has a concept of compatible types:
> > https://en.cppreference.com/w/c/language/type.
> >
> > The problems is:
> > > they are both structure/union/enumeration types, and
> > > - (c99) if one is declared with a tag, the other must also be declared
> > with the same tag.
> > > ...
> > > If two declarations refer to the same object or function and do not use
> > compatible types,
> > > the behavior of the program is undefined.
> >
> > Your structure does not have the same tag as the CRT's one.
> > Are you sure you want to rely on undefined behaviour?
> >
> > I haven't compiled your code, but the following simple example:
> >
> > struct A { int a, b, c; };
> > struct B { int a, b, c; };
> > void printA(struct A *a);
> >
> > struct B b = { 1, 2, 3 };
> > printA(&b);
> >
> > generates a
> >
> > warning: passing argument 1 of ‘printA’ from incompatible pointer type [-
> > Wincompatible-pointer-types]
> >     |     printA(&b);
> >     |            ^~
> >     |            |
> >     |            struct B *
> > note: expected ‘struct A *’ but argument is of type ‘struct B *’
> >     | void printA(struct A *a)
> >
> > Are you sure you wanna add a couple of similar warnings to the project?
>
> This is not what's happening. No warnings, not even from clang diagnostics
> with -Weverything.
>
>
> > Needless to repeat, _USE_32BIT_TIME_T is not supported.
>
> I don't think it ever was. Have you been compiling and using ffmpeg
> successfully with this?
>

_USE_32BIT_TIME_T is actually the default for 32-bit mingw-w64 builds,
so its certainly a situation on windows that needs to work - and does
in master.

- Hendrik
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Hendrik
> Leppkes
> Sent: Monday, May 23, 2022 7:12 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename,
> file_open: Support long file names on Windows
> 
> On Mon, May 23, 2022 at 5:48 PM Soft Works <softworkz@hotmail.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> > > admirari@mailo.com
> > > Sent: Monday, May 23, 2022 5:36 PM
> > > To: ffmpeg-devel@ffmpeg.org
> > > Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename,
> > > file_open: Support long file names on Windows
> > >
> > > >> Not possible for stat precisely because of function and struct
> sharing
> > > a
> > > >> name.
> > >
> > > > That's exactly what is said above and before :-)
> > >
> > > My previous question was not answered, so I had to look up the answer
> > > myself.
> > >
> > > > I'm actually wondering how does it even compile. All stat structs in
> > > code
> > > > become struct win32_stat, and all calls to stat become calls to
> > > win32_stat,
> > > > which in turn wraps _wstati64. But _wstati64 does not accept struct
> > > win32_stat*,
> > > > it accepts struct _stati64*. Content of these structs is probably
> > > identical, but
> > > > it should not matter: C is typed nominally, not structurally.
> > >
> > > Turns out C actually has a concept of compatible types:
> > > https://en.cppreference.com/w/c/language/type.
> > >
> > > The problems is:
> > > > they are both structure/union/enumeration types, and
> > > > - (c99) if one is declared with a tag, the other must also be
> declared
> > > with the same tag.
> > > > ...
> > > > If two declarations refer to the same object or function and do not
> use
> > > compatible types,
> > > > the behavior of the program is undefined.
> > >
> > > Your structure does not have the same tag as the CRT's one.
> > > Are you sure you want to rely on undefined behaviour?
> > >
> > > I haven't compiled your code, but the following simple example:
> > >
> > > struct A { int a, b, c; };
> > > struct B { int a, b, c; };
> > > void printA(struct A *a);
> > >
> > > struct B b = { 1, 2, 3 };
> > > printA(&b);
> > >
> > > generates a
> > >
> > > warning: passing argument 1 of ‘printA’ from incompatible pointer type
> [-
> > > Wincompatible-pointer-types]
> > >     |     printA(&b);
> > >     |            ^~
> > >     |            |
> > >     |            struct B *
> > > note: expected ‘struct A *’ but argument is of type ‘struct B *’
> > >     | void printA(struct A *a)
> > >
> > > Are you sure you wanna add a couple of similar warnings to the
> project?
> >
> > This is not what's happening. No warnings, not even from clang
> diagnostics
> > with -Weverything.
> >
> >
> > > Needless to repeat, _USE_32BIT_TIME_T is not supported.
> >
> > I don't think it ever was. Have you been compiling and using ffmpeg
> > successfully with this?
> >
> 
> _USE_32BIT_TIME_T is actually the default for 32-bit mingw-w64 builds,
> so its certainly a situation on windows that needs to work - and does
> in master.

Are you sure? It is not the default for 32 bit builds with MSVC.

But it could be made working with the way that I'm using in my current 
patchset (with a conditional define). So that's not really the question.

The primary question is whether the (admittedly) ugly way I'm
using is acceptable at all.

Suggestions are very welcome ;-) 
Would you have an idea perhaps?

Thanks,
softworkz



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Soft
> Works
> Sent: Tuesday, May 24, 2022 7:33 AM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename,
> file_open: Support long file names on Windows
> 
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Hendrik
> > Leppkes
> > Sent: Monday, May 23, 2022 7:12 PM
> > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename,
> > file_open: Support long file names on Windows
> >
> > On Mon, May 23, 2022 at 5:48 PM Soft Works <softworkz@hotmail.com>
> wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> nil-
> > > > admirari@mailo.com
> > > > Sent: Monday, May 23, 2022 5:36 PM
> > > > To: ffmpeg-devel@ffmpeg.org
> > > > Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename,
> > > > file_open: Support long file names on Windows
> > > >
> > > > >> Not possible for stat precisely because of function and struct
> > sharing
> > > > a
> > > > >> name.
> > > >
> > > > > That's exactly what is said above and before :-)
> > > >
> > > > My previous question was not answered, so I had to look up the
> answer
> > > > myself.
> > > >
> > > > > I'm actually wondering how does it even compile. All stat structs
> in
> > > > code
> > > > > become struct win32_stat, and all calls to stat become calls to
> > > > win32_stat,
> > > > > which in turn wraps _wstati64. But _wstati64 does not accept
> struct
> > > > win32_stat*,
> > > > > it accepts struct _stati64*. Content of these structs is probably
> > > > identical, but
> > > > > it should not matter: C is typed nominally, not structurally.
> > > >
> > > > Turns out C actually has a concept of compatible types:
> > > > https://en.cppreference.com/w/c/language/type.
> > > >
> > > > The problems is:
> > > > > they are both structure/union/enumeration types, and
> > > > > - (c99) if one is declared with a tag, the other must also be
> > declared
> > > > with the same tag.
> > > > > ...
> > > > > If two declarations refer to the same object or function and do
> not
> > use
> > > > compatible types,
> > > > > the behavior of the program is undefined.
> > > >
> > > > Your structure does not have the same tag as the CRT's one.
> > > > Are you sure you want to rely on undefined behaviour?
> > > >
> > > > I haven't compiled your code, but the following simple example:
> > > >
> > > > struct A { int a, b, c; };
> > > > struct B { int a, b, c; };
> > > > void printA(struct A *a);
> > > >
> > > > struct B b = { 1, 2, 3 };
> > > > printA(&b);
> > > >
> > > > generates a
> > > >
> > > > warning: passing argument 1 of ‘printA’ from incompatible pointer
> type
> > [-
> > > > Wincompatible-pointer-types]
> > > >     |     printA(&b);
> > > >     |            ^~
> > > >     |            |
> > > >     |            struct B *
> > > > note: expected ‘struct A *’ but argument is of type ‘struct B *’
> > > >     | void printA(struct A *a)
> > > >
> > > > Are you sure you wanna add a couple of similar warnings to the
> > project?
> > >
> > > This is not what's happening. No warnings, not even from clang
> > diagnostics
> > > with -Weverything.
> > >
> > >
> > > > Needless to repeat, _USE_32BIT_TIME_T is not supported.
> > >
> > > I don't think it ever was. Have you been compiling and using ffmpeg
> > > successfully with this?
> > >
> >
> > _USE_32BIT_TIME_T is actually the default for 32-bit mingw-w64 builds,
> > so its certainly a situation on windows that needs to work - and does
> > in master.
> 
> Are you sure? It is not the default for 32 bit builds with MSVC.

Sorry, you're right. I found it. It's in _mingw.h.

I'll update the patch.

Thanks,
sw
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v5 0/2] Support long file names on Windows

[ffmpegagent]

This patchset adds support for long file and directory paths on Windows. The
implementation follows the same logic that .NET is using internally, with
the only exception that it doesn't expand short path components in 8.3
format. .NET does this as the same function is also used for other purposes,
but in our case, that's not required. Short (8.3) paths are working as well
with the extended path prefix, even when longer than 260.

Successfully tested:

 * Regular paths wth drive letter
 * Regular UNC paths
 * Long paths wth drive letter
 * Long paths wth drive letter and forward slashes
 * Long UNC paths
 * Prefixed paths wth drive letter
 * Prefixed UNC paths

I have kept the individual functions separate on purpose, to make it easy to
compare with the .NET impl. (compilers should inlinie those anyway)

v2

 * wchar_filename: Improve comments and function documentation
 * os_support: adjust defines to use win32_stat

v3

 * removed length check in path_is_extended()
 * added path_is_device_path() check in add_extended_prefix()
 * add_extended_prefix(): clarified doc and add checks
 * clarified string allocation length calculation
 * replaced 260 with MAX_PATH
 * removed redundant checks after normalization

v4

 * rebased. no changes

v5

 * resolved the ugly struct duplication
 * compatible with _USE_32BIT_TIME_T

softworkz (2):
  avutil/wchar_filename,file_open: Support long file names on Windows
  avformat/os_support: Support long file names on Windows

 libavformat/os_support.h   |  16 ++--
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 166 +++++++++++++++++++++++++++++++++++++
 3 files changed, 178 insertions(+), 6 deletions(-)


base-commit: 6076dbcb55d0c9b6693d1acad12a63f7268301aa
Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-28%2Fsoftworkz%2Fsubmit_long_filenames-v5
Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v5
Pull-Request: https://github.com/ffstaging/FFmpeg/pull/28

Range-diff vs v4:

 1:  13118dc1fa = 1:  13118dc1fa avutil/wchar_filename,file_open: Support long file names on Windows
 2:  252ed89499 ! 2:  5313aeec0e avformat/os_support: Support long file names on Windows
     @@ libavformat/os_support.h
      +
      +    struct win32_stat
      +    {
     -+        _dev_t         st_dev;
     -+        _ino_t         st_ino;
     -+        unsigned short st_mode;
     -+        short          st_nlink;
     -+        short          st_uid;
     -+        short          st_gid;
     -+        _dev_t         st_rdev;
     -+        __int64        st_size;
     -+        __time64_t     st_atime;
     -+        __time64_t     st_mtime;
     -+        __time64_t     st_ctime;
     ++        struct _stati64;
      +    };
      +
       #  ifdef fstat

-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v5 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 166 +++++++++++++++++++++++++++++++++++++
 2 files changed, 167 insertions(+), 1 deletion(-)

diff --git a/libavutil/file_open.c b/libavutil/file_open.c
index fb64c2e4ee..58a6073353 100644
--- a/libavutil/file_open.c
+++ b/libavutil/file_open.c
@@ -45,7 +45,7 @@ static int win32_open(const char *filename_utf8, int oflag, int pmode)
     wchar_t *filename_w;
 
     /* convert UTF-8 to wide chars */
-    if (utf8towchar(filename_utf8, &filename_w))
+    if (get_extended_win32_path(filename_utf8, &filename_w))
         return -1;
     if (!filename_w)
         goto fallback;
diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h
index 90f082452c..94f8ce54b5 100644
--- a/libavutil/wchar_filename.h
+++ b/libavutil/wchar_filename.h
@@ -40,6 +40,172 @@ static inline int utf8towchar(const char *filename_utf8, wchar_t **filename_w)
     MultiByteToWideChar(CP_UTF8, 0, filename_utf8, -1, *filename_w, num_chars);
     return 0;
 }
+
+/**
+ * Checks for extended path prefixes for which normalization needs to be skipped.
+ * see .NET6: PathInternal.IsExtended()
+ */
+static inline int path_is_extended(const wchar_t *path)
+{
+    if (path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\')
+        return 1;
+
+    return 0;
+}
+
+/**
+ * Checks for a device path prefix.
+ * see .NET6: PathInternal.IsDevicePath()
+ */
+static inline int path_is_device_path(const wchar_t *path)
+{
+    if (path[0] == L'\\' && path[1] == L'\\' && path[2] == L'.' && path[3] == L'\\')
+        return 1;
+
+    return 0;
+}
+
+/**
+ * Performs path normalization by calling GetFullPathNameW().
+ * see .NET6: PathHelper.GetFullPathName()
+ */
+static inline int get_full_path_name(wchar_t **ppath_w)
+{
+    int num_chars;
+    wchar_t *temp_w;
+
+    num_chars = GetFullPathNameW(*ppath_w, 0, NULL, NULL);
+    if (num_chars <= 0) {
+        errno = EINVAL;
+        return -1;
+    }
+
+    temp_w = (wchar_t *)av_calloc(num_chars, sizeof(wchar_t));
+    if (!temp_w) {
+        errno = ENOMEM;
+        return -1;
+    }
+
+    num_chars = GetFullPathNameW(*ppath_w, num_chars, temp_w, NULL);
+    if (num_chars <= 0) {
+        errno = EINVAL;
+        return -1;
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+/**
+ * Normalizes a Windows file or folder path.
+ * Expansion of short paths (with 8.3 path components) is currently omitted
+ * as it is not required for accessing long paths.
+ * see .NET6: PathHelper.Normalize().
+ */
+static inline int path_normalize(wchar_t **ppath_w)
+{
+    int ret;
+
+    if ((ret = get_full_path_name(ppath_w)) < 0)
+        return ret;
+
+    /* What .NET does at this point is to call PathHelper.TryExpandShortFileName()
+     * in case the path contains a '~' character.
+     * We don't need to do this as we don't need to normalize the file name
+     * for presentation, and the extended path prefix works with 8.3 path
+     * components as well
+     */
+    return 0;
+}
+
+/**
+ * Adds an extended path or UNC prefix to longs paths or paths ending
+ * with a space or a dot. (' ' or '.').
+ * This function expects that the path has been normalized before by
+ * calling path_normalize() and it doesn't check whether the path is
+ * actually long (> MAX_PATH).
+ * see .NET6: PathInternal.EnsureExtendedPrefix() *
+ */
+static inline int add_extended_prefix(wchar_t **ppath_w)
+{
+    const wchar_t *unc_prefix           = L"\\\\?\\UNC\\";
+    const wchar_t *extended_path_prefix = L"\\\\?\\";
+    const wchar_t *path_w               = *ppath_w;
+    const size_t len                    = wcslen(path_w);
+    wchar_t *temp_w;
+
+    /* We're skipping the check IsPartiallyQualified() because
+     * we expect to have called GetFullPathNameW() already. */
+    if (len < 2 || path_is_extended(*ppath_w) || path_is_device_path(*ppath_w)) {
+        return 0;
+    }
+
+    if (path_w[0] == L'\\' && path_w[1] == L'\\') {
+        /* unc_prefix length is 8 plus 1 for terminating zeros,
+         * we subtract 2 for the leading '\\' of the original path */
+        temp_w = (wchar_t *)av_calloc(len - 2 + 8 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, unc_prefix);
+        wcscat(temp_w, path_w + 2);
+    } else {
+        // The length of extended_path_prefix is 4 plus 1 for terminating zeros
+        temp_w = (wchar_t *)av_calloc(len + 4 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, extended_path_prefix);
+        wcscat(temp_w, path_w);
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+/**
+ * Converts a file or folder path to wchar_t for use with Windows file
+ * APIs. Paths with extended path prefix (either '\\?\' or \??\') are
+ * left unchanged.
+ * All other paths are normalized and converted to absolute paths.
+ * Longs paths (>= MAX_PATH) are prefixed with the extended path or extended
+ * UNC path prefix.
+ * see .NET6: Path.GetFullPath() and Path.GetFullPathInternal()
+ */
+static inline int get_extended_win32_path(const char *path, wchar_t **ppath_w)
+{
+    int ret;
+    size_t len;
+
+    if ((ret = utf8towchar(path, ppath_w)) < 0)
+        return ret;
+
+    if (path_is_extended(*ppath_w)) {
+        /* Paths prefixed with '\\?\' or \??\' are considered normalized by definition.
+         * Windows doesn't normalize those paths and neither should we.
+         */
+        return 0;
+    }
+
+    if ((ret = path_normalize(ppath_w)) < 0)
+        return ret;
+
+    // see .NET6: PathInternal.EnsureExtendedPrefixIfNeeded()
+    len = wcslen(*ppath_w);
+    if (len >= MAX_PATH) {
+        if ((ret = add_extended_prefix(ppath_w)) < 0)
+            return ret;
+    }
+
+    return 0;
+}
+
 #endif
 
 #endif /* AVUTIL_WCHAR_FILENAME_H */
-- 
ffmpeg-codebot

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavformat/os_support.h | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/libavformat/os_support.h b/libavformat/os_support.h
index 5e6b32d2dc..d4c07803a5 100644
--- a/libavformat/os_support.h
+++ b/libavformat/os_support.h
@@ -49,7 +49,13 @@
 #  ifdef stat
 #   undef stat
 #  endif
-#  define stat _stati64
+#  define stat win32_stat
+
+    struct win32_stat
+    {
+        struct _stati64;
+    };
+
 #  ifdef fstat
 #   undef fstat
 #  endif
@@ -153,7 +159,7 @@ static inline int win32_##name(const char *filename_utf8) \
     wchar_t *filename_w;                                  \
     int ret;                                              \
                                                           \
-    if (utf8towchar(filename_utf8, &filename_w))          \
+    if (get_extended_win32_path(filename_utf8, &filename_w)) \
         return -1;                                        \
     if (!filename_w)                                      \
         goto fallback;                                    \
@@ -177,7 +183,7 @@ static inline int win32_##name(const char *filename_utf8, partype par) \
     wchar_t *filename_w;                                  \
     int ret;                                              \
                                                           \
-    if (utf8towchar(filename_utf8, &filename_w))          \
+    if (get_extended_win32_path(filename_utf8, &filename_w)) \
         return -1;                                        \
     if (!filename_w)                                      \
         goto fallback;                                    \
@@ -199,9 +205,9 @@ static inline int win32_rename(const char *src_utf8, const char *dest_utf8)
     wchar_t *src_w, *dest_w;
     int ret;
 
-    if (utf8towchar(src_utf8, &src_w))
+    if (get_extended_win32_path(src_utf8, &src_w))
         return -1;
-    if (utf8towchar(dest_utf8, &dest_w)) {
+    if (get_extended_win32_path(dest_utf8, &dest_w)) {
         av_free(src_w);
         return -1;
     }
-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v5 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[Martin Storsjö]

On Tue, 24 May 2022, softworkz wrote:

> From: softworkz <softworkz@hotmail.com>
>
> Signed-off-by: softworkz <softworkz@hotmail.com>
> ---
> libavutil/file_open.c      |   2 +-
> libavutil/wchar_filename.h | 166 +++++++++++++++++++++++++++++++++++++
> 2 files changed, 167 insertions(+), 1 deletion(-)
>
> diff --git a/libavutil/file_open.c b/libavutil/file_open.c
> index fb64c2e4ee..58a6073353 100644
> --- a/libavutil/file_open.c
> +++ b/libavutil/file_open.c
> @@ -45,7 +45,7 @@ static int win32_open(const char *filename_utf8, int oflag, int pmode)
>     wchar_t *filename_w;
>
>     /* convert UTF-8 to wide chars */
> -    if (utf8towchar(filename_utf8, &filename_w))
> +    if (get_extended_win32_path(filename_utf8, &filename_w))
>         return -1;

Note, the caller expects that if the function returned an error, all 
temporary allocations made by the function have been freed - the caller 
doesn't need to free those allocations.

>     if (!filename_w)
>         goto fallback;
> diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h
> index 90f082452c..94f8ce54b5 100644
> --- a/libavutil/wchar_filename.h
> +++ b/libavutil/wchar_filename.h
> @@ -40,6 +40,172 @@ static inline int utf8towchar(const char *filename_utf8, wchar_t **filename_w)
>     MultiByteToWideChar(CP_UTF8, 0, filename_utf8, -1, *filename_w, num_chars);
>     return 0;
> }
> +
> +/**
> + * Checks for extended path prefixes for which normalization needs to be skipped.
> + * see .NET6: PathInternal.IsExtended()
> + */
> +static inline int path_is_extended(const wchar_t *path)
> +{
> +    if (path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\')
> +        return 1;
> +
> +    return 0;
> +}
> +
> +/**
> + * Checks for a device path prefix.
> + * see .NET6: PathInternal.IsDevicePath()
> + */
> +static inline int path_is_device_path(const wchar_t *path)
> +{
> +    if (path[0] == L'\\' && path[1] == L'\\' && path[2] == L'.' && path[3] == L'\\')
> +        return 1;
> +
> +    return 0;
> +}
> +
> +/**
> + * Performs path normalization by calling GetFullPathNameW().
> + * see .NET6: PathHelper.GetFullPathName()
> + */
> +static inline int get_full_path_name(wchar_t **ppath_w)
> +{
> +    int num_chars;
> +    wchar_t *temp_w;
> +
> +    num_chars = GetFullPathNameW(*ppath_w, 0, NULL, NULL);
> +    if (num_chars <= 0) {
> +        errno = EINVAL;
> +        return -1;
> +    }
> +
> +    temp_w = (wchar_t *)av_calloc(num_chars, sizeof(wchar_t));
> +    if (!temp_w) {
> +        errno = ENOMEM;
> +        return -1;
> +    }
> +
> +    num_chars = GetFullPathNameW(*ppath_w, num_chars, temp_w, NULL);
> +    if (num_chars <= 0) {
> +        errno = EINVAL;
> +        return -1;

In this error handling path, you leak the allocated temp_w

> +    }
> +
> +    av_freep(ppath_w);
> +    *ppath_w = temp_w;
> +
> +    return 0;
> +}
> +
> +/**
> + * Normalizes a Windows file or folder path.
> + * Expansion of short paths (with 8.3 path components) is currently omitted
> + * as it is not required for accessing long paths.
> + * see .NET6: PathHelper.Normalize().
> + */
> +static inline int path_normalize(wchar_t **ppath_w)
> +{
> +    int ret;
> +
> +    if ((ret = get_full_path_name(ppath_w)) < 0)
> +        return ret;
> +
> +    /* What .NET does at this point is to call PathHelper.TryExpandShortFileName()
> +     * in case the path contains a '~' character.
> +     * We don't need to do this as we don't need to normalize the file name
> +     * for presentation, and the extended path prefix works with 8.3 path
> +     * components as well
> +     */
> +    return 0;
> +}
> +
> +/**
> + * Adds an extended path or UNC prefix to longs paths or paths ending
> + * with a space or a dot. (' ' or '.').
> + * This function expects that the path has been normalized before by
> + * calling path_normalize() and it doesn't check whether the path is
> + * actually long (> MAX_PATH).
> + * see .NET6: PathInternal.EnsureExtendedPrefix() *
> + */
> +static inline int add_extended_prefix(wchar_t **ppath_w)
> +{
> +    const wchar_t *unc_prefix           = L"\\\\?\\UNC\\";
> +    const wchar_t *extended_path_prefix = L"\\\\?\\";
> +    const wchar_t *path_w               = *ppath_w;
> +    const size_t len                    = wcslen(path_w);
> +    wchar_t *temp_w;
> +
> +    /* We're skipping the check IsPartiallyQualified() because
> +     * we expect to have called GetFullPathNameW() already. */
> +    if (len < 2 || path_is_extended(*ppath_w) || path_is_device_path(*ppath_w)) {
> +        return 0;
> +    }
> +
> +    if (path_w[0] == L'\\' && path_w[1] == L'\\') {
> +        /* unc_prefix length is 8 plus 1 for terminating zeros,
> +         * we subtract 2 for the leading '\\' of the original path */
> +        temp_w = (wchar_t *)av_calloc(len - 2 + 8 + 1, sizeof(wchar_t));
> +        if (!temp_w) {
> +            errno = ENOMEM;
> +            return -1;
> +        }
> +        wcscpy(temp_w, unc_prefix);
> +        wcscat(temp_w, path_w + 2);
> +    } else {
> +        // The length of extended_path_prefix is 4 plus 1 for terminating zeros
> +        temp_w = (wchar_t *)av_calloc(len + 4 + 1, sizeof(wchar_t));
> +        if (!temp_w) {
> +            errno = ENOMEM;
> +            return -1;
> +        }
> +        wcscpy(temp_w, extended_path_prefix);
> +        wcscat(temp_w, path_w);
> +    }
> +
> +    av_freep(ppath_w);
> +    *ppath_w = temp_w;
> +
> +    return 0;
> +}
> +
> +/**
> + * Converts a file or folder path to wchar_t for use with Windows file
> + * APIs. Paths with extended path prefix (either '\\?\' or \??\') are
> + * left unchanged.
> + * All other paths are normalized and converted to absolute paths.
> + * Longs paths (>= MAX_PATH) are prefixed with the extended path or extended
> + * UNC path prefix.
> + * see .NET6: Path.GetFullPath() and Path.GetFullPathInternal()
> + */
> +static inline int get_extended_win32_path(const char *path, wchar_t **ppath_w)
> +{
> +    int ret;
> +    size_t len;
> +
> +    if ((ret = utf8towchar(path, ppath_w)) < 0)
> +        return ret;
> +
> +    if (path_is_extended(*ppath_w)) {
> +        /* Paths prefixed with '\\?\' or \??\' are considered normalized by definition.
> +         * Windows doesn't normalize those paths and neither should we.
> +         */
> +        return 0;
> +    }
> +
> +    if ((ret = path_normalize(ppath_w)) < 0)
> +        return ret;

If we return an error here, we have already allocated an output in 
ppath_w - but the caller won't clean up such allocations on errors. Thus, 
if we have allocated something here (that wasn't allocated before), we 
must take care to clean it up on errors.


Additionally - with all the references to .NET6, could you add a base url 
somewhere in the code, where one can look up those references? For someone 
not familiar with the .NET ecosystem, I'm not entirely sure where I would 
start.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support long file names on Windows

[Martin Storsjö]

On Tue, 24 May 2022, softworkz wrote:

> From: softworkz <softworkz@hotmail.com>
>
> Signed-off-by: softworkz <softworkz@hotmail.com>
> ---
> libavformat/os_support.h | 16 +++++++++++-----
> 1 file changed, 11 insertions(+), 5 deletions(-)
>
> diff --git a/libavformat/os_support.h b/libavformat/os_support.h
> index 5e6b32d2dc..d4c07803a5 100644
> --- a/libavformat/os_support.h
> +++ b/libavformat/os_support.h
> @@ -49,7 +49,13 @@
> #  ifdef stat
> #   undef stat
> #  endif
> -#  define stat _stati64
> +#  define stat win32_stat
> +
> +    struct win32_stat
> +    {
> +        struct _stati64;
> +    };

Is it possible to work around this issue by doing "#define stat(a,b)" 
which only should apply on the function, not to the struct? Then we can't 
redirect "struct stat" into "struct _stati64" at the same time...

A safe way forward could be to switch code to just using "struct 
ff_stat_struct", and define ff_stat_struct to the name of the struct we 
exepct to use. It's not pretty, and it affects users which no longer can 
use the default POSIX stat form of the calls, but it would fix the issue 
of redirecting the struct and function separately, without needing to know 
what exactly is in the struct (because we really shouldn't be 
hardcoding/assuming that).

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: Martin Storsjö <martin@martin.st>
> Sent: Tuesday, May 24, 2022 11:23 AM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Cc: softworkz <softworkz@hotmail.com>; Hendrik Leppkes
> <h.leppkes@gmail.com>
> Subject: Re: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support
> long file names on Windows
> 
> On Tue, 24 May 2022, softworkz wrote:
> 
> > From: softworkz <softworkz@hotmail.com>
> >
> > Signed-off-by: softworkz <softworkz@hotmail.com>
> > ---
> > libavformat/os_support.h | 16 +++++++++++-----
> > 1 file changed, 11 insertions(+), 5 deletions(-)
> >
> > diff --git a/libavformat/os_support.h b/libavformat/os_support.h
> > index 5e6b32d2dc..d4c07803a5 100644
> > --- a/libavformat/os_support.h
> > +++ b/libavformat/os_support.h
> > @@ -49,7 +49,13 @@
> > #  ifdef stat
> > #   undef stat
> > #  endif
> > -#  define stat _stati64
> > +#  define stat win32_stat
> > +
> > +    struct win32_stat
> > +    {
> > +        struct _stati64;
> > +    };
> 
> Is it possible to work around this issue by doing "#define stat(a,b)"
> which only should apply on the function, not to the struct? 

How could this be possible? A define is only doing string replacements,
so I wouldn't know how it could be restricted to the function, but 
not the struct.

> A safe way forward could be to switch code to just using "struct
> ff_stat_struct", and define ff_stat_struct to the name of the struct we
> exepct to use. It's not pretty, and it affects users which no longer can
> use the default POSIX stat form of the calls

That's why I took the effort to make this work.

> but it would fix the issue
> of redirecting the struct and function separately, without needing to know
> what exactly is in the struct (because we really shouldn't be
> hardcoding/assuming that).

That doesn't apply because the current code already does this:

DEF_FS_FUNCTION2(stat, _wstati64, _stati64, struct stat*)

Which means that it specifically chooses _stati64 which is a public
MS API: 

https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/stat-functions?view=msvc-170

And we know that it takes _stati64 as the parameter.

This code:

    struct win32_stat
    {
        struct _stati64;
    };

makes use of a MS compiler feature called "anonymous structures":

https://docs.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-2013/z2cx9y4f(v=vs.120)?redirectedfrom=MSDN

This way, we automatically "inherit" the members of the struct
(whatever their members would be).


Thanks for reviewing,
softworkz

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Soft
> Works
> Sent: Monday, May 23, 2022 5:48 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename,
> file_open: Support long file names on Windows
> 
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> > admirari@mailo.com
> > Sent: Monday, May 23, 2022 5:36 PM
> > To: ffmpeg-devel@ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename,
> > file_open: Support long file names on Windows
> >
> > >> Not possible for stat precisely because of function and struct
> sharing
> > a
> > >> name.
> >
> > > That's exactly what is said above and before :-)
> >
> > My previous question was not answered, so I had to look up the answer
> > myself.
> >
> > > I'm actually wondering how does it even compile. All stat structs in
> > code
> > > become struct win32_stat, and all calls to stat become calls to
> > win32_stat,
> > > which in turn wraps _wstati64. But _wstati64 does not accept struct
> > win32_stat*,
> > > it accepts struct _stati64*. Content of these structs is probably
> > identical, but
> > > it should not matter: C is typed nominally, not structurally.
> >
> > Turns out C actually has a concept of compatible types:
> > https://en.cppreference.com/w/c/language/type.
> >
> > The problems is:
> > > they are both structure/union/enumeration types, and
> > > - (c99) if one is declared with a tag, the other must also be declared
> > with the same tag.
> > > ...
> > > If two declarations refer to the same object or function and do not
> use
> > compatible types,
> > > the behavior of the program is undefined.
> >
> > Your structure does not have the same tag as the CRT's one.
> > Are you sure you want to rely on undefined behaviour?
> >
> > I haven't compiled your code, but the following simple example:
> >
> > struct A { int a, b, c; };
> > struct B { int a, b, c; };
> > void printA(struct A *a);
> >
> > struct B b = { 1, 2, 3 };
> > printA(&b);
> >
> > generates a
> >
> > warning: passing argument 1 of ‘printA’ from incompatible pointer type
> [-
> > Wincompatible-pointer-types]
> >     |     printA(&b);
> >     |            ^~
> >     |            |
> >     |            struct B *
> > note: expected ‘struct A *’ but argument is of type ‘struct B *’
> >     | void printA(struct A *a)
> >
> > Are you sure you wanna add a couple of similar warnings to the project?
> 
> This is not what's happening. No warnings, not even from clang diagnostics
> with -Weverything.

I'm sorry, you were right and I was wrong. The includes and macros had hidden
away all the warnings.

For my new solution which I had submitted today, I had made a test in a code
file where I put all the new things directly and expanded the macros.

This gives in fact the kind of error you mentioned:

... function': incompatible types - from 'win32_stat *' to '_stat64 *'


I had explained the way it works to Martin in another response. The 
relevant part is:

The struct:

    struct win32_stat
    {
        struct _stati64;
    };

makes use of a MS compiler feature called "anonymous structures":
This way, we automatically "inherit" the members of the struct.

Now, that still gives the incompatible type warning. 

If we would
want to get rid of this, we could define the struct as follows:


    struct win32_stat
    {
        union
        {
            struct _stati64;
            struct _stati64 stat;
        };
    };

The union is anonymous and includes _stati64 twice: once anonymous
and once named.

This would allow us to define our win32_stat function like this:


static inline int win32_stat(const char *filename_utf8, struct win32_stat *par)
{
    wchar_t *filename_w;
    int ret;
    if (get_extended_win32_path(filename_utf8, &filename_w))
        return -1;
    if (!filename_w)
        goto fallback;
    ret = _wstat64(filename_w, &par->stat);
    av_free(filename_w);
    return ret;
fallback:
      return _stat64(filename_utf8, &par->stat);
}

so it uses the ->stat member for doing the api calls while
the calling (ffmpeg) code can use the structure as if it was the 
actual POSIX stat structure.


Kind regards,
sw





_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support long file names on Windows

[Martin Storsjö]

On Tue, 24 May 2022, Soft Works wrote:

>> -----Original Message-----
>> From: Martin Storsjö <martin@martin.st>
>> Sent: Tuesday, May 24, 2022 11:23 AM
>> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
>> Cc: softworkz <softworkz@hotmail.com>; Hendrik Leppkes
>> <h.leppkes@gmail.com>
>> Subject: Re: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support
>> long file names on Windows
>>
>> On Tue, 24 May 2022, softworkz wrote:
>>
>>> From: softworkz <softworkz@hotmail.com>
>>>
>>> Signed-off-by: softworkz <softworkz@hotmail.com>
>>> ---
>>> libavformat/os_support.h | 16 +++++++++++-----
>>> 1 file changed, 11 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/libavformat/os_support.h b/libavformat/os_support.h
>>> index 5e6b32d2dc..d4c07803a5 100644
>>> --- a/libavformat/os_support.h
>>> +++ b/libavformat/os_support.h
>>> @@ -49,7 +49,13 @@
>>> #  ifdef stat
>>> #   undef stat
>>> #  endif
>>> -#  define stat _stati64
>>> +#  define stat win32_stat
>>> +
>>> +    struct win32_stat
>>> +    {
>>> +        struct _stati64;
>>> +    };
>>
>> Is it possible to work around this issue by doing "#define stat(a,b)"
>> which only should apply on the function, not to the struct?
>
> How could this be possible? A define is only doing string replacements,
> so I wouldn't know how it could be restricted to the function, but
> not the struct.

If unsure about a tool feature, please try it out for yourself. Yes, a 
define is only a string replacement, but a define with parameters only 
matches the string occurs with parenthesis afterwards. Consider this 
example:

$ cat test.c
#define stat(a, b) win32_stat(a, b)

struct stat {
 	int a, b, c;
};

void stat (struct stat *st, const char* filename);

void func(const char* filename) {
 	struct stat st;
 	stat (&st, filename);
}
$ cc -E test.c
# 1 "test.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 31 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 32 "<command-line>" 2
# 1 "test.c"


struct stat {
  int a, b, c;
};

void win32_stat(struct stat *st, const char* filename);

void func(const char* filename) {
  struct stat st;
  win32_stat(&st, filename);
}


So here, the stat -> win32_stat rewrite only applied on the function 
declaration and call, but not on the structs.

Not saying that this necessarily is the way forward, but I was just 
mentioning it as a potential option to consider.

>> A safe way forward could be to switch code to just using "struct
>> ff_stat_struct", and define ff_stat_struct to the name of the struct we
>> exepct to use. It's not pretty, and it affects users which no longer can
>> use the default POSIX stat form of the calls
>
> That's why I took the effort to make this work.
>
>> but it would fix the issue
>> of redirecting the struct and function separately, without needing to know
>> what exactly is in the struct (because we really shouldn't be
>> hardcoding/assuming that).
>
> That doesn't apply because the current code already does this:
>
> DEF_FS_FUNCTION2(stat, _wstati64, _stati64, struct stat*)
>
> Which means that it specifically chooses _stati64 which is a public
> MS API:
>
> https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/stat-functions?view=msvc-170
>
> And we know that it takes _stati64 as the parameter.

Yes, there is no disagreement about that part.

> This code:
>
>    struct win32_stat
>    {
>        struct _stati64;
>    };
>
> makes use of a MS compiler feature called "anonymous structures":
>
> https://docs.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-2013/z2cx9y4f(v=vs.120)?redirectedfrom=MSDN
>
> This way, we automatically "inherit" the members of the struct
> (whatever their members would be).

This, as the article itself clearly declares, is a C language extension. 
GCC allows it in mingw mode, but Clang doesn't. (It's possible to use it 
in Clang too if you enable it with -fms-extensions though.)

$ cat test2.c
struct orig_struct {
 	int a, b, c;
};
struct my_struct {
 	struct orig_struct;
};
void set(struct my_struct *st) {
 	st->a = 42;
}
$ clang -target x86_64-w64-mingw32 -c test2.c
test2.c:5:2: warning: declaration does not declare anything [-Wmissing-declarations]
         struct orig_struct;
         ^
test2.c:8:6: error: no member named 'a' in 'struct my_struct'
         st->a = 42;
         ~~  ^
1 warning and 1 error generated.
$ clang -target x86_64-w64-mingw32 -c test2.c -fms-extensions
test2.c:5:2: warning: anonymous structs are a Microsoft extension [-Wmicrosoft-anon-tag]
         struct orig_struct;
         ^
1 warning generated.


// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: Martin Storsjö <martin@martin.st>
> Sent: Tuesday, May 24, 2022 12:26 PM
> To: Soft Works <softworkz@hotmail.com>
> Cc: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>;
> Hendrik Leppkes <h.leppkes@gmail.com>
> Subject: RE: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support
> long file names on Windows
> 
> On Tue, 24 May 2022, Soft Works wrote:
> 
> >> -----Original Message-----
> >> From: Martin Storsjö <martin@martin.st>
> >> Sent: Tuesday, May 24, 2022 11:23 AM
> >> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> >> Cc: softworkz <softworkz@hotmail.com>; Hendrik Leppkes
> >> <h.leppkes@gmail.com>
> >> Subject: Re: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support
> >> long file names on Windows
> >>
> >> On Tue, 24 May 2022, softworkz wrote:
> >>
> >>> From: softworkz <softworkz@hotmail.com>
> >>>
> >>> Signed-off-by: softworkz <softworkz@hotmail.com>
> >>> ---
> >>> libavformat/os_support.h | 16 +++++++++++-----
> >>> 1 file changed, 11 insertions(+), 5 deletions(-)
> >>>
> >>> diff --git a/libavformat/os_support.h b/libavformat/os_support.h
> >>> index 5e6b32d2dc..d4c07803a5 100644
> >>> --- a/libavformat/os_support.h
> >>> +++ b/libavformat/os_support.h
> >>> @@ -49,7 +49,13 @@
> >>> #  ifdef stat
> >>> #   undef stat
> >>> #  endif
> >>> -#  define stat _stati64
> >>> +#  define stat win32_stat
> >>> +
> >>> +    struct win32_stat
> >>> +    {
> >>> +        struct _stati64;
> >>> +    };
> >>
> >> Is it possible to work around this issue by doing "#define stat(a,b)"
> >> which only should apply on the function, not to the struct?
> >
> > How could this be possible? A define is only doing string replacements,
> > so I wouldn't know how it could be restricted to the function, but
> > not the struct.
> 
> If unsure about a tool feature, please try it out for yourself.

I did :-)
(very extensively in fact)

> Yes, a
> define is only a string replacement, but a define with parameters only
> matches the string occurs with parenthesis afterwards. 

Yes, that's true, but we need to rename both, the function and the
struct, not just the function.


Your example doesn't quite match the situation.

// Let's start with the pre-requisites which cannot be changed
// Posix definitions (stat.h)
struct stat {
 	int a, b, c;
};

// This is the regular definition
int stat(char *fileName, struct stat *par)

// With your define, we replace the function, but not the 
// struct. So we would call the WinAPI function with 
// the wrong struct.
#define stat(a, b) win32_stat(a, b)

// I don't know why you have this in your example at
// this place. It comes from some include file and
// it is not the one we need to use.
// if you mean to redefine it here: that was my previous
// approach. You need to copy the struct (but rename it)
struct stat {
 	int a, b, c;
};

// ----------------- other example:

// This function needs to call the windows function,
// so we cannot use the regular stat struct as parameter
void win32_stat(struct stat *st, const char* filename);



> So here, the stat -> win32_stat rewrite only applied on the function
> declaration and call, but not on the structs.

Exactly, but the struct must be rewritten as well and this is not
possible. 

Neither this:

#define stat _stati64
#define stat(a, b) win32_stat(a, b)

nor this:

#define stat(a, b) win32_stat(a, b)
#define stat _stati64

is working (yes, I tried ;-)

> This, as the article itself clearly declares, is a C language extension.
> GCC allows it in mingw mode

Yes I had read about that.

> but Clang doesn't. (It's possible to use it
> in Clang too if you enable it with -fms-extensions though.)

Is it possible to compile ffmpeg for Windows using Clang?
And if yes, does it even work without that flag?
(assuming it was introduced in order to be able to 
compile Windows stuff).


Thanks again,
sw
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support long file names on Windows

[Martin Storsjö]

On Tue, 24 May 2022, Soft Works wrote:

>> -----Original Message-----
>> From: Martin Storsjö <martin@martin.st>
>> Sent: Tuesday, May 24, 2022 12:26 PM
>> To: Soft Works <softworkz@hotmail.com>
>> Cc: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>;
>> Hendrik Leppkes <h.leppkes@gmail.com>
>> Subject: RE: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support
>> long file names on Windows
>>
>> On Tue, 24 May 2022, Soft Works wrote:
>>
>>>> -----Original Message-----
>>>> From: Martin Storsjö <martin@martin.st>
>>>> Sent: Tuesday, May 24, 2022 11:23 AM
>>>> To: FFmpeg development discussions and patches <ffmpeg-
>> devel@ffmpeg.org>
>>>> Cc: softworkz <softworkz@hotmail.com>; Hendrik Leppkes
>>>> <h.leppkes@gmail.com>
>>>> Subject: Re: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support
>>>> long file names on Windows
>>>>
>>>> On Tue, 24 May 2022, softworkz wrote:
>>>>
>>>>> From: softworkz <softworkz@hotmail.com>
>>>>>
>>>>> Signed-off-by: softworkz <softworkz@hotmail.com>
>>>>> ---
>>>>> libavformat/os_support.h | 16 +++++++++++-----
>>>>> 1 file changed, 11 insertions(+), 5 deletions(-)
>>>>>
>>>>> diff --git a/libavformat/os_support.h b/libavformat/os_support.h
>>>>> index 5e6b32d2dc..d4c07803a5 100644
>>>>> --- a/libavformat/os_support.h
>>>>> +++ b/libavformat/os_support.h
>>>>> @@ -49,7 +49,13 @@
>>>>> #  ifdef stat
>>>>> #   undef stat
>>>>> #  endif
>>>>> -#  define stat _stati64
>>>>> +#  define stat win32_stat
>>>>> +
>>>>> +    struct win32_stat
>>>>> +    {
>>>>> +        struct _stati64;
>>>>> +    };
>>>>
>>>> Is it possible to work around this issue by doing "#define stat(a,b)"
>>>> which only should apply on the function, not to the struct?
>>>
>>> How could this be possible? A define is only doing string replacements,
>>> so I wouldn't know how it could be restricted to the function, but
>>> not the struct.
>>
>> If unsure about a tool feature, please try it out for yourself.
>
> I did :-)
> (very extensively in fact)
>
>> Yes, a
>> define is only a string replacement, but a define with parameters only
>> matches the string occurs with parenthesis afterwards.
>
> Yes, that's true, but we need to rename both, the function and the
> struct, not just the function.


I know. But you said:

> How could this be possible? A define is only doing string replacements, 
> so I wouldn't know how it could be restricted to the function, but not 
> the struct.

And I showed how a define can apply to only one but not the other. Which 
seemed to be news to in your prior mail.

Note how I also said:

"Not saying that this necessarily is the way forward, but I was just 
mentioning it as a potential option to consider."

> Your example doesn't quite match the situation.

Yes I know.

I just brought it up as a possibly thing for discussion, and you derailed 
it by discussing whether it even works. Yes it works, but after looking 
more into it, I agree that it probably won't help in this situation.

>> but Clang doesn't. (It's possible to use it
>> in Clang too if you enable it with -fms-extensions though.)
>
> Is it possible to compile ffmpeg for Windows using Clang?
> And if yes, does it even work without that flag?
> (assuming it was introduced in order to be able to
> compile Windows stuff).

Yes, it is possible to build it with Clang without any custom extra flags 
to enable nondefault modes. In fact, it's tested continuously on FATE too:

http://fate.ffmpeg.org/history.cgi?slot=x86_64-mingw32-clang-trunk

Also for other architectures, e.g.:

http://fate.ffmpeg.org/history.cgi?slot=aarch64-mingw32-clang-trunk

// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[nil-admirari]

> If we would
> want to get rid of this, we could define the struct as follows:
>
> struct win32_stat
> {
> union
> {
> struct _stati64;
> struct _stati64 stat;
> };
> };
>
> The union is anonymous and includes _stati64 twice: once anonymous
> and once named.
>
> This would allow us to define our win32_stat function like this:
>
> static inline int win32_stat(const char *filename_utf8, struct win32_stat *par)
> {
> wchar_t *filename_w;
> int ret;
> if (get_extended_win32_path(filename_utf8, &filename_w))
> return -1;
> if (!filename_w)
> goto fallback;
> ret = _wstat64(filename_w, &par->stat);
> av_free(filename_w);
> return ret;
> fallback:
> return _stat64(filename_utf8, &par->stat);
> }
>
> so it uses the ->stat member for doing the api calls while
> the calling (ffmpeg) code can use the structure as if it was the 
> actual POSIX stat structure.

I'm fine with anonymous union and a rewrite of win32_stat.
But, anonymous unions are a C11 feature: https://en.cppreference.com/w/c/language/union,
and C11 is apparently not allowed https://ffmpeg.org/developer.html#C-language-features.



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: Martin Storsjö <martin@martin.st>
> Sent: Tuesday, May 24, 2022 1:26 PM
> To: Soft Works <softworkz@hotmail.com>
> Cc: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>;
> Hendrik Leppkes <h.leppkes@gmail.com>
> Subject: RE: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support
> long file names on Windows
> 
> On Tue, 24 May 2022, Soft Works wrote:
> 
> >> -----Original Message-----
> >> From: Martin Storsjö <martin@martin.st>
> >> Sent: Tuesday, May 24, 2022 12:26 PM
> >> To: Soft Works <softworkz@hotmail.com>
> >> Cc: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>;
> >> Hendrik Leppkes <h.leppkes@gmail.com>
> >> Subject: RE: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support
> >> long file names on Windows
> >>
> >> On Tue, 24 May 2022, Soft Works wrote:
> >>
> >>>> -----Original Message-----
> >>>> From: Martin Storsjö <martin@martin.st>
> >>>> Sent: Tuesday, May 24, 2022 11:23 AM
> >>>> To: FFmpeg development discussions and patches <ffmpeg-
> >> devel@ffmpeg.org>
> >>>> Cc: softworkz <softworkz@hotmail.com>; Hendrik Leppkes
> >>>> <h.leppkes@gmail.com>
> >>>> Subject: Re: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support:
> Support
> >>>> long file names on Windows
> >>>>
> >>>> On Tue, 24 May 2022, softworkz wrote:
> >>>>
> >>>>> From: softworkz <softworkz@hotmail.com>
> >>>>>
> >>>>> Signed-off-by: softworkz <softworkz@hotmail.com>
> >>>>> ---
> >>>>> libavformat/os_support.h | 16 +++++++++++-----
> >>>>> 1 file changed, 11 insertions(+), 5 deletions(-)
> >>>>>
> >>>>> diff --git a/libavformat/os_support.h b/libavformat/os_support.h
> >>>>> index 5e6b32d2dc..d4c07803a5 100644
> >>>>> --- a/libavformat/os_support.h
> >>>>> +++ b/libavformat/os_support.h
> >>>>> @@ -49,7 +49,13 @@
> >>>>> #  ifdef stat
> >>>>> #   undef stat
> >>>>> #  endif
> >>>>> -#  define stat _stati64
> >>>>> +#  define stat win32_stat
> >>>>> +
> >>>>> +    struct win32_stat
> >>>>> +    {
> >>>>> +        struct _stati64;
> >>>>> +    };
> >>>>
> >>>> Is it possible to work around this issue by doing "#define stat(a,b)"
> >>>> which only should apply on the function, not to the struct?
> >>>
> >>> How could this be possible? A define is only doing string
> replacements,
> >>> so I wouldn't know how it could be restricted to the function, but
> >>> not the struct.
> >>
> >> If unsure about a tool feature, please try it out for yourself.
> >
> > I did :-)
> > (very extensively in fact)
> >
> >> Yes, a
> >> define is only a string replacement, but a define with parameters only
> >> matches the string occurs with parenthesis afterwards.
> >
> > Yes, that's true, but we need to rename both, the function and the
> > struct, not just the function.
> 
> 
> I know. But you said:
> 
> > How could this be possible? A define is only doing string replacements,
> > so I wouldn't know how it could be restricted to the function, but not
> > the struct.
> 
> And I showed how a define can apply to only one but not the other. Which
> seemed to be news to in your prior mail.

Alright yea - thanks for pointing this out. I knew about this kind of
macros, but I hadn't taken them into account in this context because
all my attempts were focusing on the struct side and getting this
separated with typedef and #undef and those things, so it was indeed
opening up a dimension I hadn't considered.
Thanks for the example!


> Note how I also said:
> 
> "Not saying that this necessarily is the way forward, but I was just
> mentioning it as a potential option to consider."
> 
> > Your example doesn't quite match the situation.
> 
> Yes I know.

Sorry, I had understood your message in a way that you would be saying
that it would be working and just not a nice solution.


> I just brought it up as a possibly thing for discussion, and you derailed
> it by discussing whether it even works. 

Nah - I meant whether it works for the given situation, I had no doubt
that your example is working as-is.

 
> >> but Clang doesn't. (It's possible to use it
> >> in Clang too if you enable it with -fms-extensions though.)
> >
> > Is it possible to compile ffmpeg for Windows using Clang?
> > And if yes, does it even work without that flag?
> > (assuming it was introduced in order to be able to
> > compile Windows stuff).
> 
> Yes, it is possible to build it with Clang without any custom extra flags
> to enable nondefault modes. In fact, it's tested continuously on FATE too:
> 
> http://fate.ffmpeg.org/history.cgi?slot=x86_64-mingw32-clang-trunk
> 
> Also for other architectures, e.g.:
> 
> http://fate.ffmpeg.org/history.cgi?slot=aarch64-mingw32-clang-trunk


OK, thanks for the pointers. I'm not sure whether it would be 
acceptable to require this compilation flag for Windows builds?

Can you think of any other ideas?

Thank you very much,
softworkz
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support long file names on Windows

[Martin Storsjö]

On Tue, 24 May 2022, Soft Works wrote:

>> -----Original Message-----
>> From: Martin Storsjö <martin@martin.st>
>> Sent: Tuesday, May 24, 2022 1:26 PM
>> To: Soft Works <softworkz@hotmail.com>
>> Cc: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>;
>> Hendrik Leppkes <h.leppkes@gmail.com>
>> Subject: RE: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support
>> long file names on Windows
>>
>> On Tue, 24 May 2022, Soft Works wrote:
>>
>>>> but Clang doesn't. (It's possible to use it
>>>> in Clang too if you enable it with -fms-extensions though.)
>>>
>>> Is it possible to compile ffmpeg for Windows using Clang?
>>> And if yes, does it even work without that flag?
>>> (assuming it was introduced in order to be able to
>>> compile Windows stuff).
>>
>> Yes, it is possible to build it with Clang without any custom extra flags
>> to enable nondefault modes. In fact, it's tested continuously on FATE too:
>>
>> http://fate.ffmpeg.org/history.cgi?slot=x86_64-mingw32-clang-trunk
>>
>> Also for other architectures, e.g.:
>>
>> http://fate.ffmpeg.org/history.cgi?slot=aarch64-mingw32-clang-trunk
>
>
> OK, thanks for the pointers. I'm not sure whether it would be
> acceptable to require this compilation flag for Windows builds?

I would very much prefer not to require adding -fms-extensions when 
building with Clang - that option unlocks a lot of stuff that we generally 
shouldn't be enabling.

> Can you think of any other ideas?

Right now, mainly doing a #define ff_stat_struct which would require 
updating the calling code. It's not ideal but worse things have been done 
anyway (there's not that many stat calls).

I was exploring the idea of just redefining the struct, but e.g. "typedef 
struct _stati64 win32_stat", but that only works when referring to the 
type as "win32_stat", not "struct win32_stat". So that doesn't seem like a 
good path forward either.

I'd prefer to slow down and think more about other alternatives here, 
rather than rushing forward with adding -fms-extensions.


Also note that currently, we don't even have a proper automatic redirect 
from stat to win32_stat, see the ifdef in libavformat/file.c. So this is 
new development, or, raising the bar even further, in one sense.


// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Martin
> Storsjö
> Sent: Tuesday, May 24, 2022 2:44 PM
> To: Soft Works <softworkz@hotmail.com>
> Cc: Hendrik Leppkes <h.leppkes@gmail.com>; FFmpeg development discussions
> and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support
> long file names on Windows
> 
> On Tue, 24 May 2022, Soft Works wrote:
> 
> >> -----Original Message-----
> >> From: Martin Storsjö <martin@martin.st>
> >> Sent: Tuesday, May 24, 2022 1:26 PM
> >> To: Soft Works <softworkz@hotmail.com>
> >> Cc: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>;
> >> Hendrik Leppkes <h.leppkes@gmail.com>
> >> Subject: RE: [FFmpeg-devel] [PATCH v5 2/2] avformat/os_support: Support
> >> long file names on Windows
> >>
> >> On Tue, 24 May 2022, Soft Works wrote:
> >>
> >>>> but Clang doesn't. (It's possible to use it
> >>>> in Clang too if you enable it with -fms-extensions though.)
> >>>
> >>> Is it possible to compile ffmpeg for Windows using Clang?
> >>> And if yes, does it even work without that flag?
> >>> (assuming it was introduced in order to be able to
> >>> compile Windows stuff).
> >>
> >> Yes, it is possible to build it with Clang without any custom extra
> flags
> >> to enable nondefault modes. In fact, it's tested continuously on FATE
> too:
> >>
> >> http://fate.ffmpeg.org/history.cgi?slot=x86_64-mingw32-clang-trunk
> >>
> >> Also for other architectures, e.g.:
> >>
> >> http://fate.ffmpeg.org/history.cgi?slot=aarch64-mingw32-clang-trunk
> >
> >
> > OK, thanks for the pointers. I'm not sure whether it would be
> > acceptable to require this compilation flag for Windows builds?
> 
> I would very much prefer not to require adding -fms-extensions when
> building with Clang - that option unlocks a lot of stuff that we generally
> shouldn't be enabling.

OK, sure, it always smells when doing something like that just to
achieve a single thing.

> 
> > Can you think of any other ideas?
> 
> Right now, mainly doing a #define ff_stat_struct which would require
> updating the calling code. It's not ideal but worse things have been done
> anyway (there's not that many stat calls).
> 
> I was exploring the idea of just redefining the struct, but e.g. "typedef
> struct _stati64 win32_stat", but that only works when referring to the
> type as "win32_stat", not "struct win32_stat". So that doesn't seem like a
> good path forward either.
> 
> I'd prefer to slow down and think more about other alternatives here,
> rather than rushing forward with adding -fms-extensions.

I have a new idea, see below

> Also note that currently, we don't even have a proper automatic redirect
> from stat to win32_stat, see the ifdef in libavformat/file.c. 

Yes, that can be dropped (once we got it)...


What do you think of the following:

We could define our own win32_stat struct, but not in a way that matches
the Windows API, just matching the POSIX definition (like the consuming 
code expects), e.g.:

struct win_32stat {
    dev_t          st_dev;     /* ID of device containing file */
    ino_t          st_ino;     /* inode number */
    unsigned short st_mode;    /* protection */
    short          st_nlink;   /* number of hard links */
    short          st_uid;     /* user ID of owner */
    short          st_gid;     /* group ID of owner */
    dev_t          st_rdev;    /* device ID (if special file) */
    off_t          st_size;    /* total size, in bytes */
    time_t         st_atime;   /* time of last access */
    time_t         st_mtime;   /* time of last modification */
    time_t         st_ctime;   /* time of last status change */
};

And then, in our win32_stat() function, we call the win api with
the "right" struct and simply copy over the values..:

static int win32_stat(const char *filename_utf8, struct stat *par)
{
    wchar_t *filename_w;
    int ret;
    struct _stati64 winstat;

    if (get_extended_win32_path(filename_utf8, &filename_w))
        return -1;

    if (filename_w) {
        ret = _wstat64(filename_w, &winstat);
        av_free(filename_w);
    } else
        ret = _stat64(filename_utf8, &winstat);

    par->st_dev   = winstat.st_dev;
    par->st_ino   = winstat.st_ino;
    par->st_mode  = winstat.st_mode;
    par->st_nlink = winstat.st_nlink;
    par->st_uid   = winstat.st_uid;
    par->st_gid   = winstat.st_gid;
    par->st_rdev  = winstat.st_rdev;
    par->st_size  = winstat.st_size;
    par->st_atime = winstat.st_atime;
    par->st_mtime = winstat.st_mtime;
    par->st_ctime = winstat.st_ctime;

    return ret;
}

This would be safe and without any weirdness (just a bit more
code).

What do you think about it?


Thanks,
sw




_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v6 0/2] Support long file names on Windows

[ffmpegagent]

This patchset adds support for long file and directory paths on Windows. The
implementation follows the same logic that .NET is using internally, with
the only exception that it doesn't expand short path components in 8.3
format. .NET does this as the same function is also used for other purposes,
but in our case, that's not required. Short (8.3) paths are working as well
with the extended path prefix, even when longer than 260.

Successfully tested:

 * Regular paths wth drive letter
 * Regular UNC paths
 * Long paths wth drive letter
 * Long paths wth drive letter and forward slashes
 * Long UNC paths
 * Prefixed paths wth drive letter
 * Prefixed UNC paths

I have kept the individual functions separate on purpose, to make it easy to
compare with the .NET impl. (compilers should inlinie those anyway)

v2

 * wchar_filename: Improve comments and function documentation
 * os_support: adjust defines to use win32_stat

v3

 * removed length check in path_is_extended()
 * added path_is_device_path() check in add_extended_prefix()
 * add_extended_prefix(): clarified doc and add checks
 * clarified string allocation length calculation
 * replaced 260 with MAX_PATH
 * removed redundant checks after normalization

v4

 * rebased. no changes

v5

 * resolved the ugly struct duplication
 * compatible with _USE_32BIT_TIME_T

v6

 * wchar_filename.h: added links to .NET source code
 * wchar_filename.h: free allocations on error
 * os_support.hs: use clean and safe way to redirect stat() calls

softworkz (2):
  avutil/wchar_filename,file_open: Support long file names on Windows
  avformat/os_support: Support long file names on Windows

 libavformat/os_support.h   |  87 +++++++++++++-----
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 180 +++++++++++++++++++++++++++++++++++++
 3 files changed, 244 insertions(+), 25 deletions(-)


base-commit: 6076dbcb55d0c9b6693d1acad12a63f7268301aa
Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-28%2Fsoftworkz%2Fsubmit_long_filenames-v6
Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v6
Pull-Request: https://github.com/ffstaging/FFmpeg/pull/28

Range-diff vs v5:

 1:  13118dc1fa ! 1:  960aa795ff avutil/wchar_filename,file_open: Support long file names on Windows
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      +/**
      + * Checks for extended path prefixes for which normalization needs to be skipped.
      + * see .NET6: PathInternal.IsExtended()
     ++ * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L165
      + */
      +static inline int path_is_extended(const wchar_t *path)
      +{
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      +
      +/**
      + * Checks for a device path prefix.
     -+ * see .NET6: PathInternal.IsDevicePath()
     ++ * see .NET6: PathInternal.IsDevice()
     ++ * we don't check forward slashes and extended paths (as already done)
     ++ * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L132
      + */
      +static inline int path_is_device_path(const wchar_t *path)
      +{
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      +/**
      + * Performs path normalization by calling GetFullPathNameW().
      + * see .NET6: PathHelper.GetFullPathName()
     ++ * https://github.com/dotnet/runtime/blob/2a99e18eedabcf1add064c099da59d9301ce45e0/src/libraries/System.Private.CoreLib/src/System/IO/PathHelper.Windows.cs#L70
      + */
      +static inline int get_full_path_name(wchar_t **ppath_w)
      +{
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      +
      +    num_chars = GetFullPathNameW(*ppath_w, num_chars, temp_w, NULL);
      +    if (num_chars <= 0) {
     ++        av_free(temp_w);
      +        errno = EINVAL;
      +        return -1;
      +    }
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      + * Normalizes a Windows file or folder path.
      + * Expansion of short paths (with 8.3 path components) is currently omitted
      + * as it is not required for accessing long paths.
     -+ * see .NET6: PathHelper.Normalize().
     ++ * see .NET6: PathHelper.Normalize()
     ++ * https://github.com/dotnet/runtime/blob/2a99e18eedabcf1add064c099da59d9301ce45e0/src/libraries/System.Private.CoreLib/src/System/IO/PathHelper.Windows.cs#L25
      + */
      +static inline int path_normalize(wchar_t **ppath_w)
      +{
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      + * This function expects that the path has been normalized before by
      + * calling path_normalize() and it doesn't check whether the path is
      + * actually long (> MAX_PATH).
     -+ * see .NET6: PathInternal.EnsureExtendedPrefix() *
     ++ * see .NET6: PathInternal.EnsureExtendedPrefix()
     ++ * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L107
      + */
      +static inline int add_extended_prefix(wchar_t **ppath_w)
      +{
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      + * Longs paths (>= MAX_PATH) are prefixed with the extended path or extended
      + * UNC path prefix.
      + * see .NET6: Path.GetFullPath() and Path.GetFullPathInternal()
     ++ * https://github.com/dotnet/runtime/blob/2a99e18eedabcf1add064c099da59d9301ce45e0/src/libraries/System.Private.CoreLib/src/System/IO/Path.Windows.cs#L126
      + */
      +static inline int get_extended_win32_path(const char *path, wchar_t **ppath_w)
      +{
     @@ libavutil/wchar_filename.h: static inline int utf8towchar(const char *filename_u
      +        return 0;
      +    }
      +
     -+    if ((ret = path_normalize(ppath_w)) < 0)
     ++    if ((ret = path_normalize(ppath_w)) < 0) {
     ++        av_freep(ppath_w);
      +        return ret;
     ++    }
      +
     -+    // see .NET6: PathInternal.EnsureExtendedPrefixIfNeeded()
     ++    /* see .NET6: PathInternal.EnsureExtendedPrefixIfNeeded()
     ++     * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L92
     ++     */
      +    len = wcslen(*ppath_w);
      +    if (len >= MAX_PATH) {
     -+        if ((ret = add_extended_prefix(ppath_w)) < 0)
     ++        if ((ret = add_extended_prefix(ppath_w)) < 0) {
     ++            av_freep(ppath_w);
      +            return ret;
     ++        }
      +    }
      +
      +    return 0;
 2:  5313aeec0e ! 2:  6f8d400db7 avformat/os_support: Support long file names on Windows
     @@ libavformat/os_support.h
       #   undef stat
       #  endif
      -#  define stat _stati64
     ++
      +#  define stat win32_stat
      +
      +    struct win32_stat
      +    {
     -+        struct _stati64;
     ++        _dev_t         st_dev;     /* ID of device containing file */
     ++        _ino_t         st_ino;     /* inode number */
     ++        unsigned short st_mode;    /* protection */
     ++        short          st_nlink;   /* number of hard links */
     ++        short          st_uid;     /* user ID of owner */
     ++        short          st_gid;     /* group ID of owner */
     ++        _dev_t         st_rdev;    /* device ID (if special file) */
     ++        long           st_size;    /* total size, in bytes */
     ++        time_t         st_atime;   /* time of last access */
     ++        time_t         st_mtime;   /* time of last modification */
     ++        time_t         st_ctime;   /* time of last status change */
      +    };
      +
       #  ifdef fstat
     @@ libavformat/os_support.h: static inline int win32_##name(const char *filename_ut
               return -1;                                        \
           if (!filename_w)                                      \
               goto fallback;                                    \
     -@@ libavformat/os_support.h: static inline int win32_##name(const char *filename_utf8, partype par) \
     -     wchar_t *filename_w;                                  \
     -     int ret;                                              \
     -                                                           \
     +@@ libavformat/os_support.h: DEF_FS_FUNCTION(unlink, _wunlink, _unlink)
     + DEF_FS_FUNCTION(mkdir,  _wmkdir,  _mkdir)
     + DEF_FS_FUNCTION(rmdir,  _wrmdir , _rmdir)
     + 
     +-#define DEF_FS_FUNCTION2(name, wfunc, afunc, partype)     \
     +-static inline int win32_##name(const char *filename_utf8, partype par) \
     +-{                                                         \
     +-    wchar_t *filename_w;                                  \
     +-    int ret;                                              \
     +-                                                          \
      -    if (utf8towchar(filename_utf8, &filename_w))          \
     -+    if (get_extended_win32_path(filename_utf8, &filename_w)) \
     -         return -1;                                        \
     -     if (!filename_w)                                      \
     -         goto fallback;                                    \
     -@@ libavformat/os_support.h: static inline int win32_rename(const char *src_utf8, const char *dest_utf8)
     +-        return -1;                                        \
     +-    if (!filename_w)                                      \
     +-        goto fallback;                                    \
     +-                                                          \
     +-    ret = wfunc(filename_w, par);                         \
     +-    av_free(filename_w);                                  \
     +-    return ret;                                           \
     +-                                                          \
     +-fallback:                                                 \
     +-    /* filename may be be in CP_ACP */                    \
     +-    return afunc(filename_utf8, par);                     \
     ++static inline int win32_access(const char *filename_utf8, int par)
     ++{
     ++    wchar_t *filename_w;
     ++    int ret;
     ++    if (get_extended_win32_path(filename_utf8, &filename_w))
     ++        return -1;
     ++    if (!filename_w)
     ++        goto fallback;
     ++    ret = _waccess(filename_w, par);
     ++    av_free(filename_w);
     ++    return ret;
     ++fallback:
     ++    return _access(filename_utf8, par);
     + }
     + 
     +-DEF_FS_FUNCTION2(access, _waccess, _access, int)
     +-DEF_FS_FUNCTION2(stat, _wstati64, _stati64, struct stat*)
     ++static inline int win32_stat(const char *filename_utf8, struct stat *par)
     ++{
     ++    wchar_t *filename_w;
     ++    int ret;
     ++    struct _stati64 winstat = { 0 };
     ++
     ++    if (get_extended_win32_path(filename_utf8, &filename_w))
     ++        return -1;
     ++
     ++    if (filename_w) {
     ++        ret = _wstat64(filename_w, &winstat);
     ++        av_free(filename_w);
     ++    } else
     ++        ret = _stat64(filename_utf8, &winstat);
     ++
     ++    par->st_dev   = winstat.st_dev;
     ++    par->st_ino   = winstat.st_ino;
     ++    par->st_mode  = winstat.st_mode;
     ++    par->st_nlink = winstat.st_nlink;
     ++    par->st_uid   = winstat.st_uid;
     ++    par->st_gid   = winstat.st_gid;
     ++    par->st_rdev  = winstat.st_rdev;
     ++    par->st_size  = winstat.st_size;
     ++    par->st_atime = winstat.st_atime;
     ++    par->st_mtime = winstat.st_mtime;
     ++    par->st_ctime = winstat.st_ctime;
     ++
     ++    return ret;
     ++}
     + 
     + static inline int win32_rename(const char *src_utf8, const char *dest_utf8)
     + {
           wchar_t *src_w, *dest_w;
           int ret;
       

-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v6 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 180 +++++++++++++++++++++++++++++++++++++
 2 files changed, 181 insertions(+), 1 deletion(-)

diff --git a/libavutil/file_open.c b/libavutil/file_open.c
index fb64c2e4ee..58a6073353 100644
--- a/libavutil/file_open.c
+++ b/libavutil/file_open.c
@@ -45,7 +45,7 @@ static int win32_open(const char *filename_utf8, int oflag, int pmode)
     wchar_t *filename_w;
 
     /* convert UTF-8 to wide chars */
-    if (utf8towchar(filename_utf8, &filename_w))
+    if (get_extended_win32_path(filename_utf8, &filename_w))
         return -1;
     if (!filename_w)
         goto fallback;
diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h
index 90f082452c..f36d9dfea3 100644
--- a/libavutil/wchar_filename.h
+++ b/libavutil/wchar_filename.h
@@ -40,6 +40,186 @@ static inline int utf8towchar(const char *filename_utf8, wchar_t **filename_w)
     MultiByteToWideChar(CP_UTF8, 0, filename_utf8, -1, *filename_w, num_chars);
     return 0;
 }
+
+/**
+ * Checks for extended path prefixes for which normalization needs to be skipped.
+ * see .NET6: PathInternal.IsExtended()
+ * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L165
+ */
+static inline int path_is_extended(const wchar_t *path)
+{
+    if (path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\')
+        return 1;
+
+    return 0;
+}
+
+/**
+ * Checks for a device path prefix.
+ * see .NET6: PathInternal.IsDevice()
+ * we don't check forward slashes and extended paths (as already done)
+ * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L132
+ */
+static inline int path_is_device_path(const wchar_t *path)
+{
+    if (path[0] == L'\\' && path[1] == L'\\' && path[2] == L'.' && path[3] == L'\\')
+        return 1;
+
+    return 0;
+}
+
+/**
+ * Performs path normalization by calling GetFullPathNameW().
+ * see .NET6: PathHelper.GetFullPathName()
+ * https://github.com/dotnet/runtime/blob/2a99e18eedabcf1add064c099da59d9301ce45e0/src/libraries/System.Private.CoreLib/src/System/IO/PathHelper.Windows.cs#L70
+ */
+static inline int get_full_path_name(wchar_t **ppath_w)
+{
+    int num_chars;
+    wchar_t *temp_w;
+
+    num_chars = GetFullPathNameW(*ppath_w, 0, NULL, NULL);
+    if (num_chars <= 0) {
+        errno = EINVAL;
+        return -1;
+    }
+
+    temp_w = (wchar_t *)av_calloc(num_chars, sizeof(wchar_t));
+    if (!temp_w) {
+        errno = ENOMEM;
+        return -1;
+    }
+
+    num_chars = GetFullPathNameW(*ppath_w, num_chars, temp_w, NULL);
+    if (num_chars <= 0) {
+        av_free(temp_w);
+        errno = EINVAL;
+        return -1;
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+/**
+ * Normalizes a Windows file or folder path.
+ * Expansion of short paths (with 8.3 path components) is currently omitted
+ * as it is not required for accessing long paths.
+ * see .NET6: PathHelper.Normalize()
+ * https://github.com/dotnet/runtime/blob/2a99e18eedabcf1add064c099da59d9301ce45e0/src/libraries/System.Private.CoreLib/src/System/IO/PathHelper.Windows.cs#L25
+ */
+static inline int path_normalize(wchar_t **ppath_w)
+{
+    int ret;
+
+    if ((ret = get_full_path_name(ppath_w)) < 0)
+        return ret;
+
+    /* What .NET does at this point is to call PathHelper.TryExpandShortFileName()
+     * in case the path contains a '~' character.
+     * We don't need to do this as we don't need to normalize the file name
+     * for presentation, and the extended path prefix works with 8.3 path
+     * components as well
+     */
+    return 0;
+}
+
+/**
+ * Adds an extended path or UNC prefix to longs paths or paths ending
+ * with a space or a dot. (' ' or '.').
+ * This function expects that the path has been normalized before by
+ * calling path_normalize() and it doesn't check whether the path is
+ * actually long (> MAX_PATH).
+ * see .NET6: PathInternal.EnsureExtendedPrefix()
+ * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L107
+ */
+static inline int add_extended_prefix(wchar_t **ppath_w)
+{
+    const wchar_t *unc_prefix           = L"\\\\?\\UNC\\";
+    const wchar_t *extended_path_prefix = L"\\\\?\\";
+    const wchar_t *path_w               = *ppath_w;
+    const size_t len                    = wcslen(path_w);
+    wchar_t *temp_w;
+
+    /* We're skipping the check IsPartiallyQualified() because
+     * we expect to have called GetFullPathNameW() already. */
+    if (len < 2 || path_is_extended(*ppath_w) || path_is_device_path(*ppath_w)) {
+        return 0;
+    }
+
+    if (path_w[0] == L'\\' && path_w[1] == L'\\') {
+        /* unc_prefix length is 8 plus 1 for terminating zeros,
+         * we subtract 2 for the leading '\\' of the original path */
+        temp_w = (wchar_t *)av_calloc(len - 2 + 8 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, unc_prefix);
+        wcscat(temp_w, path_w + 2);
+    } else {
+        // The length of extended_path_prefix is 4 plus 1 for terminating zeros
+        temp_w = (wchar_t *)av_calloc(len + 4 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, extended_path_prefix);
+        wcscat(temp_w, path_w);
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+/**
+ * Converts a file or folder path to wchar_t for use with Windows file
+ * APIs. Paths with extended path prefix (either '\\?\' or \??\') are
+ * left unchanged.
+ * All other paths are normalized and converted to absolute paths.
+ * Longs paths (>= MAX_PATH) are prefixed with the extended path or extended
+ * UNC path prefix.
+ * see .NET6: Path.GetFullPath() and Path.GetFullPathInternal()
+ * https://github.com/dotnet/runtime/blob/2a99e18eedabcf1add064c099da59d9301ce45e0/src/libraries/System.Private.CoreLib/src/System/IO/Path.Windows.cs#L126
+ */
+static inline int get_extended_win32_path(const char *path, wchar_t **ppath_w)
+{
+    int ret;
+    size_t len;
+
+    if ((ret = utf8towchar(path, ppath_w)) < 0)
+        return ret;
+
+    if (path_is_extended(*ppath_w)) {
+        /* Paths prefixed with '\\?\' or \??\' are considered normalized by definition.
+         * Windows doesn't normalize those paths and neither should we.
+         */
+        return 0;
+    }
+
+    if ((ret = path_normalize(ppath_w)) < 0) {
+        av_freep(ppath_w);
+        return ret;
+    }
+
+    /* see .NET6: PathInternal.EnsureExtendedPrefixIfNeeded()
+     * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L92
+     */
+    len = wcslen(*ppath_w);
+    if (len >= MAX_PATH) {
+        if ((ret = add_extended_prefix(ppath_w)) < 0) {
+            av_freep(ppath_w);
+            return ret;
+        }
+    }
+
+    return 0;
+}
+
 #endif
 
 #endif /* AVUTIL_WCHAR_FILENAME_H */
-- 
ffmpeg-codebot

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v6 2/2] avformat/os_support: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavformat/os_support.h | 87 +++++++++++++++++++++++++++++-----------
 1 file changed, 63 insertions(+), 24 deletions(-)

diff --git a/libavformat/os_support.h b/libavformat/os_support.h
index 5e6b32d2dc..179b926293 100644
--- a/libavformat/os_support.h
+++ b/libavformat/os_support.h
@@ -49,7 +49,24 @@
 #  ifdef stat
 #   undef stat
 #  endif
-#  define stat _stati64
+
+#  define stat win32_stat
+
+    struct win32_stat
+    {
+        _dev_t         st_dev;     /* ID of device containing file */
+        _ino_t         st_ino;     /* inode number */
+        unsigned short st_mode;    /* protection */
+        short          st_nlink;   /* number of hard links */
+        short          st_uid;     /* user ID of owner */
+        short          st_gid;     /* group ID of owner */
+        _dev_t         st_rdev;    /* device ID (if special file) */
+        long           st_size;    /* total size, in bytes */
+        time_t         st_atime;   /* time of last access */
+        time_t         st_mtime;   /* time of last modification */
+        time_t         st_ctime;   /* time of last status change */
+    };
+
 #  ifdef fstat
 #   undef fstat
 #  endif
@@ -153,7 +170,7 @@ static inline int win32_##name(const char *filename_utf8) \
     wchar_t *filename_w;                                  \
     int ret;                                              \
                                                           \
-    if (utf8towchar(filename_utf8, &filename_w))          \
+    if (get_extended_win32_path(filename_utf8, &filename_w)) \
         return -1;                                        \
     if (!filename_w)                                      \
         goto fallback;                                    \
@@ -171,37 +188,59 @@ DEF_FS_FUNCTION(unlink, _wunlink, _unlink)
 DEF_FS_FUNCTION(mkdir,  _wmkdir,  _mkdir)
 DEF_FS_FUNCTION(rmdir,  _wrmdir , _rmdir)
 
-#define DEF_FS_FUNCTION2(name, wfunc, afunc, partype)     \
-static inline int win32_##name(const char *filename_utf8, partype par) \
-{                                                         \
-    wchar_t *filename_w;                                  \
-    int ret;                                              \
-                                                          \
-    if (utf8towchar(filename_utf8, &filename_w))          \
-        return -1;                                        \
-    if (!filename_w)                                      \
-        goto fallback;                                    \
-                                                          \
-    ret = wfunc(filename_w, par);                         \
-    av_free(filename_w);                                  \
-    return ret;                                           \
-                                                          \
-fallback:                                                 \
-    /* filename may be be in CP_ACP */                    \
-    return afunc(filename_utf8, par);                     \
+static inline int win32_access(const char *filename_utf8, int par)
+{
+    wchar_t *filename_w;
+    int ret;
+    if (get_extended_win32_path(filename_utf8, &filename_w))
+        return -1;
+    if (!filename_w)
+        goto fallback;
+    ret = _waccess(filename_w, par);
+    av_free(filename_w);
+    return ret;
+fallback:
+    return _access(filename_utf8, par);
 }
 
-DEF_FS_FUNCTION2(access, _waccess, _access, int)
-DEF_FS_FUNCTION2(stat, _wstati64, _stati64, struct stat*)
+static inline int win32_stat(const char *filename_utf8, struct stat *par)
+{
+    wchar_t *filename_w;
+    int ret;
+    struct _stati64 winstat = { 0 };
+
+    if (get_extended_win32_path(filename_utf8, &filename_w))
+        return -1;
+
+    if (filename_w) {
+        ret = _wstat64(filename_w, &winstat);
+        av_free(filename_w);
+    } else
+        ret = _stat64(filename_utf8, &winstat);
+
+    par->st_dev   = winstat.st_dev;
+    par->st_ino   = winstat.st_ino;
+    par->st_mode  = winstat.st_mode;
+    par->st_nlink = winstat.st_nlink;
+    par->st_uid   = winstat.st_uid;
+    par->st_gid   = winstat.st_gid;
+    par->st_rdev  = winstat.st_rdev;
+    par->st_size  = winstat.st_size;
+    par->st_atime = winstat.st_atime;
+    par->st_mtime = winstat.st_mtime;
+    par->st_ctime = winstat.st_ctime;
+
+    return ret;
+}
 
 static inline int win32_rename(const char *src_utf8, const char *dest_utf8)
 {
     wchar_t *src_w, *dest_w;
     int ret;
 
-    if (utf8towchar(src_utf8, &src_w))
+    if (get_extended_win32_path(src_utf8, &src_w))
         return -1;
-    if (utf8towchar(dest_utf8, &dest_w)) {
+    if (get_extended_win32_path(dest_utf8, &dest_w)) {
         av_free(src_w);
         return -1;
     }
-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v6 1/2] avutil/wchar_filename, file_open: Support long file names on Windows

[Martin Storsjö]

On Tue, 24 May 2022, softworkz wrote:

> From: softworkz <softworkz@hotmail.com>
>
> Signed-off-by: softworkz <softworkz@hotmail.com>
> ---
> libavutil/file_open.c      |   2 +-
> libavutil/wchar_filename.h | 180 +++++++++++++++++++++++++++++++++++++
> 2 files changed, 181 insertions(+), 1 deletion(-)

This looks ok to me now, thanks!

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v6 2/2] avformat/os_support: Support long file names on Windows

[Martin Storsjö]

On Tue, 24 May 2022, softworkz wrote:

> From: softworkz <softworkz@hotmail.com>
>
> Signed-off-by: softworkz <softworkz@hotmail.com>
> ---
> libavformat/os_support.h | 87 +++++++++++++++++++++++++++++-----------
> 1 file changed, 63 insertions(+), 24 deletions(-)
>
> diff --git a/libavformat/os_support.h b/libavformat/os_support.h
> index 5e6b32d2dc..179b926293 100644
> --- a/libavformat/os_support.h
> +++ b/libavformat/os_support.h
> @@ -49,7 +49,24 @@
> #  ifdef stat
> #   undef stat
> #  endif
> -#  define stat _stati64
> +
> +#  define stat win32_stat
> +
> +    struct win32_stat
> +    {
> +        _dev_t         st_dev;     /* ID of device containing file */
> +        _ino_t         st_ino;     /* inode number */
> +        unsigned short st_mode;    /* protection */
> +        short          st_nlink;   /* number of hard links */
> +        short          st_uid;     /* user ID of owner */
> +        short          st_gid;     /* group ID of owner */
> +        _dev_t         st_rdev;    /* device ID (if special file) */
> +        long           st_size;    /* total size, in bytes */
> +        time_t         st_atime;   /* time of last access */
> +        time_t         st_mtime;   /* time of last modification */
> +        time_t         st_ctime;   /* time of last status change */
> +    };

Please use int64_t for both st_size and st_?time. We already use _stati64 
so far, so we get 64 bit sizes (and long definitely isn't a 64 bit type on 
Windows!), and with int64_t in the outer struct, we won't accidentally 
truncate any valid data that we got from the lower level stat function 
call.



> +
> #  ifdef fstat
> #   undef fstat
> #  endif
> @@ -153,7 +170,7 @@ static inline int win32_##name(const char *filename_utf8) \
>     wchar_t *filename_w;                                  \
>     int ret;                                              \
>                                                           \
> -    if (utf8towchar(filename_utf8, &filename_w))          \
> +    if (get_extended_win32_path(filename_utf8, &filename_w)) \
>         return -1;                                        \
>     if (!filename_w)                                      \
>         goto fallback;                                    \
> @@ -171,37 +188,59 @@ DEF_FS_FUNCTION(unlink, _wunlink, _unlink)
> DEF_FS_FUNCTION(mkdir,  _wmkdir,  _mkdir)
> DEF_FS_FUNCTION(rmdir,  _wrmdir , _rmdir)
>
> -#define DEF_FS_FUNCTION2(name, wfunc, afunc, partype)     \
> -static inline int win32_##name(const char *filename_utf8, partype par) \
> -{                                                         \
> -    wchar_t *filename_w;                                  \
> -    int ret;                                              \
> -                                                          \
> -    if (utf8towchar(filename_utf8, &filename_w))          \
> -        return -1;                                        \
> -    if (!filename_w)                                      \
> -        goto fallback;                                    \
> -                                                          \
> -    ret = wfunc(filename_w, par);                         \
> -    av_free(filename_w);                                  \
> -    return ret;                                           \
> -                                                          \
> -fallback:                                                 \
> -    /* filename may be be in CP_ACP */                    \
> -    return afunc(filename_utf8, par);                     \
> +static inline int win32_access(const char *filename_utf8, int par)
> +{
> +    wchar_t *filename_w;
> +    int ret;
> +    if (get_extended_win32_path(filename_utf8, &filename_w))
> +        return -1;
> +    if (!filename_w)
> +        goto fallback;
> +    ret = _waccess(filename_w, par);
> +    av_free(filename_w);
> +    return ret;
> +fallback:
> +    return _access(filename_utf8, par);
> }
>
> -DEF_FS_FUNCTION2(access, _waccess, _access, int)
> -DEF_FS_FUNCTION2(stat, _wstati64, _stati64, struct stat*)
> +static inline int win32_stat(const char *filename_utf8, struct stat *par)
> +{

Maybe "struct win32_stat" in the parameter here too, for consistency?

> +    wchar_t *filename_w;
> +    int ret;
> +    struct _stati64 winstat = { 0 };
> +
> +    if (get_extended_win32_path(filename_utf8, &filename_w))
> +        return -1;
> +
> +    if (filename_w) {
> +        ret = _wstat64(filename_w, &winstat);
> +        av_free(filename_w);
> +    } else
> +        ret = _stat64(filename_utf8, &winstat);
> +
> +    par->st_dev   = winstat.st_dev;
> +    par->st_ino   = winstat.st_ino;
> +    par->st_mode  = winstat.st_mode;
> +    par->st_nlink = winstat.st_nlink;
> +    par->st_uid   = winstat.st_uid;
> +    par->st_gid   = winstat.st_gid;
> +    par->st_rdev  = winstat.st_rdev;
> +    par->st_size  = winstat.st_size;
> +    par->st_atime = winstat.st_atime;
> +    par->st_mtime = winstat.st_mtime;
> +    par->st_ctime = winstat.st_ctime;

Thanks, this approach seems robust and safe to me!

With this change in place, shouldn't we drop the #ifdef for 
stat/win32_stat in file.c at the same time?

Other than that, this starts to look ok now.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v6 2/2] avformat/os_support: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: Martin Storsjö <martin@martin.st>
> Sent: Tuesday, May 24, 2022 10:59 PM
> To: softworkz <ffmpegagent@gmail.com>
> Cc: ffmpeg-devel@ffmpeg.org; Soft Works <softworkz@hotmail.com>; Hendrik
> Leppkes <h.leppkes@gmail.com>
> Subject: Re: [PATCH v6 2/2] avformat/os_support: Support long file names
> on Windows
> 
> On Tue, 24 May 2022, softworkz wrote:
> 
> > From: softworkz <softworkz@hotmail.com>
> >
> > Signed-off-by: softworkz <softworkz@hotmail.com>
> > ---
> > libavformat/os_support.h | 87 +++++++++++++++++++++++++++++-----------
> > 1 file changed, 63 insertions(+), 24 deletions(-)
> >
> > diff --git a/libavformat/os_support.h b/libavformat/os_support.h
> > index 5e6b32d2dc..179b926293 100644
> > --- a/libavformat/os_support.h
> > +++ b/libavformat/os_support.h
> > @@ -49,7 +49,24 @@
> > #  ifdef stat
> > #   undef stat
> > #  endif
> > -#  define stat _stati64
> > +
> > +#  define stat win32_stat
> > +
> > +    struct win32_stat
> > +    {
> > +        _dev_t         st_dev;     /* ID of device containing file */
> > +        _ino_t         st_ino;     /* inode number */
> > +        unsigned short st_mode;    /* protection */
> > +        short          st_nlink;   /* number of hard links */
> > +        short          st_uid;     /* user ID of owner */
> > +        short          st_gid;     /* group ID of owner */
> > +        _dev_t         st_rdev;    /* device ID (if special file) */
> > +        long           st_size;    /* total size, in bytes */
> > +        time_t         st_atime;   /* time of last access */
> > +        time_t         st_mtime;   /* time of last modification */
> > +        time_t         st_ctime;   /* time of last status change */
> > +    };
> 
> Please use int64_t for both st_size and st_?time. We already use _stati64
> so far, so we get 64 bit sizes (and long definitely isn't a 64 bit type on
> Windows!), and with int64_t in the outer struct, we won't accidentally
> truncate any valid data that we got from the lower level stat function
> call.

I came to long by looking up _off_t in the Windows headers, but you are 
right: as we're explicitly using _stat64, we'll get int64 st_size values,
even on 32bit Windows.

Done.

> > +
> > #  ifdef fstat
> > #   undef fstat
> > #  endif
> > @@ -153,7 +170,7 @@ static inline int win32_##name(const char
> *filename_utf8) \
> >     wchar_t *filename_w;                                  \
> >     int ret;                                              \
> >                                                           \
> > -    if (utf8towchar(filename_utf8, &filename_w))          \
> > +    if (get_extended_win32_path(filename_utf8, &filename_w)) \
> >         return -1;                                        \
> >     if (!filename_w)                                      \
> >         goto fallback;                                    \
> > @@ -171,37 +188,59 @@ DEF_FS_FUNCTION(unlink, _wunlink, _unlink)
> > DEF_FS_FUNCTION(mkdir,  _wmkdir,  _mkdir)
> > DEF_FS_FUNCTION(rmdir,  _wrmdir , _rmdir)
> >
> > -#define DEF_FS_FUNCTION2(name, wfunc, afunc, partype)     \
> > -static inline int win32_##name(const char *filename_utf8, partype par)
> \
> > -{                                                         \
> > -    wchar_t *filename_w;                                  \
> > -    int ret;                                              \
> > -                                                          \
> > -    if (utf8towchar(filename_utf8, &filename_w))          \
> > -        return -1;                                        \
> > -    if (!filename_w)                                      \
> > -        goto fallback;                                    \
> > -                                                          \
> > -    ret = wfunc(filename_w, par);                         \
> > -    av_free(filename_w);                                  \
> > -    return ret;                                           \
> > -                                                          \
> > -fallback:                                                 \
> > -    /* filename may be be in CP_ACP */                    \
> > -    return afunc(filename_utf8, par);                     \
> > +static inline int win32_access(const char *filename_utf8, int par)
> > +{
> > +    wchar_t *filename_w;
> > +    int ret;
> > +    if (get_extended_win32_path(filename_utf8, &filename_w))
> > +        return -1;
> > +    if (!filename_w)
> > +        goto fallback;
> > +    ret = _waccess(filename_w, par);
> > +    av_free(filename_w);
> > +    return ret;
> > +fallback:
> > +    return _access(filename_utf8, par);
> > }
> >
> > -DEF_FS_FUNCTION2(access, _waccess, _access, int)
> > -DEF_FS_FUNCTION2(stat, _wstati64, _stati64, struct stat*)
> > +static inline int win32_stat(const char *filename_utf8, struct stat
> *par)
> > +{
> 
> Maybe "struct win32_stat" in the parameter here too, for consistency?

Yup. (it didn't work in an earlier iteration, but now it does)


> > +    wchar_t *filename_w;
> > +    int ret;
> > +    struct _stati64 winstat = { 0 };
> > +
> > +    if (get_extended_win32_path(filename_utf8, &filename_w))
> > +        return -1;
> > +
> > +    if (filename_w) {
> > +        ret = _wstat64(filename_w, &winstat);
> > +        av_free(filename_w);
> > +    } else
> > +        ret = _stat64(filename_utf8, &winstat);
> > +
> > +    par->st_dev   = winstat.st_dev;
> > +    par->st_ino   = winstat.st_ino;
> > +    par->st_mode  = winstat.st_mode;
> > +    par->st_nlink = winstat.st_nlink;
> > +    par->st_uid   = winstat.st_uid;
> > +    par->st_gid   = winstat.st_gid;
> > +    par->st_rdev  = winstat.st_rdev;
> > +    par->st_size  = winstat.st_size;
> > +    par->st_atime = winstat.st_atime;
> > +    par->st_mtime = winstat.st_mtime;
> > +    par->st_ctime = winstat.st_ctime;
> 
> Thanks, this approach seems robust and safe to me!
> 
> With this change in place, shouldn't we drop the #ifdef for
> stat/win32_stat in file.c at the same time?

Done. While doing that, I realized that fstat needs
to be remapped as well, otherwise _ftati64 would be
called with the win32_stat structure. Done that 
as well.

> Other than that, this starts to look ok now.

Thanks for your great help, much appreciated!

softworkz
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v7 0/3] Support long file names on Windows

[ffmpegagent]

This patchset adds support for long file and directory paths on Windows. The
implementation follows the same logic that .NET is using internally, with
the only exception that it doesn't expand short path components in 8.3
format. .NET does this as the same function is also used for other purposes,
but in our case, that's not required. Short (8.3) paths are working as well
with the extended path prefix, even when longer than 260.

Successfully tested:

 * Regular paths wth drive letter
 * Regular UNC paths
 * Long paths wth drive letter
 * Long paths wth drive letter and forward slashes
 * Long UNC paths
 * Prefixed paths wth drive letter
 * Prefixed UNC paths

I have kept the individual functions separate on purpose, to make it easy to
compare with the .NET impl. (compilers should inlinie those anyway)

v2

 * wchar_filename: Improve comments and function documentation
 * os_support: adjust defines to use win32_stat

v3

 * removed length check in path_is_extended()
 * added path_is_device_path() check in add_extended_prefix()
 * add_extended_prefix(): clarified doc and add checks
 * clarified string allocation length calculation
 * replaced 260 with MAX_PATH
 * removed redundant checks after normalization

v4

 * rebased. no changes

v5

 * resolved the ugly struct duplication
 * compatible with _USE_32BIT_TIME_T

v6

 * wchar_filename.h: added links to .NET source code
 * wchar_filename.h: free allocations on error
 * os_support.hs: use clean and safe way to redirect stat() calls

v7

 * os_support.h: remapped fstat with win32_stat structure
 * os_support.h: use int64_t for some members
 * avformat/file: remove _WIN32 condition

softworkz (3):
  avutil/wchar_filename,file_open: Support long file names on Windows
  avformat/os_support: Support long file names on Windows
  avformat/file: remove _WIN32 condition

 libavformat/file.c         |   4 -
 libavformat/os_support.h   | 106 ++++++++++++++++------
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 180 +++++++++++++++++++++++++++++++++++++
 4 files changed, 262 insertions(+), 30 deletions(-)


base-commit: 6076dbcb55d0c9b6693d1acad12a63f7268301aa
Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-28%2Fsoftworkz%2Fsubmit_long_filenames-v7
Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v7
Pull-Request: https://github.com/ffstaging/FFmpeg/pull/28

Range-diff vs v6:

 1:  960aa795ff = 1:  960aa795ff avutil/wchar_filename,file_open: Support long file names on Windows
 2:  6f8d400db7 ! 2:  7751335906 avformat/os_support: Support long file names on Windows
     @@ libavformat/os_support.h
      +        short          st_uid;     /* user ID of owner */
      +        short          st_gid;     /* group ID of owner */
      +        _dev_t         st_rdev;    /* device ID (if special file) */
     -+        long           st_size;    /* total size, in bytes */
     -+        time_t         st_atime;   /* time of last access */
     -+        time_t         st_mtime;   /* time of last modification */
     -+        time_t         st_ctime;   /* time of last status change */
     ++        int64_t        st_size;    /* total size, in bytes */
     ++        int64_t        st_atime;   /* time of last access */
     ++        int64_t        st_mtime;   /* time of last modification */
     ++        int64_t        st_ctime;   /* time of last status change */
      +    };
      +
       #  ifdef fstat
       #   undef fstat
       #  endif
     +-#  define fstat(f,s) _fstati64((f), (s))
     ++#  define fstat win32_fstat
     + #endif /* defined(_WIN32) */
     + 
     + 
      @@ libavformat/os_support.h: static inline int win32_##name(const char *filename_utf8) \
           wchar_t *filename_w;                                  \
           int ret;                                              \
     @@ libavformat/os_support.h: DEF_FS_FUNCTION(unlink, _wunlink, _unlink)
      +    return ret;
      +fallback:
      +    return _access(filename_utf8, par);
     ++}
     ++
     ++static inline void copy_stat(struct _stati64 *winstat, struct win32_stat *par)
     ++{
     ++    par->st_dev   = winstat->st_dev;
     ++    par->st_ino   = winstat->st_ino;
     ++    par->st_mode  = winstat->st_mode;
     ++    par->st_nlink = winstat->st_nlink;
     ++    par->st_uid   = winstat->st_uid;
     ++    par->st_gid   = winstat->st_gid;
     ++    par->st_rdev  = winstat->st_rdev;
     ++    par->st_size  = winstat->st_size;
     ++    par->st_atime = winstat->st_atime;
     ++    par->st_mtime = winstat->st_mtime;
     ++    par->st_ctime = winstat->st_ctime;
       }
       
      -DEF_FS_FUNCTION2(access, _waccess, _access, int)
      -DEF_FS_FUNCTION2(stat, _wstati64, _stati64, struct stat*)
     -+static inline int win32_stat(const char *filename_utf8, struct stat *par)
     ++static inline int win32_stat(const char *filename_utf8, struct win32_stat *par)
      +{
     ++    struct _stati64 winstat = { 0 };
      +    wchar_t *filename_w;
      +    int ret;
     -+    struct _stati64 winstat = { 0 };
      +
      +    if (get_extended_win32_path(filename_utf8, &filename_w))
      +        return -1;
     @@ libavformat/os_support.h: DEF_FS_FUNCTION(unlink, _wunlink, _unlink)
      +    } else
      +        ret = _stat64(filename_utf8, &winstat);
      +
     -+    par->st_dev   = winstat.st_dev;
     -+    par->st_ino   = winstat.st_ino;
     -+    par->st_mode  = winstat.st_mode;
     -+    par->st_nlink = winstat.st_nlink;
     -+    par->st_uid   = winstat.st_uid;
     -+    par->st_gid   = winstat.st_gid;
     -+    par->st_rdev  = winstat.st_rdev;
     -+    par->st_size  = winstat.st_size;
     -+    par->st_atime = winstat.st_atime;
     -+    par->st_mtime = winstat.st_mtime;
     -+    par->st_ctime = winstat.st_ctime;
     ++    copy_stat(&winstat, par);
     ++
     ++    return ret;
     ++}
     ++
     ++static inline int win32_fstat(int fd, struct win32_stat *par)
     ++{
     ++    struct _stati64 winstat = { 0 };
     ++    int ret;
     ++
     ++    ret = _fstat64(fd, &winstat);
     ++
     ++    copy_stat(&winstat, par);
      +
      +    return ret;
      +}
 -:  ---------- > 3:  0522fc2315 avformat/file: remove _WIN32 condition

-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v7 1/3] avutil/wchar_filename, file_open: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 180 +++++++++++++++++++++++++++++++++++++
 2 files changed, 181 insertions(+), 1 deletion(-)

diff --git a/libavutil/file_open.c b/libavutil/file_open.c
index fb64c2e4ee..58a6073353 100644
--- a/libavutil/file_open.c
+++ b/libavutil/file_open.c
@@ -45,7 +45,7 @@ static int win32_open(const char *filename_utf8, int oflag, int pmode)
     wchar_t *filename_w;
 
     /* convert UTF-8 to wide chars */
-    if (utf8towchar(filename_utf8, &filename_w))
+    if (get_extended_win32_path(filename_utf8, &filename_w))
         return -1;
     if (!filename_w)
         goto fallback;
diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h
index 90f082452c..f36d9dfea3 100644
--- a/libavutil/wchar_filename.h
+++ b/libavutil/wchar_filename.h
@@ -40,6 +40,186 @@ static inline int utf8towchar(const char *filename_utf8, wchar_t **filename_w)
     MultiByteToWideChar(CP_UTF8, 0, filename_utf8, -1, *filename_w, num_chars);
     return 0;
 }
+
+/**
+ * Checks for extended path prefixes for which normalization needs to be skipped.
+ * see .NET6: PathInternal.IsExtended()
+ * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L165
+ */
+static inline int path_is_extended(const wchar_t *path)
+{
+    if (path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\')
+        return 1;
+
+    return 0;
+}
+
+/**
+ * Checks for a device path prefix.
+ * see .NET6: PathInternal.IsDevice()
+ * we don't check forward slashes and extended paths (as already done)
+ * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L132
+ */
+static inline int path_is_device_path(const wchar_t *path)
+{
+    if (path[0] == L'\\' && path[1] == L'\\' && path[2] == L'.' && path[3] == L'\\')
+        return 1;
+
+    return 0;
+}
+
+/**
+ * Performs path normalization by calling GetFullPathNameW().
+ * see .NET6: PathHelper.GetFullPathName()
+ * https://github.com/dotnet/runtime/blob/2a99e18eedabcf1add064c099da59d9301ce45e0/src/libraries/System.Private.CoreLib/src/System/IO/PathHelper.Windows.cs#L70
+ */
+static inline int get_full_path_name(wchar_t **ppath_w)
+{
+    int num_chars;
+    wchar_t *temp_w;
+
+    num_chars = GetFullPathNameW(*ppath_w, 0, NULL, NULL);
+    if (num_chars <= 0) {
+        errno = EINVAL;
+        return -1;
+    }
+
+    temp_w = (wchar_t *)av_calloc(num_chars, sizeof(wchar_t));
+    if (!temp_w) {
+        errno = ENOMEM;
+        return -1;
+    }
+
+    num_chars = GetFullPathNameW(*ppath_w, num_chars, temp_w, NULL);
+    if (num_chars <= 0) {
+        av_free(temp_w);
+        errno = EINVAL;
+        return -1;
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+/**
+ * Normalizes a Windows file or folder path.
+ * Expansion of short paths (with 8.3 path components) is currently omitted
+ * as it is not required for accessing long paths.
+ * see .NET6: PathHelper.Normalize()
+ * https://github.com/dotnet/runtime/blob/2a99e18eedabcf1add064c099da59d9301ce45e0/src/libraries/System.Private.CoreLib/src/System/IO/PathHelper.Windows.cs#L25
+ */
+static inline int path_normalize(wchar_t **ppath_w)
+{
+    int ret;
+
+    if ((ret = get_full_path_name(ppath_w)) < 0)
+        return ret;
+
+    /* What .NET does at this point is to call PathHelper.TryExpandShortFileName()
+     * in case the path contains a '~' character.
+     * We don't need to do this as we don't need to normalize the file name
+     * for presentation, and the extended path prefix works with 8.3 path
+     * components as well
+     */
+    return 0;
+}
+
+/**
+ * Adds an extended path or UNC prefix to longs paths or paths ending
+ * with a space or a dot. (' ' or '.').
+ * This function expects that the path has been normalized before by
+ * calling path_normalize() and it doesn't check whether the path is
+ * actually long (> MAX_PATH).
+ * see .NET6: PathInternal.EnsureExtendedPrefix()
+ * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L107
+ */
+static inline int add_extended_prefix(wchar_t **ppath_w)
+{
+    const wchar_t *unc_prefix           = L"\\\\?\\UNC\\";
+    const wchar_t *extended_path_prefix = L"\\\\?\\";
+    const wchar_t *path_w               = *ppath_w;
+    const size_t len                    = wcslen(path_w);
+    wchar_t *temp_w;
+
+    /* We're skipping the check IsPartiallyQualified() because
+     * we expect to have called GetFullPathNameW() already. */
+    if (len < 2 || path_is_extended(*ppath_w) || path_is_device_path(*ppath_w)) {
+        return 0;
+    }
+
+    if (path_w[0] == L'\\' && path_w[1] == L'\\') {
+        /* unc_prefix length is 8 plus 1 for terminating zeros,
+         * we subtract 2 for the leading '\\' of the original path */
+        temp_w = (wchar_t *)av_calloc(len - 2 + 8 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, unc_prefix);
+        wcscat(temp_w, path_w + 2);
+    } else {
+        // The length of extended_path_prefix is 4 plus 1 for terminating zeros
+        temp_w = (wchar_t *)av_calloc(len + 4 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, extended_path_prefix);
+        wcscat(temp_w, path_w);
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+/**
+ * Converts a file or folder path to wchar_t for use with Windows file
+ * APIs. Paths with extended path prefix (either '\\?\' or \??\') are
+ * left unchanged.
+ * All other paths are normalized and converted to absolute paths.
+ * Longs paths (>= MAX_PATH) are prefixed with the extended path or extended
+ * UNC path prefix.
+ * see .NET6: Path.GetFullPath() and Path.GetFullPathInternal()
+ * https://github.com/dotnet/runtime/blob/2a99e18eedabcf1add064c099da59d9301ce45e0/src/libraries/System.Private.CoreLib/src/System/IO/Path.Windows.cs#L126
+ */
+static inline int get_extended_win32_path(const char *path, wchar_t **ppath_w)
+{
+    int ret;
+    size_t len;
+
+    if ((ret = utf8towchar(path, ppath_w)) < 0)
+        return ret;
+
+    if (path_is_extended(*ppath_w)) {
+        /* Paths prefixed with '\\?\' or \??\' are considered normalized by definition.
+         * Windows doesn't normalize those paths and neither should we.
+         */
+        return 0;
+    }
+
+    if ((ret = path_normalize(ppath_w)) < 0) {
+        av_freep(ppath_w);
+        return ret;
+    }
+
+    /* see .NET6: PathInternal.EnsureExtendedPrefixIfNeeded()
+     * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L92
+     */
+    len = wcslen(*ppath_w);
+    if (len >= MAX_PATH) {
+        if ((ret = add_extended_prefix(ppath_w)) < 0) {
+            av_freep(ppath_w);
+            return ret;
+        }
+    }
+
+    return 0;
+}
+
 #endif
 
 #endif /* AVUTIL_WCHAR_FILENAME_H */
-- 
ffmpeg-codebot

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v7 2/3] avformat/os_support: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavformat/os_support.h | 106 ++++++++++++++++++++++++++++++---------
 1 file changed, 81 insertions(+), 25 deletions(-)

diff --git a/libavformat/os_support.h b/libavformat/os_support.h
index 5e6b32d2dc..1c3b234b06 100644
--- a/libavformat/os_support.h
+++ b/libavformat/os_support.h
@@ -49,11 +49,28 @@
 #  ifdef stat
 #   undef stat
 #  endif
-#  define stat _stati64
+
+#  define stat win32_stat
+
+    struct win32_stat
+    {
+        _dev_t         st_dev;     /* ID of device containing file */
+        _ino_t         st_ino;     /* inode number */
+        unsigned short st_mode;    /* protection */
+        short          st_nlink;   /* number of hard links */
+        short          st_uid;     /* user ID of owner */
+        short          st_gid;     /* group ID of owner */
+        _dev_t         st_rdev;    /* device ID (if special file) */
+        int64_t        st_size;    /* total size, in bytes */
+        int64_t        st_atime;   /* time of last access */
+        int64_t        st_mtime;   /* time of last modification */
+        int64_t        st_ctime;   /* time of last status change */
+    };
+
 #  ifdef fstat
 #   undef fstat
 #  endif
-#  define fstat(f,s) _fstati64((f), (s))
+#  define fstat win32_fstat
 #endif /* defined(_WIN32) */
 
 
@@ -153,7 +170,7 @@ static inline int win32_##name(const char *filename_utf8) \
     wchar_t *filename_w;                                  \
     int ret;                                              \
                                                           \
-    if (utf8towchar(filename_utf8, &filename_w))          \
+    if (get_extended_win32_path(filename_utf8, &filename_w)) \
         return -1;                                        \
     if (!filename_w)                                      \
         goto fallback;                                    \
@@ -171,37 +188,76 @@ DEF_FS_FUNCTION(unlink, _wunlink, _unlink)
 DEF_FS_FUNCTION(mkdir,  _wmkdir,  _mkdir)
 DEF_FS_FUNCTION(rmdir,  _wrmdir , _rmdir)
 
-#define DEF_FS_FUNCTION2(name, wfunc, afunc, partype)     \
-static inline int win32_##name(const char *filename_utf8, partype par) \
-{                                                         \
-    wchar_t *filename_w;                                  \
-    int ret;                                              \
-                                                          \
-    if (utf8towchar(filename_utf8, &filename_w))          \
-        return -1;                                        \
-    if (!filename_w)                                      \
-        goto fallback;                                    \
-                                                          \
-    ret = wfunc(filename_w, par);                         \
-    av_free(filename_w);                                  \
-    return ret;                                           \
-                                                          \
-fallback:                                                 \
-    /* filename may be be in CP_ACP */                    \
-    return afunc(filename_utf8, par);                     \
+static inline int win32_access(const char *filename_utf8, int par)
+{
+    wchar_t *filename_w;
+    int ret;
+    if (get_extended_win32_path(filename_utf8, &filename_w))
+        return -1;
+    if (!filename_w)
+        goto fallback;
+    ret = _waccess(filename_w, par);
+    av_free(filename_w);
+    return ret;
+fallback:
+    return _access(filename_utf8, par);
+}
+
+static inline void copy_stat(struct _stati64 *winstat, struct win32_stat *par)
+{
+    par->st_dev   = winstat->st_dev;
+    par->st_ino   = winstat->st_ino;
+    par->st_mode  = winstat->st_mode;
+    par->st_nlink = winstat->st_nlink;
+    par->st_uid   = winstat->st_uid;
+    par->st_gid   = winstat->st_gid;
+    par->st_rdev  = winstat->st_rdev;
+    par->st_size  = winstat->st_size;
+    par->st_atime = winstat->st_atime;
+    par->st_mtime = winstat->st_mtime;
+    par->st_ctime = winstat->st_ctime;
 }
 
-DEF_FS_FUNCTION2(access, _waccess, _access, int)
-DEF_FS_FUNCTION2(stat, _wstati64, _stati64, struct stat*)
+static inline int win32_stat(const char *filename_utf8, struct win32_stat *par)
+{
+    struct _stati64 winstat = { 0 };
+    wchar_t *filename_w;
+    int ret;
+
+    if (get_extended_win32_path(filename_utf8, &filename_w))
+        return -1;
+
+    if (filename_w) {
+        ret = _wstat64(filename_w, &winstat);
+        av_free(filename_w);
+    } else
+        ret = _stat64(filename_utf8, &winstat);
+
+    copy_stat(&winstat, par);
+
+    return ret;
+}
+
+static inline int win32_fstat(int fd, struct win32_stat *par)
+{
+    struct _stati64 winstat = { 0 };
+    int ret;
+
+    ret = _fstat64(fd, &winstat);
+
+    copy_stat(&winstat, par);
+
+    return ret;
+}
 
 static inline int win32_rename(const char *src_utf8, const char *dest_utf8)
 {
     wchar_t *src_w, *dest_w;
     int ret;
 
-    if (utf8towchar(src_utf8, &src_w))
+    if (get_extended_win32_path(src_utf8, &src_w))
         return -1;
-    if (utf8towchar(dest_utf8, &dest_w)) {
+    if (get_extended_win32_path(dest_utf8, &dest_w)) {
         av_free(src_w);
         return -1;
     }
-- 
ffmpeg-codebot

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v7 3/3] avformat/file: remove _WIN32 condition

[softworkz]

From: softworkz <softworkz@hotmail.com>

stat is now re-mapped with long path support
in os_support.h

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavformat/file.c | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/libavformat/file.c b/libavformat/file.c
index 063d7c5aa2..98c9e81bcb 100644
--- a/libavformat/file.c
+++ b/libavformat/file.c
@@ -154,11 +154,7 @@ static int file_check(URLContext *h, int mask)
             ret |= AVIO_FLAG_WRITE;
 #else
     struct stat st;
-#   ifndef _WIN32
     ret = stat(filename, &st);
-#   else
-    ret = win32_stat(filename, &st);
-#   endif
     if (ret < 0)
         return AVERROR(errno);
 
-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v6 2/2] avformat/os_support: Support long file names on Windows

[Martin Storsjö]

On Tue, 24 May 2022, Soft Works wrote:

>> -----Original Message-----
>> From: Martin Storsjö <martin@martin.st>
>> Sent: Tuesday, May 24, 2022 10:59 PM
>> To: softworkz <ffmpegagent@gmail.com>
>> Cc: ffmpeg-devel@ffmpeg.org; Soft Works <softworkz@hotmail.com>; Hendrik
>> Leppkes <h.leppkes@gmail.com>
>> Subject: Re: [PATCH v6 2/2] avformat/os_support: Support long file names
>> on Windows
>>
>>> +    wchar_t *filename_w;
>>> +    int ret;
>>> +    struct _stati64 winstat = { 0 };
>>> +
>>> +    if (get_extended_win32_path(filename_utf8, &filename_w))
>>> +        return -1;
>>> +
>>> +    if (filename_w) {
>>> +        ret = _wstat64(filename_w, &winstat);
>>> +        av_free(filename_w);
>>> +    } else
>>> +        ret = _stat64(filename_utf8, &winstat);
>>> +
>>> +    par->st_dev   = winstat.st_dev;
>>> +    par->st_ino   = winstat.st_ino;
>>> +    par->st_mode  = winstat.st_mode;
>>> +    par->st_nlink = winstat.st_nlink;
>>> +    par->st_uid   = winstat.st_uid;
>>> +    par->st_gid   = winstat.st_gid;
>>> +    par->st_rdev  = winstat.st_rdev;
>>> +    par->st_size  = winstat.st_size;
>>> +    par->st_atime = winstat.st_atime;
>>> +    par->st_mtime = winstat.st_mtime;
>>> +    par->st_ctime = winstat.st_ctime;
>>
>> Thanks, this approach seems robust and safe to me!
>>
>> With this change in place, shouldn't we drop the #ifdef for
>> stat/win32_stat in file.c at the same time?
>
> Done. While doing that, I realized that fstat needs
> to be remapped as well, otherwise _ftati64 would be
> called with the win32_stat structure. Done that
> as well.

Good - I also just realized the same while grepping around for "struct 
stat".

// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v7 0/3] Support long file names on Windows

[Martin Storsjö]

On Tue, 24 May 2022, ffmpegagent wrote:

> This patchset adds support for long file and directory paths on Windows. The
> implementation follows the same logic that .NET is using internally, with
> the only exception that it doesn't expand short path components in 8.3
> format. .NET does this as the same function is also used for other purposes,
> but in our case, that's not required. Short (8.3) paths are working as well
> with the extended path prefix, even when longer than 260.
>
> Successfully tested:
>
> * Regular paths wth drive letter
> * Regular UNC paths
> * Long paths wth drive letter
> * Long paths wth drive letter and forward slashes
> * Long UNC paths
> * Prefixed paths wth drive letter
> * Prefixed UNC paths
>
> I have kept the individual functions separate on purpose, to make it easy to
> compare with the .NET impl. (compilers should inlinie those anyway)
>
> v2
>
> * wchar_filename: Improve comments and function documentation
> * os_support: adjust defines to use win32_stat
>
> v3
>
> * removed length check in path_is_extended()
> * added path_is_device_path() check in add_extended_prefix()
> * add_extended_prefix(): clarified doc and add checks
> * clarified string allocation length calculation
> * replaced 260 with MAX_PATH
> * removed redundant checks after normalization
>
> v4
>
> * rebased. no changes
>
> v5
>
> * resolved the ugly struct duplication
> * compatible with _USE_32BIT_TIME_T
>
> v6
>
> * wchar_filename.h: added links to .NET source code
> * wchar_filename.h: free allocations on error
> * os_support.hs: use clean and safe way to redirect stat() calls
>
> v7
>
> * os_support.h: remapped fstat with win32_stat structure
> * os_support.h: use int64_t for some members
> * avformat/file: remove _WIN32 condition
>
> softworkz (3):
>  avutil/wchar_filename,file_open: Support long file names on Windows
>  avformat/os_support: Support long file names on Windows
>  avformat/file: remove _WIN32 condition

Thanks, I think this iteration of the patchset is fine - I don't have 
anything else to add to it at the moment. Let's wait for others to chime 
in still before proceeding further with it, though.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v7 2/3] avformat/os_support: Support long file names on Windows

[nil-admirari]

> + struct win32_stat
> + {
> + _dev_t st_dev; /* ID of device containing file */
> + _ino_t st_ino; /* inode number */
> + unsigned short st_mode; /* protection */
> + short st_nlink; /* number of hard links */
> + short st_uid; /* user ID of owner */
> + short st_gid; /* group ID of owner */
> + _dev_t st_rdev; /* device ID (if special file) */
> + int64_t st_size; /* total size, in bytes */
> + int64_t st_atime; /* time of last access */
> + int64_t st_mtime; /* time of last modification */
> + int64_t st_ctime; /* time of last status change */
> + };

Wouldn't it make sense to add a
static_assert(sizeof(struct win32_stat) == sizeof(struct _stati64))
somewhere?

> +static inline int win32_access(const char *filename_utf8, int par)
> +static inline void copy_stat(struct _stati64 *winstat, struct win32_stat *par)
> +static inline int win32_stat(const char *filename_utf8, struct win32_stat *par)
> +static inline int win32_fstat(int fd, struct win32_stat *par)

How about renaming par to something more appropriate?

> +static inline void copy_stat(struct _stati64 *winstat, struct win32_stat *par)
> +{
> + par->st_dev = winstat->st_dev;
> + par->st_ino = winstat->st_ino;
> + par->st_mode = winstat->st_mode;
> + par->st_nlink = winstat->st_nlink;
> + par->st_uid = winstat->st_uid;
> + par->st_gid = winstat->st_gid;
> + par->st_rdev = winstat->st_rdev;
> + par->st_size = winstat->st_size;
> + par->st_atime = winstat->st_atime;
> + par->st_mtime = winstat->st_mtime;
> + par->st_ctime = winstat->st_ctime;
> }

Would memcpy make more sense here?

> +static inline int win32_stat(const char *filename_utf8, struct win32_stat *par)
> +{
> + struct _stati64 winstat = { 0 };
> ...
> 
> +static inline int win32_fstat(int fd, struct win32_stat *par)
> +{
> + struct _stati64 winstat = { 0 };
> ...

Functions use _stati64 internally, which is affected by _USE_32BIT_TIME_T.
win32_stat does not take _USE_32BIT_TIME_T into account at all.
It was already pointed out that _USE_32BIT_TIME_T is actually used:
https://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/296731.html.



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v7 2/3] avformat/os_support: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Wednesday, May 25, 2022 4:48 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v7 2/3] avformat/os_support: Support
> long file names on Windows
> 
> > + struct win32_stat
> > + {
> > + _dev_t st_dev; /* ID of device containing file */
> > + _ino_t st_ino; /* inode number */
> > + unsigned short st_mode; /* protection */
> > + short st_nlink; /* number of hard links */
> > + short st_uid; /* user ID of owner */
> > + short st_gid; /* group ID of owner */
> > + _dev_t st_rdev; /* device ID (if special file) */
> > + int64_t st_size; /* total size, in bytes */
> > + int64_t st_atime; /* time of last access */
> > + int64_t st_mtime; /* time of last modification */
> > + int64_t st_ctime; /* time of last status change */
> > + };
> 
> Wouldn't it make sense to add a
> static_assert(sizeof(struct win32_stat) == sizeof(struct _stati64))
> somewhere?

No, it is intended and expected that the structs are different.

> 
> > +static inline int win32_access(const char *filename_utf8, int par)
> > +static inline void copy_stat(struct _stati64 *winstat, struct
> win32_stat *par)
> > +static inline int win32_stat(const char *filename_utf8, struct
> win32_stat *par)
> > +static inline int win32_fstat(int fd, struct win32_stat *par)
> 
> How about renaming par to something more appropriate?

How? And why?

These functions were always named like that (it just wasn't visible
as these were constructed through macros).

> 
> > +static inline void copy_stat(struct _stati64 *winstat, struct
> win32_stat *par)
> > +{
> > + par->st_dev = winstat->st_dev;
> > + par->st_ino = winstat->st_ino;
> > + par->st_mode = winstat->st_mode;
> > + par->st_nlink = winstat->st_nlink;
> > + par->st_uid = winstat->st_uid;
> > + par->st_gid = winstat->st_gid;
> > + par->st_rdev = winstat->st_rdev;
> > + par->st_size = winstat->st_size;
> > + par->st_atime = winstat->st_atime;
> > + par->st_mtime = winstat->st_mtime;
> > + par->st_ctime = winstat->st_ctime;
> > }
> 
> Would memcpy make more sense here?

No, it is intended and expected that the structs are different.

> 
> > +static inline int win32_stat(const char *filename_utf8, struct
> win32_stat *par)
> > +{
> > + struct _stati64 winstat = { 0 };
> > ...
> >
> > +static inline int win32_fstat(int fd, struct win32_stat *par)
> > +{
> > + struct _stati64 winstat = { 0 };
> > ...
> 
> Functions use _stati64 internally, which is affected by _USE_32BIT_TIME_T.
> win32_stat does not take _USE_32BIT_TIME_T into account at all.
> It was already pointed out that _USE_32BIT_TIME_T is actually used:
> https://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/296731.html.

That's why the structs are different and the fields of
win32_stat always large enough, no matter which struct 
is being used internally.

The point of this solution is to provide independence 
between what the application "sees" and what is used 
to call the Windows API.

Kind regards,
softworkz




_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v7 2/3] avformat/os_support: Support long file names on Windows

[Martin Storsjö]

On Wed, 25 May 2022, nil-admirari@mailo.com wrote:

>> + struct win32_stat
>> + {
>> + _dev_t st_dev; /* ID of device containing file */
>> + _ino_t st_ino; /* inode number */
>> + unsigned short st_mode; /* protection */
>> + short st_nlink; /* number of hard links */
>> + short st_uid; /* user ID of owner */
>> + short st_gid; /* group ID of owner */
>> + _dev_t st_rdev; /* device ID (if special file) */
>> + int64_t st_size; /* total size, in bytes */
>> + int64_t st_atime; /* time of last access */
>> + int64_t st_mtime; /* time of last modification */
>> + int64_t st_ctime; /* time of last status change */
>> + };
>
> Wouldn't it make sense to add a
> static_assert(sizeof(struct win32_stat) == sizeof(struct _stati64))
> somewhere?
>
>> +static inline void copy_stat(struct _stati64 *winstat, struct win32_stat *par)
>> +{
>> + par->st_dev = winstat->st_dev;
>> + par->st_ino = winstat->st_ino;
>> + par->st_mode = winstat->st_mode;
>> + par->st_nlink = winstat->st_nlink;
>> + par->st_uid = winstat->st_uid;
>> + par->st_gid = winstat->st_gid;
>> + par->st_rdev = winstat->st_rdev;
>> + par->st_size = winstat->st_size;
>> + par->st_atime = winstat->st_atime;
>> + par->st_mtime = winstat->st_mtime;
>> + par->st_ctime = winstat->st_ctime;
>> }
>
> Would memcpy make more sense here?

As explained elsewhere too, the explicit intent is that this is a 
different struct than the real _stati64 or whichever happens to be used, 
not necessarily identical.

We don't know the exact layout of the real stat struct (and technically, 
different C runtimes, e.g. msvcrt.dll, msvcr100.dll, msvcr120.dll, UCRT, 
could all have different layouts/sizes), and it's brittle to try to 
guess/mimic it. So instead of trying to mimic it, we just make our own 
(which is what ends up used in the calling libavformat code) - our wrapper 
then explicitly uses one from the C runtime (which we don't know the 
size/layout of), and we just copy it field by field into the one we expose 
to the caller.

This could use any random layout, as long as it contains the subset of 
fields from stat that we actually use anywhere.

And if we wanted a static assert, the only relevant assert would be to 
make sure that our wrapper struct's fields are as large as, or larger, 
than the ones that the original stat returns.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v7 2/3] avformat/os_support: Support long file names on Windows

[nil-admirari]

> No, it is intended and expected that the structs are different.
> ...
> That's why the structs are different and the fields of
> win32_stat always large enough, no matter which struct 
> is being used internally.

Please document that there is a potential difference in time types
and that the difference is intentional, and that the chosen
time type is always large enough.

Probably it's worthwhile to document that the entire machinery was created
because of POSIX stat function and struct being identically named,
which is not possible to accommodate by a simple macro.

> > > +static inline int win32_access(const char *filename_utf8, int par)
> > > +static inline void copy_stat(struct _stati64 *winstat, struct
> > win32_stat *par)
> > > +static inline int win32_stat(const char *filename_utf8, struct
> > win32_stat *par)
> > > +static inline int win32_fstat(int fd, struct win32_stat *par)
> > 
> > How about renaming par to something more appropriate?
> 
> How? And why?
>
> These functions were always named like that (it just wasn't visible
> as these were constructed through macros).

_access argument is called mode:
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/access-waccess?view=msvc-170.

_stat and _fstat argument is called a buffer:
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/stat-functions?view=msvc-170
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/fstat-fstat32-fstat64-fstati64-fstat32i64-fstat64i32?view=msvc-170
It's no better than par, but you don't have to follow MS.

Somehow winstat refers to parameters of type _stati64, not win32_stat.
I would've called win32_stat params winstat, and _stati64 params crtstat,
but you can always come up with better names.

Inside a macro generic parameter names are a necessity. Functions better be more specific.



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v7 2/3] avformat/os_support: Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Wednesday, May 25, 2022 9:17 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v7 2/3] avformat/os_support: Support
> long file names on Windows
> 
> > No, it is intended and expected that the structs are different.
> > ...
> > That's why the structs are different and the fields of
> > win32_stat always large enough, no matter which struct
> > is being used internally.
> 
> Please document that there is a potential difference in time types
> and that the difference is intentional, and that the chosen
> time type is always large enough.
> 
> Probably it's worthwhile to document that the entire machinery was created
> because of POSIX stat function and struct being identically named,
> which is not possible to accommodate by a simple macro.

Yes, it makes sense to explain that a bit. Will do.


> > > > +static inline int win32_access(const char *filename_utf8, int par)
> > > > +static inline void copy_stat(struct _stati64 *winstat, struct
> > > win32_stat *par)
> > > > +static inline int win32_stat(const char *filename_utf8, struct
> > > win32_stat *par)
> > > > +static inline int win32_fstat(int fd, struct win32_stat *par)
> > >
> > > How about renaming par to something more appropriate?
> >
> > How? And why?
> >
> > These functions were always named like that (it just wasn't visible
> > as these were constructed through macros).
> 
> _access argument is called mode:
> https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/access-
> waccess?view=msvc-170.
> _stat and _fstat argument is called a buffer:
> https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/stat-
> functions?view=msvc-170
> https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/fstat-
> fstat32-fstat64-fstati64-fstat32i64-fstat64i32?view=msvc-170
> It's no better than par, but you don't have to follow MS.

I misread, thinking you wanted to rename the functions. Yes, the par
comes from the macro (which was for building the two functions with 
different parameter name). I have expanded the macro for stat, 
and then the macro would have only existed for access, so I did 
expand that one as well.

So, no problem, I will rename the arguments according to posix.

> Somehow winstat refers to parameters of type _stati64, not win32_stat.

I named it winstat for "win api stat". When you want to go strict
you could say that all the naming is incorrect, because 'win32' 
suggests that all those are about calls to the Win32 API, while
in fact, they are calls to msvcrt.

> I would've called win32_stat params winstat, and _stati64 params crtstat,
> but you can always come up with better names.

win32_stat has nothing to do with Windows, it is actually more a posix_stat,
but it needs to have the same name as the function

Even crtstat would not be quite correct and would rather need to 
be named msvcrtstat :-)

We could drive this further and further and probably it would never
be totally "right". Though, I will make the replacements you asked for,
and then we'll see...


Regarding your concerns about _USE_32BIT_TIME_T, I wanted to mention
that we still have an alternative, to get around this.

Currently, we are using _wstati64, _ffstati64 and _stati64. All of 
those are re-defined depending on whether _USE_32BIT_TIME_T is
defined or not.

Instead of that, we could use _wstat64, _ffstat64 and _stat64. In 
this case, we would always get 64bit time values, independent
of the definition of _USE_32BIT_TIME_T.

The only difference it makes would be whether we can have file times
beyond the year 2038.

I'm fine with the way it is right now, I just wanted to have mentioned
it.

Thanks for reviewing,
softworkz







_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows

[ffmpegagent]

This patchset adds support for long file and directory paths on Windows. The
implementation follows the same logic that .NET is using internally, with
the only exception that it doesn't expand short path components in 8.3
format. .NET does this as the same function is also used for other purposes,
but in our case, that's not required. Short (8.3) paths are working as well
with the extended path prefix, even when longer than 260.

Successfully tested:

 * Regular paths wth drive letter
 * Regular UNC paths
 * Long paths wth drive letter
 * Long paths wth drive letter and forward slashes
 * Long UNC paths
 * Prefixed paths wth drive letter
 * Prefixed UNC paths

I have kept the individual functions separate on purpose, to make it easy to
compare with the .NET impl. (compilers should inlinie those anyway)

v2

 * wchar_filename: Improve comments and function documentation
 * os_support: adjust defines to use win32_stat

v3

 * removed length check in path_is_extended()
 * added path_is_device_path() check in add_extended_prefix()
 * add_extended_prefix(): clarified doc and add checks
 * clarified string allocation length calculation
 * replaced 260 with MAX_PATH
 * removed redundant checks after normalization

v4

 * rebased. no changes

v5

 * resolved the ugly struct duplication
 * compatible with _USE_32BIT_TIME_T

v6

 * wchar_filename.h: added links to .NET source code
 * wchar_filename.h: free allocations on error
 * os_support.hs: use clean and safe way to redirect stat() calls

v7

 * os_support.h: remapped fstat with win32_stat structure
 * os_support.h: use int64_t for some members
 * avformat/file: remove _WIN32 condition

v8

 * os_support.h: documented win32_stat structure
 * os_support.h: renamed function parameters

softworkz (3):
  avutil/wchar_filename,file_open: Support long file names on Windows
  avformat/os_support: Support long file names on Windows
  avformat/file: remove _WIN32 condition

 libavformat/file.c         |   4 -
 libavformat/os_support.h   | 116 ++++++++++++++++++------
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 180 +++++++++++++++++++++++++++++++++++++
 4 files changed, 272 insertions(+), 30 deletions(-)


base-commit: 6076dbcb55d0c9b6693d1acad12a63f7268301aa
Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-28%2Fsoftworkz%2Fsubmit_long_filenames-v8
Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-28/softworkz/submit_long_filenames-v8
Pull-Request: https://github.com/ffstaging/FFmpeg/pull/28

Range-diff vs v7:

 1:  960aa795ff = 1:  960aa795ff avutil/wchar_filename,file_open: Support long file names on Windows
 2:  7751335906 ! 2:  d0cc40a0d4 avformat/os_support: Support long file names on Windows
     @@ libavformat/os_support.h
      +
      +#  define stat win32_stat
      +
     ++    /*
     ++     * The POSIX definition for the stat() function uses a struct of the
     ++     * same name (struct stat), that why it takes this extra effort  for
     ++     * redirecting/replacing the stat() function with our own one which
     ++     * is capable to handle long path names on Windows.
     ++     * The struct below roughly follows the POSIX definition. Time values
     ++     * are 64bit, but in cases when _USE_32BIT_TIME_T is defined, they
     ++     * will be set to values no larger than INT32_MAX which corresponds
     ++     * to file times up to the year 2038.
     ++     */
      +    struct win32_stat
      +    {
      +        _dev_t         st_dev;     /* ID of device containing file */
     @@ libavformat/os_support.h: DEF_FS_FUNCTION(unlink, _wunlink, _unlink)
      -fallback:                                                 \
      -    /* filename may be be in CP_ACP */                    \
      -    return afunc(filename_utf8, par);                     \
     -+static inline int win32_access(const char *filename_utf8, int par)
     ++static inline int win32_access(const char *filename_utf8, int mode)
      +{
      +    wchar_t *filename_w;
      +    int ret;
     @@ libavformat/os_support.h: DEF_FS_FUNCTION(unlink, _wunlink, _unlink)
      +        return -1;
      +    if (!filename_w)
      +        goto fallback;
     -+    ret = _waccess(filename_w, par);
     ++    ret = _waccess(filename_w, mode);
      +    av_free(filename_w);
      +    return ret;
      +fallback:
     -+    return _access(filename_utf8, par);
     ++    return _access(filename_utf8, mode);
      +}
      +
     -+static inline void copy_stat(struct _stati64 *winstat, struct win32_stat *par)
     ++static inline void copy_stat(struct _stati64 *crtstat, struct win32_stat *buf)
      +{
     -+    par->st_dev   = winstat->st_dev;
     -+    par->st_ino   = winstat->st_ino;
     -+    par->st_mode  = winstat->st_mode;
     -+    par->st_nlink = winstat->st_nlink;
     -+    par->st_uid   = winstat->st_uid;
     -+    par->st_gid   = winstat->st_gid;
     -+    par->st_rdev  = winstat->st_rdev;
     -+    par->st_size  = winstat->st_size;
     -+    par->st_atime = winstat->st_atime;
     -+    par->st_mtime = winstat->st_mtime;
     -+    par->st_ctime = winstat->st_ctime;
     ++    buf->st_dev   = crtstat->st_dev;
     ++    buf->st_ino   = crtstat->st_ino;
     ++    buf->st_mode  = crtstat->st_mode;
     ++    buf->st_nlink = crtstat->st_nlink;
     ++    buf->st_uid   = crtstat->st_uid;
     ++    buf->st_gid   = crtstat->st_gid;
     ++    buf->st_rdev  = crtstat->st_rdev;
     ++    buf->st_size  = crtstat->st_size;
     ++    buf->st_atime = crtstat->st_atime;
     ++    buf->st_mtime = crtstat->st_mtime;
     ++    buf->st_ctime = crtstat->st_ctime;
       }
       
      -DEF_FS_FUNCTION2(access, _waccess, _access, int)
      -DEF_FS_FUNCTION2(stat, _wstati64, _stati64, struct stat*)
     -+static inline int win32_stat(const char *filename_utf8, struct win32_stat *par)
     ++static inline int win32_stat(const char *filename_utf8, struct win32_stat *buf)
      +{
     -+    struct _stati64 winstat = { 0 };
     ++    struct _stati64 crtstat = { 0 };
      +    wchar_t *filename_w;
      +    int ret;
      +
     @@ libavformat/os_support.h: DEF_FS_FUNCTION(unlink, _wunlink, _unlink)
      +        return -1;
      +
      +    if (filename_w) {
     -+        ret = _wstat64(filename_w, &winstat);
     ++        ret = _wstat64(filename_w, &crtstat);
      +        av_free(filename_w);
      +    } else
     -+        ret = _stat64(filename_utf8, &winstat);
     ++        ret = _stat64(filename_utf8, &crtstat);
      +
     -+    copy_stat(&winstat, par);
     ++    copy_stat(&crtstat, buf);
      +
      +    return ret;
      +}
      +
     -+static inline int win32_fstat(int fd, struct win32_stat *par)
     ++static inline int win32_fstat(int fd, struct win32_stat *buf)
      +{
     -+    struct _stati64 winstat = { 0 };
     ++    struct _stati64 crtstat = { 0 };
      +    int ret;
      +
     -+    ret = _fstat64(fd, &winstat);
     ++    ret = _fstat64(fd, &crtstat);
      +
     -+    copy_stat(&winstat, par);
     ++    copy_stat(&crtstat, buf);
      +
      +    return ret;
      +}
 3:  0522fc2315 = 3:  e13c6b0aaa avformat/file: remove _WIN32 condition

-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v8 1/3] avutil/wchar_filename, file_open: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavutil/file_open.c      |   2 +-
 libavutil/wchar_filename.h | 180 +++++++++++++++++++++++++++++++++++++
 2 files changed, 181 insertions(+), 1 deletion(-)

diff --git a/libavutil/file_open.c b/libavutil/file_open.c
index fb64c2e4ee..58a6073353 100644
--- a/libavutil/file_open.c
+++ b/libavutil/file_open.c
@@ -45,7 +45,7 @@ static int win32_open(const char *filename_utf8, int oflag, int pmode)
     wchar_t *filename_w;
 
     /* convert UTF-8 to wide chars */
-    if (utf8towchar(filename_utf8, &filename_w))
+    if (get_extended_win32_path(filename_utf8, &filename_w))
         return -1;
     if (!filename_w)
         goto fallback;
diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h
index 90f082452c..f36d9dfea3 100644
--- a/libavutil/wchar_filename.h
+++ b/libavutil/wchar_filename.h
@@ -40,6 +40,186 @@ static inline int utf8towchar(const char *filename_utf8, wchar_t **filename_w)
     MultiByteToWideChar(CP_UTF8, 0, filename_utf8, -1, *filename_w, num_chars);
     return 0;
 }
+
+/**
+ * Checks for extended path prefixes for which normalization needs to be skipped.
+ * see .NET6: PathInternal.IsExtended()
+ * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L165
+ */
+static inline int path_is_extended(const wchar_t *path)
+{
+    if (path[0] == L'\\' && (path[1] == L'\\' || path[1] == L'?') && path[2] == L'?' && path[3] == L'\\')
+        return 1;
+
+    return 0;
+}
+
+/**
+ * Checks for a device path prefix.
+ * see .NET6: PathInternal.IsDevice()
+ * we don't check forward slashes and extended paths (as already done)
+ * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L132
+ */
+static inline int path_is_device_path(const wchar_t *path)
+{
+    if (path[0] == L'\\' && path[1] == L'\\' && path[2] == L'.' && path[3] == L'\\')
+        return 1;
+
+    return 0;
+}
+
+/**
+ * Performs path normalization by calling GetFullPathNameW().
+ * see .NET6: PathHelper.GetFullPathName()
+ * https://github.com/dotnet/runtime/blob/2a99e18eedabcf1add064c099da59d9301ce45e0/src/libraries/System.Private.CoreLib/src/System/IO/PathHelper.Windows.cs#L70
+ */
+static inline int get_full_path_name(wchar_t **ppath_w)
+{
+    int num_chars;
+    wchar_t *temp_w;
+
+    num_chars = GetFullPathNameW(*ppath_w, 0, NULL, NULL);
+    if (num_chars <= 0) {
+        errno = EINVAL;
+        return -1;
+    }
+
+    temp_w = (wchar_t *)av_calloc(num_chars, sizeof(wchar_t));
+    if (!temp_w) {
+        errno = ENOMEM;
+        return -1;
+    }
+
+    num_chars = GetFullPathNameW(*ppath_w, num_chars, temp_w, NULL);
+    if (num_chars <= 0) {
+        av_free(temp_w);
+        errno = EINVAL;
+        return -1;
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+/**
+ * Normalizes a Windows file or folder path.
+ * Expansion of short paths (with 8.3 path components) is currently omitted
+ * as it is not required for accessing long paths.
+ * see .NET6: PathHelper.Normalize()
+ * https://github.com/dotnet/runtime/blob/2a99e18eedabcf1add064c099da59d9301ce45e0/src/libraries/System.Private.CoreLib/src/System/IO/PathHelper.Windows.cs#L25
+ */
+static inline int path_normalize(wchar_t **ppath_w)
+{
+    int ret;
+
+    if ((ret = get_full_path_name(ppath_w)) < 0)
+        return ret;
+
+    /* What .NET does at this point is to call PathHelper.TryExpandShortFileName()
+     * in case the path contains a '~' character.
+     * We don't need to do this as we don't need to normalize the file name
+     * for presentation, and the extended path prefix works with 8.3 path
+     * components as well
+     */
+    return 0;
+}
+
+/**
+ * Adds an extended path or UNC prefix to longs paths or paths ending
+ * with a space or a dot. (' ' or '.').
+ * This function expects that the path has been normalized before by
+ * calling path_normalize() and it doesn't check whether the path is
+ * actually long (> MAX_PATH).
+ * see .NET6: PathInternal.EnsureExtendedPrefix()
+ * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L107
+ */
+static inline int add_extended_prefix(wchar_t **ppath_w)
+{
+    const wchar_t *unc_prefix           = L"\\\\?\\UNC\\";
+    const wchar_t *extended_path_prefix = L"\\\\?\\";
+    const wchar_t *path_w               = *ppath_w;
+    const size_t len                    = wcslen(path_w);
+    wchar_t *temp_w;
+
+    /* We're skipping the check IsPartiallyQualified() because
+     * we expect to have called GetFullPathNameW() already. */
+    if (len < 2 || path_is_extended(*ppath_w) || path_is_device_path(*ppath_w)) {
+        return 0;
+    }
+
+    if (path_w[0] == L'\\' && path_w[1] == L'\\') {
+        /* unc_prefix length is 8 plus 1 for terminating zeros,
+         * we subtract 2 for the leading '\\' of the original path */
+        temp_w = (wchar_t *)av_calloc(len - 2 + 8 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, unc_prefix);
+        wcscat(temp_w, path_w + 2);
+    } else {
+        // The length of extended_path_prefix is 4 plus 1 for terminating zeros
+        temp_w = (wchar_t *)av_calloc(len + 4 + 1, sizeof(wchar_t));
+        if (!temp_w) {
+            errno = ENOMEM;
+            return -1;
+        }
+        wcscpy(temp_w, extended_path_prefix);
+        wcscat(temp_w, path_w);
+    }
+
+    av_freep(ppath_w);
+    *ppath_w = temp_w;
+
+    return 0;
+}
+
+/**
+ * Converts a file or folder path to wchar_t for use with Windows file
+ * APIs. Paths with extended path prefix (either '\\?\' or \??\') are
+ * left unchanged.
+ * All other paths are normalized and converted to absolute paths.
+ * Longs paths (>= MAX_PATH) are prefixed with the extended path or extended
+ * UNC path prefix.
+ * see .NET6: Path.GetFullPath() and Path.GetFullPathInternal()
+ * https://github.com/dotnet/runtime/blob/2a99e18eedabcf1add064c099da59d9301ce45e0/src/libraries/System.Private.CoreLib/src/System/IO/Path.Windows.cs#L126
+ */
+static inline int get_extended_win32_path(const char *path, wchar_t **ppath_w)
+{
+    int ret;
+    size_t len;
+
+    if ((ret = utf8towchar(path, ppath_w)) < 0)
+        return ret;
+
+    if (path_is_extended(*ppath_w)) {
+        /* Paths prefixed with '\\?\' or \??\' are considered normalized by definition.
+         * Windows doesn't normalize those paths and neither should we.
+         */
+        return 0;
+    }
+
+    if ((ret = path_normalize(ppath_w)) < 0) {
+        av_freep(ppath_w);
+        return ret;
+    }
+
+    /* see .NET6: PathInternal.EnsureExtendedPrefixIfNeeded()
+     * https://github.com/dotnet/runtime/blob/9260c249140ef90b4299d0fe1aa3037e25228518/src/libraries/Common/src/System/IO/PathInternal.Windows.cs#L92
+     */
+    len = wcslen(*ppath_w);
+    if (len >= MAX_PATH) {
+        if ((ret = add_extended_prefix(ppath_w)) < 0) {
+            av_freep(ppath_w);
+            return ret;
+        }
+    }
+
+    return 0;
+}
+
 #endif
 
 #endif /* AVUTIL_WCHAR_FILENAME_H */
-- 
ffmpeg-codebot

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v8 2/3] avformat/os_support: Support long file names on Windows

[softworkz]

From: softworkz <softworkz@hotmail.com>

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavformat/os_support.h | 116 ++++++++++++++++++++++++++++++---------
 1 file changed, 91 insertions(+), 25 deletions(-)

diff --git a/libavformat/os_support.h b/libavformat/os_support.h
index 5e6b32d2dc..958976cc6f 100644
--- a/libavformat/os_support.h
+++ b/libavformat/os_support.h
@@ -49,11 +49,38 @@
 #  ifdef stat
 #   undef stat
 #  endif
-#  define stat _stati64
+
+#  define stat win32_stat
+
+    /*
+     * The POSIX definition for the stat() function uses a struct of the
+     * same name (struct stat), that why it takes this extra effort  for
+     * redirecting/replacing the stat() function with our own one which
+     * is capable to handle long path names on Windows.
+     * The struct below roughly follows the POSIX definition. Time values
+     * are 64bit, but in cases when _USE_32BIT_TIME_T is defined, they
+     * will be set to values no larger than INT32_MAX which corresponds
+     * to file times up to the year 2038.
+     */
+    struct win32_stat
+    {
+        _dev_t         st_dev;     /* ID of device containing file */
+        _ino_t         st_ino;     /* inode number */
+        unsigned short st_mode;    /* protection */
+        short          st_nlink;   /* number of hard links */
+        short          st_uid;     /* user ID of owner */
+        short          st_gid;     /* group ID of owner */
+        _dev_t         st_rdev;    /* device ID (if special file) */
+        int64_t        st_size;    /* total size, in bytes */
+        int64_t        st_atime;   /* time of last access */
+        int64_t        st_mtime;   /* time of last modification */
+        int64_t        st_ctime;   /* time of last status change */
+    };
+
 #  ifdef fstat
 #   undef fstat
 #  endif
-#  define fstat(f,s) _fstati64((f), (s))
+#  define fstat win32_fstat
 #endif /* defined(_WIN32) */
 
 
@@ -153,7 +180,7 @@ static inline int win32_##name(const char *filename_utf8) \
     wchar_t *filename_w;                                  \
     int ret;                                              \
                                                           \
-    if (utf8towchar(filename_utf8, &filename_w))          \
+    if (get_extended_win32_path(filename_utf8, &filename_w)) \
         return -1;                                        \
     if (!filename_w)                                      \
         goto fallback;                                    \
@@ -171,37 +198,76 @@ DEF_FS_FUNCTION(unlink, _wunlink, _unlink)
 DEF_FS_FUNCTION(mkdir,  _wmkdir,  _mkdir)
 DEF_FS_FUNCTION(rmdir,  _wrmdir , _rmdir)
 
-#define DEF_FS_FUNCTION2(name, wfunc, afunc, partype)     \
-static inline int win32_##name(const char *filename_utf8, partype par) \
-{                                                         \
-    wchar_t *filename_w;                                  \
-    int ret;                                              \
-                                                          \
-    if (utf8towchar(filename_utf8, &filename_w))          \
-        return -1;                                        \
-    if (!filename_w)                                      \
-        goto fallback;                                    \
-                                                          \
-    ret = wfunc(filename_w, par);                         \
-    av_free(filename_w);                                  \
-    return ret;                                           \
-                                                          \
-fallback:                                                 \
-    /* filename may be be in CP_ACP */                    \
-    return afunc(filename_utf8, par);                     \
+static inline int win32_access(const char *filename_utf8, int mode)
+{
+    wchar_t *filename_w;
+    int ret;
+    if (get_extended_win32_path(filename_utf8, &filename_w))
+        return -1;
+    if (!filename_w)
+        goto fallback;
+    ret = _waccess(filename_w, mode);
+    av_free(filename_w);
+    return ret;
+fallback:
+    return _access(filename_utf8, mode);
+}
+
+static inline void copy_stat(struct _stati64 *crtstat, struct win32_stat *buf)
+{
+    buf->st_dev   = crtstat->st_dev;
+    buf->st_ino   = crtstat->st_ino;
+    buf->st_mode  = crtstat->st_mode;
+    buf->st_nlink = crtstat->st_nlink;
+    buf->st_uid   = crtstat->st_uid;
+    buf->st_gid   = crtstat->st_gid;
+    buf->st_rdev  = crtstat->st_rdev;
+    buf->st_size  = crtstat->st_size;
+    buf->st_atime = crtstat->st_atime;
+    buf->st_mtime = crtstat->st_mtime;
+    buf->st_ctime = crtstat->st_ctime;
 }
 
-DEF_FS_FUNCTION2(access, _waccess, _access, int)
-DEF_FS_FUNCTION2(stat, _wstati64, _stati64, struct stat*)
+static inline int win32_stat(const char *filename_utf8, struct win32_stat *buf)
+{
+    struct _stati64 crtstat = { 0 };
+    wchar_t *filename_w;
+    int ret;
+
+    if (get_extended_win32_path(filename_utf8, &filename_w))
+        return -1;
+
+    if (filename_w) {
+        ret = _wstat64(filename_w, &crtstat);
+        av_free(filename_w);
+    } else
+        ret = _stat64(filename_utf8, &crtstat);
+
+    copy_stat(&crtstat, buf);
+
+    return ret;
+}
+
+static inline int win32_fstat(int fd, struct win32_stat *buf)
+{
+    struct _stati64 crtstat = { 0 };
+    int ret;
+
+    ret = _fstat64(fd, &crtstat);
+
+    copy_stat(&crtstat, buf);
+
+    return ret;
+}
 
 static inline int win32_rename(const char *src_utf8, const char *dest_utf8)
 {
     wchar_t *src_w, *dest_w;
     int ret;
 
-    if (utf8towchar(src_utf8, &src_w))
+    if (get_extended_win32_path(src_utf8, &src_w))
         return -1;
-    if (utf8towchar(dest_utf8, &dest_w)) {
+    if (get_extended_win32_path(dest_utf8, &dest_w)) {
         av_free(src_w);
         return -1;
     }
-- 
ffmpeg-codebot

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v8 3/3] avformat/file: remove _WIN32 condition

[softworkz]

From: softworkz <softworkz@hotmail.com>

stat is now re-mapped with long path support
in os_support.h

Signed-off-by: softworkz <softworkz@hotmail.com>
---
 libavformat/file.c | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/libavformat/file.c b/libavformat/file.c
index 063d7c5aa2..98c9e81bcb 100644
--- a/libavformat/file.c
+++ b/libavformat/file.c
@@ -154,11 +154,7 @@ static int file_check(URLContext *h, int mask)
             ret |= AVIO_FLAG_WRITE;
 #else
     struct stat st;
-#   ifndef _WIN32
     ret = stat(filename, &st);
-#   else
-    ret = win32_stat(filename, &st);
-#   endif
     if (ret < 0)
         return AVERROR(errno);
 
-- 
ffmpeg-codebot
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows

[Martin Storsjö]

On Thu, 26 May 2022, ffmpegagent wrote:

> This patchset adds support for long file and directory paths on Windows. The
> implementation follows the same logic that .NET is using internally, with
> the only exception that it doesn't expand short path components in 8.3
> format. .NET does this as the same function is also used for other purposes,
> but in our case, that's not required. Short (8.3) paths are working as well
> with the extended path prefix, even when longer than 260.
>
> Successfully tested:
>
> * Regular paths wth drive letter
> * Regular UNC paths
> * Long paths wth drive letter
> * Long paths wth drive letter and forward slashes
> * Long UNC paths
> * Prefixed paths wth drive letter
> * Prefixed UNC paths
>
> I have kept the individual functions separate on purpose, to make it easy to
> compare with the .NET impl. (compilers should inlinie those anyway)
>
> v2
>
> * wchar_filename: Improve comments and function documentation
> * os_support: adjust defines to use win32_stat
>
> v3
>
> * removed length check in path_is_extended()
> * added path_is_device_path() check in add_extended_prefix()
> * add_extended_prefix(): clarified doc and add checks
> * clarified string allocation length calculation
> * replaced 260 with MAX_PATH
> * removed redundant checks after normalization
>
> v4
>
> * rebased. no changes
>
> v5
>
> * resolved the ugly struct duplication
> * compatible with _USE_32BIT_TIME_T
>
> v6
>
> * wchar_filename.h: added links to .NET source code
> * wchar_filename.h: free allocations on error
> * os_support.hs: use clean and safe way to redirect stat() calls
>
> v7
>
> * os_support.h: remapped fstat with win32_stat structure
> * os_support.h: use int64_t for some members
> * avformat/file: remove _WIN32 condition
>
> v8
>
> * os_support.h: documented win32_stat structure
> * os_support.h: renamed function parameters
>
> softworkz (3):
>  avutil/wchar_filename,file_open: Support long file names on Windows
>  avformat/os_support: Support long file names on Windows
>  avformat/file: remove _WIN32 condition
>
> libavformat/file.c         |   4 -
> libavformat/os_support.h   | 116 ++++++++++++++++++------
> libavutil/file_open.c      |   2 +-
> libavutil/wchar_filename.h | 180 +++++++++++++++++++++++++++++++++++++
> 4 files changed, 272 insertions(+), 30 deletions(-)

This is still ok with me, fwiw.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows

[Martin Storsjö]

On Thu, 26 May 2022, ffmpegagent wrote:

> This patchset adds support for long file and directory paths on Windows. The
> implementation follows the same logic that .NET is using internally, with
> the only exception that it doesn't expand short path components in 8.3
> format. .NET does this as the same function is also used for other purposes,
> but in our case, that's not required. Short (8.3) paths are working as well
> with the extended path prefix, even when longer than 260.
>
> Successfully tested:
>
> * Regular paths wth drive letter
> * Regular UNC paths
> * Long paths wth drive letter
> * Long paths wth drive letter and forward slashes
> * Long UNC paths
> * Prefixed paths wth drive letter
> * Prefixed UNC paths
>
> I have kept the individual functions separate on purpose, to make it easy to
> compare with the .NET impl. (compilers should inlinie those anyway)
>
> v2
>
> * wchar_filename: Improve comments and function documentation
> * os_support: adjust defines to use win32_stat
>
> v3
>
> * removed length check in path_is_extended()
> * added path_is_device_path() check in add_extended_prefix()
> * add_extended_prefix(): clarified doc and add checks
> * clarified string allocation length calculation
> * replaced 260 with MAX_PATH
> * removed redundant checks after normalization
>
> v4
>
> * rebased. no changes
>
> v5
>
> * resolved the ugly struct duplication
> * compatible with _USE_32BIT_TIME_T
>
> v6
>
> * wchar_filename.h: added links to .NET source code
> * wchar_filename.h: free allocations on error
> * os_support.hs: use clean and safe way to redirect stat() calls
>
> v7
>
> * os_support.h: remapped fstat with win32_stat structure
> * os_support.h: use int64_t for some members
> * avformat/file: remove _WIN32 condition
>
> v8
>
> * os_support.h: documented win32_stat structure
> * os_support.h: renamed function parameters
>
> softworkz (3):
>  avutil/wchar_filename,file_open: Support long file names on Windows
>  avformat/os_support: Support long file names on Windows
>  avformat/file: remove _WIN32 condition
>
> libavformat/file.c         |   4 -
> libavformat/os_support.h   | 116 ++++++++++++++++++------
> libavutil/file_open.c      |   2 +-
> libavutil/wchar_filename.h | 180 +++++++++++++++++++++++++++++++++++++
> 4 files changed, 272 insertions(+), 30 deletions(-)

This looks fine to me, and the discussion seems to have converged, so I'll 
go ahead and push this.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows

[nil-admirari]

> This looks fine to me, and the discussion seems to have converged, so I'll 
> go ahead and push this.

Build is now failing: https://github.com/yt-dlp/FFmpeg-Builds/runs/6819319105?check_suite_focus=true.

In file included from /opt/ct-ng/i686-w64-mingw32/sysroot/mingw/include/dshow.h:33,
                 from libavcodec/mf_utils.h:32,
                 from libavcodec/mfenc.c:26:
./libavutil/wchar_filename.h: In function 'add_extended_prefix':
./libavutil/wchar_filename.h:211:9: error: 'wcscpy_instead_use_StringCbCopyW_or_StringCchCopyW' undeclared (first use in this function)
  211 |         wcscpy(temp_w, unc_prefix);
      |         ^~~~~~
./libavutil/wchar_filename.h:211:9: note: each undeclared identifier is reported only once for each function it appears in

wchar.h is indeed not included in wchar_filename.h.



Incompatible pointer types warning is still there as well:

In file included from ./libavformat/internal.h:30,
                 from libavdevice/alldevices.c:21:
./libavformat/os_support.h: In function 'win32_stat':
./libavformat/os_support.h:241:36: warning: passing argument 2 of '_wstat64' from incompatible pointer type [-Wincompatible-pointer-types]
  241 |         ret = _wstat64(filename_w, &crtstat);
      |                                    ^~~~~~~~
      |                                    |
      |                                    struct _stati64 *
In file included from ./libavformat/os_support.h:32,
                 from ./libavformat/internal.h:30,
                 from libavdevice/alldevices.c:21:
/opt/ct-ng/i686-w64-mingw32/sysroot/mingw/include/sys/stat.h:129:69: note: expected 'struct _stat64 *' but argument is of type 'struct _stati64 *'
  129 |   _CRTIMP int __cdecl _wstat64(const wchar_t *_Name,struct _stat64 *_Stat);
      |                                                     ~~~~~~~~~~~~~~~~^~~~~





_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Thursday, June 9, 2022 9:37 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows
> 
> > This looks fine to me, and the discussion seems to have converged, so I'll
> > go ahead and push this.
> 
> Build is now failing: https://github.com/yt-dlp/FFmpeg-
> Builds/runs/6819319105?check_suite_focus=true.
> 
> In file included from /opt/ct-ng/i686-w64-
> mingw32/sysroot/mingw/include/dshow.h:33,
>                  from libavcodec/mf_utils.h:32,
>                  from libavcodec/mfenc.c:26:
> ./libavutil/wchar_filename.h: In function 'add_extended_prefix':
> ./libavutil/wchar_filename.h:211:9: error:
> 'wcscpy_instead_use_StringCbCopyW_or_StringCchCopyW' undeclared (first use in
> this function)
>   211 |         wcscpy(temp_w, unc_prefix);
>       |         ^~~~~~
> ./libavutil/wchar_filename.h:211:9: note: each undeclared identifier is
> reported only once for each function it appears in
> 
> wchar.h is indeed not included in wchar_filename.h.

wcscpy is defined in corecrt_wstring.h, included in string.h, 
included in winnt.h included in windef.h, included in windows.h


> Incompatible pointer types warning is still there as well:
> 
> In file included from ./libavformat/internal.h:30,
>                  from libavdevice/alldevices.c:21:
> ./libavformat/os_support.h: In function 'win32_stat':
> ./libavformat/os_support.h:241:36: warning: passing argument 2 of '_wstat64'
> from incompatible pointer type [-Wincompatible-pointer-types]
>   241 |         ret = _wstat64(filename_w, &crtstat);
>       |                                    ^~~~~~~~
>       |                                    |
>       |                                    struct _stati64 *
> In file included from ./libavformat/os_support.h:32,
>                  from ./libavformat/internal.h:30,
>                  from libavdevice/alldevices.c:21:
> /opt/ct-ng/i686-w64-mingw32/sysroot/mingw/include/sys/stat.h:129:69: note:
> expected 'struct _stat64 *' but argument is of type 'struct _stati64 *'
>   129 |   _CRTIMP int __cdecl _wstat64(const wchar_t *_Name,struct _stat64
> *_Stat);
>       |
> ~~~~~~~~~~~~~~~~^~~~~

Yea, right. We need to get rid of the 'i' in the struct
types for getting fully independent of USE_32BIT_TIME_T.

Will send an update in a minute.

Thanks for reporting.

sw
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows

[Martin Storsjö]

On Thu, 9 Jun 2022, nil-admirari@mailo.com wrote:

>> This looks fine to me, and the discussion seems to have converged, so I'll
>> go ahead and push this.
>
> Build is now failing: https://github.com/yt-dlp/FFmpeg-Builds/runs/6819319105?check_suite_focus=true.
>
> In file included from /opt/ct-ng/i686-w64-mingw32/sysroot/mingw/include/dshow.h:33,
>                 from libavcodec/mf_utils.h:32,
>                 from libavcodec/mfenc.c:26:
> ./libavutil/wchar_filename.h: In function 'add_extended_prefix':
> ./libavutil/wchar_filename.h:211:9: error: 'wcscpy_instead_use_StringCbCopyW_or_StringCchCopyW' undeclared (first use in this function)
>  211 |         wcscpy(temp_w, unc_prefix);
>      |         ^~~~~~
> ./libavutil/wchar_filename.h:211:9: note: each undeclared identifier is reported only once for each function it appears in

This error isn't reproducible in git master - it's triggered by your 
yet-unmerged patches (that include wchar_filename.h in w32dlfcn.h).

> Incompatible pointer types warning is still there as well:
>
> In file included from ./libavformat/internal.h:30,
>                 from libavdevice/alldevices.c:21:
> ./libavformat/os_support.h: In function 'win32_stat':
> ./libavformat/os_support.h:241:36: warning: passing argument 2 of '_wstat64' from incompatible pointer type [-Wincompatible-pointer-types]
>  241 |         ret = _wstat64(filename_w, &crtstat);
>      |                                    ^~~~~~~~
>      |                                    |
>      |                                    struct _stati64 *
> In file included from ./libavformat/os_support.h:32,
>                 from ./libavformat/internal.h:30,
>                 from libavdevice/alldevices.c:21:
> /opt/ct-ng/i686-w64-mingw32/sysroot/mingw/include/sys/stat.h:129:69: note: expected 'struct _stat64 *' but argument is of type 'struct _stati64 *'
>  129 |   _CRTIMP int __cdecl _wstat64(const wchar_t *_Name,struct _stat64 *_Stat);
>      |                                                     ~~~~~~~~~~~~~~~~^~~~~

This I can indeed reproduce now. I did build these patches in a couple 
other environments (both mingw and msvc) where those warnings didn't 
appear, but now I managed to find one that shows those warnings.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows

[nil-admirari]

> wcscpy is defined in corecrt_wstring.h, included in string.h, 
> included in winnt.h included in windef.h, included in windows.h

Not so easy, it's the same as https://trac.ffmpeg.org/ticket/999:

> In file included from /opt/ct-ng/i686-w64-
> mingw32/sysroot/mingw/include/dshow.h:33

NO_DSHOW_STRSAFE must be defined somewhere.



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows

[nil-admirari]

> This error isn't reproducible in git master - it's triggered by your 
> yet-unmerged patches (that include wchar_filename.h in w32dlfcn.h).

Ok. It can be fixed by either
- defining NO_DSHOW_STRSAFE in libavcodec/mf_utils.h
- or by migrating os_support.h to StrSafe.h functions.

Which way is preferable?



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows

[Martin Storsjö]

On Thu, 9 Jun 2022, nil-admirari@mailo.com wrote:

>> This error isn't reproducible in git master - it's triggered by your
>> yet-unmerged patches (that include wchar_filename.h in w32dlfcn.h).
>
> Ok. It can be fixed by either
> - defining NO_DSHOW_STRSAFE in libavcodec/mf_utils.h
> - or by migrating os_support.h to StrSafe.h functions.
>
> Which way is preferable?

I think avoiding wcscat (with whatever usable alternative, not necessarily 
from strsafe.h) is the more robust solution, instead of having to play 
whack-a-mole with silencing such warnings. The 10 year old trac you 
referenced mentioned that the strsafe.h alternative functions weren't 
available in all toolchains used at that time though.

Or if we'd add the define projectwide in e.g. configure it probably 
wouldn't be that bad? Kinda like how we already add 
"-D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_WARNINGS" in MSVC builds. 
Then we wouldn't need to worry about missing it somewhere accidentally.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> admirari@mailo.com
> Sent: Thursday, June 9, 2022 10:58 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows
> 
> > This error isn't reproducible in git master - it's triggered by your
> > yet-unmerged patches (that include wchar_filename.h in w32dlfcn.h).
> 
> Ok. It can be fixed by either
> - defining NO_DSHOW_STRSAFE in libavcodec/mf_utils.h
> - or by migrating os_support.h to StrSafe.h functions.
> 
> Which way is preferable?

I think the def is at least better for an instant fix.
Also, nowhere in ffmpeg are those "secure" functions being used, so it
aligns to current practice.

sw
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Soft Works
> Sent: Thursday, June 9, 2022 10:16 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows
> 
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of nil-
> > admirari@mailo.com
> > Sent: Thursday, June 9, 2022 9:37 PM
> > To: ffmpeg-devel@ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on
> Windows
> >
> > > This looks fine to me, and the discussion seems to have converged, so
> I'll
> > > go ahead and push this.
> >
> > Build is now failing: https://github.com/yt-dlp/FFmpeg-
> > Builds/runs/6819319105?check_suite_focus=true.
> >
> > In file included from /opt/ct-ng/i686-w64-
> > mingw32/sysroot/mingw/include/dshow.h:33,
> >                  from libavcodec/mf_utils.h:32,
> >                  from libavcodec/mfenc.c:26:
> > ./libavutil/wchar_filename.h: In function 'add_extended_prefix':
> > ./libavutil/wchar_filename.h:211:9: error:
> > 'wcscpy_instead_use_StringCbCopyW_or_StringCchCopyW' undeclared (first use
> in
> > this function)
> >   211 |         wcscpy(temp_w, unc_prefix);
> >       |         ^~~~~~
> > ./libavutil/wchar_filename.h:211:9: note: each undeclared identifier is
> > reported only once for each function it appears in
> >
> > wchar.h is indeed not included in wchar_filename.h.
> 
> wcscpy is defined in corecrt_wstring.h, included in string.h,
> included in winnt.h included in windef.h, included in windows.h
> 
> 
> > Incompatible pointer types warning is still there as well:
> >
> > In file included from ./libavformat/internal.h:30,
> >                  from libavdevice/alldevices.c:21:
> > ./libavformat/os_support.h: In function 'win32_stat':
> > ./libavformat/os_support.h:241:36: warning: passing argument 2 of
> '_wstat64'
> > from incompatible pointer type [-Wincompatible-pointer-types]
> >   241 |         ret = _wstat64(filename_w, &crtstat);
> >       |                                    ^~~~~~~~
> >       |                                    |
> >       |                                    struct _stati64 *
> > In file included from ./libavformat/os_support.h:32,
> >                  from ./libavformat/internal.h:30,
> >                  from libavdevice/alldevices.c:21:
> > /opt/ct-ng/i686-w64-mingw32/sysroot/mingw/include/sys/stat.h:129:69: note:
> > expected 'struct _stat64 *' but argument is of type 'struct _stati64 *'
> >   129 |   _CRTIMP int __cdecl _wstat64(const wchar_t *_Name,struct _stat64
> > *_Stat);
> >       |
> > ~~~~~~~~~~~~~~~~^~~~~
> 
> Yea, right. We need to get rid of the 'i' in the struct
> types for getting fully independent of USE_32BIT_TIME_T.
> 
> Will send an update in a minute.

Here's a patch for this part:

https://github.com/ffstaging/FFmpeg/pull/34

https://github.com/ffstaging/FFmpeg/pull/34.patch

Can't send it right now because Google decided to no longer
username/password login (which affects ffmpegagent). It's
said to work with an "app password" which in turn can only
be used when 2step verification is set up, and that again
requires either a phone or a security key :-(

Does anybody know a "security key" emulator.

softworkz
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v8 0/3] Support long file names on Windows

[nil-admirari]

> ...
> > - defining NO_DSHOW_STRSAFE in libavcodec/mf_utils.h
> ...
> Or if we'd add the define projectwide in e.g. configure it probably 
> wouldn't be that bad? Kinda like how we already add 
> "-D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_WARNINGS" in MSVC builds. 
> Then we wouldn't need to worry about missing it somewhere accidentally.

Ended up defining NO_DSHOW_STRSAFE in mf_utils.h,
just like dshow_capture.h does (these are the only two uses).

-D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_WARNINGS
are defined only for MVSC:

elif test_${pfx}cpp_condition crtversion.h "defined _VC_CRT_MAJOR_VERSION"; then

NO_DSHOW_STRSAFE should cover MinGW as well, and probably others.
If you still want global NO_DSHOW_STRSAFE, please point where to add it exactly.



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".