Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
 help / color / mirror / Atom feed
* [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
@ 2022-05-20 21:12 Martin Storsjö
  2022-05-20 21:12 ` [FFmpeg-devel] [PATCH 2/3] libavutil: Deprecate av_fopen_utf8, provide an avpriv version Martin Storsjö
                   ` (2 more replies)
  0 siblings, 3 replies; 15+ messages in thread
From: Martin Storsjö @ 2022-05-20 21:12 UTC (permalink / raw)
  To: ffmpeg-devel

Provide a header based inline reimplementation of it.

Using av_fopen_utf8 doesn't work outside of the libraries when built
with MSVC as shared libraries (in the default configuration, where
each DLL gets a separate statically linked CRT).
---
 fftools/ffmpeg_opt.c |  3 +-
 fftools/fopen_utf8.h | 71 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 73 insertions(+), 1 deletion(-)
 create mode 100644 fftools/fopen_utf8.h

diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c
index 47e8b9b7bd..a5cd989d35 100644
--- a/fftools/ffmpeg_opt.c
+++ b/fftools/ffmpeg_opt.c
@@ -28,6 +28,7 @@
 #endif
 
 #include "ffmpeg.h"
+#include "fopen_utf8.h"
 #include "cmdutils.h"
 #include "opt_common.h"
 
@@ -1882,7 +1883,7 @@ static OutputStream *new_video_stream(OptionsContext *o, AVFormatContext *oc, in
                     video_enc->stats_in = logbuffer;
                 }
                 if (video_enc->flags & AV_CODEC_FLAG_PASS1) {
-                    f = av_fopen_utf8(logfilename, "wb");
+                    f = fopen_utf8(logfilename, "wb");
                     if (!f) {
                         av_log(NULL, AV_LOG_FATAL,
                                "Cannot write log file '%s' for pass-1 encoding: %s\n",
diff --git a/fftools/fopen_utf8.h b/fftools/fopen_utf8.h
new file mode 100644
index 0000000000..db57fcaec4
--- /dev/null
+++ b/fftools/fopen_utf8.h
@@ -0,0 +1,71 @@
+/*
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef FFTOOLS_FOPEN_UTF8_H
+#define FFTOOLS_FOPEN_UTF8_H
+
+#include <stdio.h>
+
+/* The fopen_utf8 function here is essentially equivalent to av_fopen_utf8,
+ * except that it doesn't set O_CLOEXEC, and that it isn't exported
+ * from a different library. (On Windows, each DLL might use a different
+ * CRT, and FILE* handles can't be shared across them.) */
+
+#ifdef _WIN32
+#include "libavutil/wchar_filename.h"
+
+static inline FILE *fopen_utf8(const char *path_utf8, const char *mode)
+{
+    wchar_t *path_w, *mode_w;
+    FILE *f;
+
+    /* convert UTF-8 to wide chars */
+    if (utf8towchar(path_utf8, &path_w)) /* This sets errno on error. */
+        return NULL;
+    if (!path_w)
+        goto fallback;
+
+    if (utf8towchar(mode, &mode_w))
+        return NULL;
+    if (!mode_w) {
+        /* If failing to interpret the mode string as utf8, it is an invalid
+         * parameter. */
+        av_freep(&path_w);
+        errno = EINVAL;
+        return NULL;
+    }
+
+    f = _wfopen(path_w, mode_w);
+    av_freep(&path_w);
+    av_freep(&mode_w);
+
+    return f;
+fallback:
+    /* path may be in CP_ACP */
+    return fopen(path_utf8, mode);
+}
+
+#else
+
+static inline FILE *fopen_utf8(const char *path, const char *mode)
+{
+    return fopen(path, mode);
+}
+#endif
+
+#endif /* FFTOOLS_FOPEN_UTF8_H */
-- 
2.32.0 (Apple Git-132)

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [FFmpeg-devel] [PATCH 2/3] libavutil: Deprecate av_fopen_utf8, provide an avpriv version
  2022-05-20 21:12 [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8 Martin Storsjö
@ 2022-05-20 21:12 ` Martin Storsjö
  2022-05-20 21:12 ` [FFmpeg-devel] [PATCH 3/3] Switch uses of av_fopen_utf8 to avpriv_fopen_utf8 Martin Storsjö
  2022-05-21  5:07 ` [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8 Soft Works
  2 siblings, 0 replies; 15+ messages in thread
From: Martin Storsjö @ 2022-05-20 21:12 UTC (permalink / raw)
  To: ffmpeg-devel

Since every DLL can use an individual CRT on Windows, having
an exported function that opens a FILE* won't work if that
FILE* is going to be used from a different DLL (or from user
application code).

Internally within the libraries, the issue can be worked around
by duplicating the function in all libraries (this already happened
implicitly because the function resided in file_open.c) and renaming
the function to ff_fopen_utf8 (so that it doesn't end up exported from
the DLLs) and duplicating it in all libraries that use it.

That mechanism doesn't work for external users, thus deprecate the
existing function.
---
 doc/APIchanges          | 3 +++
 fftools/fopen_utf8.h    | 2 +-
 libavfilter/Makefile    | 1 +
 libavfilter/file_open.c | 1 +
 libavutil/avutil.h      | 6 ++++++
 libavutil/file_open.c   | 9 ++++++++-
 libavutil/internal.h    | 8 ++++++++
 libavutil/version.h     | 1 +
 tests/ref/fate/source   | 1 +
 9 files changed, 30 insertions(+), 2 deletions(-)
 create mode 100644 libavfilter/file_open.c

diff --git a/doc/APIchanges b/doc/APIchanges
index 1a9f0a303e..c3a1649079 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,9 @@ libavutil:     2021-04-27
 
 API changes, most recent first:
 
+2022-xx-xx - xxxxxxxxx - lavu 57.xx.xxx - avutil.h
+  Deprecate av_fopen_utf8() without replacement.
+
 2022-03-16 - xxxxxxxxxx - all libraries - version_major.h
   Add lib<name>/version_major.h as new installed headers, which only
   contain the major version number (and corresponding API deprecation
diff --git a/fftools/fopen_utf8.h b/fftools/fopen_utf8.h
index db57fcaec4..cd18fe8ce1 100644
--- a/fftools/fopen_utf8.h
+++ b/fftools/fopen_utf8.h
@@ -21,7 +21,7 @@
 
 #include <stdio.h>
 
-/* The fopen_utf8 function here is essentially equivalent to av_fopen_utf8,
+/* The fopen_utf8 function here is essentially equivalent to avpriv_fopen_utf8,
  * except that it doesn't set O_CLOEXEC, and that it isn't exported
  * from a different library. (On Windows, each DLL might use a different
  * CRT, and FILE* handles can't be shared across them.) */
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index ee2ea51e69..78ccfa37d3 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -23,6 +23,7 @@ OBJS = allfilters.o                                                     \
        version.o                                                        \
        video.o                                                          \
 
+OBJS-$(HAVE_LIBC_MSVCRT)                     += file_open.o
 OBJS-$(HAVE_THREADS)                         += pthread.o
 
 # subsystems
diff --git a/libavfilter/file_open.c b/libavfilter/file_open.c
new file mode 100644
index 0000000000..494a5d37a4
--- /dev/null
+++ b/libavfilter/file_open.c
@@ -0,0 +1 @@
+#include "libavutil/file_open.c"
diff --git a/libavutil/avutil.h b/libavutil/avutil.h
index 4d633156d1..64b68bdbd3 100644
--- a/libavutil/avutil.h
+++ b/libavutil/avutil.h
@@ -331,12 +331,18 @@ unsigned av_int_list_length_for_size(unsigned elsize,
 #define av_int_list_length(list, term) \
     av_int_list_length_for_size(sizeof(*(list)), list, term)
 
+#if FF_API_AV_FOPEN_UTF8
 /**
  * Open a file using a UTF-8 filename.
  * The API of this function matches POSIX fopen(), errors are returned through
  * errno.
+ * @deprecated Avoid using it, as on Windows, the FILE* allocated by this
+ *             function may be allocated with a different CRT than the caller
+ *             who uses the FILE*. No replacement provided in public API.
  */
+attribute_deprecated
 FILE *av_fopen_utf8(const char *path, const char *mode);
+#endif
 
 /**
  * Return the fractional representation of the internal time base.
diff --git a/libavutil/file_open.c b/libavutil/file_open.c
index cc302f2f76..fb64c2e4ee 100644
--- a/libavutil/file_open.c
+++ b/libavutil/file_open.c
@@ -155,7 +155,7 @@ int avpriv_tempfile(const char *prefix, char **filename, int log_offset, void *l
     return fd; /* success */
 }
 
-FILE *av_fopen_utf8(const char *path, const char *mode)
+FILE *avpriv_fopen_utf8(const char *path, const char *mode)
 {
     int fd;
     int access;
@@ -188,3 +188,10 @@ FILE *av_fopen_utf8(const char *path, const char *mode)
         return NULL;
     return fdopen(fd, mode);
 }
+
+#if FF_API_AV_FOPEN_UTF8
+FILE *av_fopen_utf8(const char *path, const char *mode)
+{
+    return avpriv_fopen_utf8(path, mode);
+}
+#endif
diff --git a/libavutil/internal.h b/libavutil/internal.h
index 79c2130be0..b44cbaaa7b 100644
--- a/libavutil/internal.h
+++ b/libavutil/internal.h
@@ -37,6 +37,7 @@
 #include <stdint.h>
 #include <stddef.h>
 #include <assert.h>
+#include <stdio.h>
 #include "config.h"
 #include "attributes.h"
 #include "timer.h"
@@ -183,8 +184,10 @@ void avpriv_request_sample(void *avc,
 #pragma comment(linker, "/include:" EXTERN_PREFIX "avpriv_snprintf")
 #endif
 
+#define avpriv_fopen_utf8 ff_fopen_utf8
 #define avpriv_open ff_open
 #define avpriv_tempfile ff_tempfile
+
 #define PTRDIFF_SPECIFIER "Id"
 #define SIZE_SPECIFIER "Iu"
 #else
@@ -256,6 +259,11 @@ static av_always_inline av_const int64_t ff_rint64_clip(double a, int64_t amin,
 av_warn_unused_result
 int avpriv_open(const char *filename, int flags, ...);
 
+/**
+ * Open a file using a UTF-8 filename.
+ */
+FILE *avpriv_fopen_utf8(const char *path, const char *mode);
+
 /**
  * Wrapper to work around the lack of mkstemp() on mingw.
  * Also, tries to create file in /tmp first, if possible.
diff --git a/libavutil/version.h b/libavutil/version.h
index 6735c20090..8532051c00 100644
--- a/libavutil/version.h
+++ b/libavutil/version.h
@@ -113,6 +113,7 @@
 #define FF_API_FIFO_OLD_API             (LIBAVUTIL_VERSION_MAJOR < 58)
 #define FF_API_XVMC                     (LIBAVUTIL_VERSION_MAJOR < 58)
 #define FF_API_OLD_CHANNEL_LAYOUT       (LIBAVUTIL_VERSION_MAJOR < 58)
+#define FF_API_AV_FOPEN_UTF8            (LIBAVUTIL_VERSION_MAJOR < 58)
 
 /**
  * @}
diff --git a/tests/ref/fate/source b/tests/ref/fate/source
index 69dcdc4f27..16ea7ef9c1 100644
--- a/tests/ref/fate/source
+++ b/tests/ref/fate/source
@@ -8,6 +8,7 @@ libavcodec/reverse.c
 libavdevice/file_open.c
 libavdevice/reverse.c
 libavfilter/af_arnndn.c
+libavfilter/file_open.c
 libavfilter/log2_tab.c
 libavformat/file_open.c
 libavformat/golomb_tab.c
-- 
2.32.0 (Apple Git-132)

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [FFmpeg-devel] [PATCH 3/3] Switch uses of av_fopen_utf8 to avpriv_fopen_utf8
  2022-05-20 21:12 [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8 Martin Storsjö
  2022-05-20 21:12 ` [FFmpeg-devel] [PATCH 2/3] libavutil: Deprecate av_fopen_utf8, provide an avpriv version Martin Storsjö
@ 2022-05-20 21:12 ` Martin Storsjö
  2022-05-21  5:07 ` [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8 Soft Works
  2 siblings, 0 replies; 15+ messages in thread
From: Martin Storsjö @ 2022-05-20 21:12 UTC (permalink / raw)
  To: ffmpeg-devel

---
 libavfilter/af_arnndn.c       | 2 +-
 libavfilter/opencl.c          | 2 +-
 libavfilter/vf_curves.c       | 2 +-
 libavfilter/vf_dnn_classify.c | 2 +-
 libavfilter/vf_dnn_detect.c   | 2 +-
 libavfilter/vf_fieldhint.c    | 2 +-
 libavfilter/vf_lut3d.c        | 4 ++--
 libavfilter/vf_nnedi.c        | 2 +-
 libavfilter/vf_paletteuse.c   | 2 +-
 libavformat/ipfsgateway.c     | 2 +-
 10 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/libavfilter/af_arnndn.c b/libavfilter/af_arnndn.c
index fe562e81a1..5e3403ca6a 100644
--- a/libavfilter/af_arnndn.c
+++ b/libavfilter/af_arnndn.c
@@ -1478,7 +1478,7 @@ static int open_model(AVFilterContext *ctx, RNNModel **model)
 
     if (!s->model_name)
         return AVERROR(EINVAL);
-    f = av_fopen_utf8(s->model_name, "r");
+    f = avpriv_fopen_utf8(s->model_name, "r");
     if (!f) {
         av_log(ctx, AV_LOG_ERROR, "Failed to open model file: %s\n", s->model_name);
         return AVERROR(EINVAL);
diff --git a/libavfilter/opencl.c b/libavfilter/opencl.c
index 70d5edb78c..c8e7e6e1a5 100644
--- a/libavfilter/opencl.c
+++ b/libavfilter/opencl.c
@@ -210,7 +210,7 @@ int ff_opencl_filter_load_program_from_file(AVFilterContext *avctx,
     const char *src_const;
     int err;
 
-    file = av_fopen_utf8(filename, "r");
+    file = avpriv_fopen_utf8(filename, "r");
     if (!file) {
         av_log(avctx, AV_LOG_ERROR, "Unable to open program "
                "source file \"%s\".\n", filename);
diff --git a/libavfilter/vf_curves.c b/libavfilter/vf_curves.c
index 22a1f8aa70..82e2753f01 100644
--- a/libavfilter/vf_curves.c
+++ b/libavfilter/vf_curves.c
@@ -416,7 +416,7 @@ static int dump_curves(const char *fname, uint16_t *graph[NB_COMP + 1],
     AVBPrint buf;
     const double scale = 1. / (lut_size - 1);
     static const char * const colors[] = { "red", "green", "blue", "#404040", };
-    FILE *f = av_fopen_utf8(fname, "w");
+    FILE *f = avpriv_fopen_utf8(fname, "w");
 
     av_assert0(FF_ARRAY_ELEMS(colors) == NB_COMP + 1);
 
diff --git a/libavfilter/vf_dnn_classify.c b/libavfilter/vf_dnn_classify.c
index c612ba8e80..852f5ddcee 100644
--- a/libavfilter/vf_dnn_classify.c
+++ b/libavfilter/vf_dnn_classify.c
@@ -131,7 +131,7 @@ static int read_classify_label_file(AVFilterContext *context)
     FILE *file;
     DnnClassifyContext *ctx = context->priv;
 
-    file = av_fopen_utf8(ctx->labels_filename, "r");
+    file = avpriv_fopen_utf8(ctx->labels_filename, "r");
     if (!file){
         av_log(context, AV_LOG_ERROR, "failed to open file %s\n", ctx->labels_filename);
         return AVERROR(EINVAL);
diff --git a/libavfilter/vf_dnn_detect.c b/libavfilter/vf_dnn_detect.c
index dd4507250f..68bd2cd0c3 100644
--- a/libavfilter/vf_dnn_detect.c
+++ b/libavfilter/vf_dnn_detect.c
@@ -244,7 +244,7 @@ static int read_detect_label_file(AVFilterContext *context)
     FILE *file;
     DnnDetectContext *ctx = context->priv;
 
-    file = av_fopen_utf8(ctx->labels_filename, "r");
+    file = avpriv_fopen_utf8(ctx->labels_filename, "r");
     if (!file){
         av_log(context, AV_LOG_ERROR, "failed to open file %s\n", ctx->labels_filename);
         return AVERROR(EINVAL);
diff --git a/libavfilter/vf_fieldhint.c b/libavfilter/vf_fieldhint.c
index e7afac1116..e6061c6d3c 100644
--- a/libavfilter/vf_fieldhint.c
+++ b/libavfilter/vf_fieldhint.c
@@ -73,7 +73,7 @@ static av_cold int init(AVFilterContext *ctx)
         av_log(ctx, AV_LOG_ERROR, "Hint file must be set.\n");
         return AVERROR(EINVAL);
     }
-    s->hint = av_fopen_utf8(s->hint_file_str, "r");
+    s->hint = avpriv_fopen_utf8(s->hint_file_str, "r");
     if (!s->hint) {
         ret = AVERROR(errno);
         av_log(ctx, AV_LOG_ERROR, "%s: %s\n", s->hint_file_str, av_err2str(ret));
diff --git a/libavfilter/vf_lut3d.c b/libavfilter/vf_lut3d.c
index a5190b6688..c7ecb7018e 100644
--- a/libavfilter/vf_lut3d.c
+++ b/libavfilter/vf_lut3d.c
@@ -1243,7 +1243,7 @@ static av_cold int lut3d_init(AVFilterContext *ctx)
         return set_identity_matrix(ctx, 32);
     }
 
-    f = av_fopen_utf8(lut3d->file, "r");
+    f = avpriv_fopen_utf8(lut3d->file, "r");
     if (!f) {
         ret = AVERROR(errno);
         av_log(ctx, AV_LOG_ERROR, "%s: %s\n", lut3d->file, av_err2str(ret));
@@ -2134,7 +2134,7 @@ static av_cold int lut1d_init(AVFilterContext *ctx)
         return 0;
     }
 
-    f = av_fopen_utf8(lut1d->file, "r");
+    f = avpriv_fopen_utf8(lut1d->file, "r");
     if (!f) {
         ret = AVERROR(errno);
         av_log(ctx, AV_LOG_ERROR, "%s: %s\n", lut1d->file, av_err2str(ret));
diff --git a/libavfilter/vf_nnedi.c b/libavfilter/vf_nnedi.c
index 370f643678..e5a16918bd 100644
--- a/libavfilter/vf_nnedi.c
+++ b/libavfilter/vf_nnedi.c
@@ -957,7 +957,7 @@ static av_cold int init(AVFilterContext *ctx)
     size_t bytes_read;
     int ret = 0;
 
-    weights_file = av_fopen_utf8(s->weights_file, "rb");
+    weights_file = avpriv_fopen_utf8(s->weights_file, "rb");
     if (!weights_file) {
         av_log(ctx, AV_LOG_ERROR, "No weights file provided, aborting!\n");
         return AVERROR(EINVAL);
diff --git a/libavfilter/vf_paletteuse.c b/libavfilter/vf_paletteuse.c
index e57882a64c..5c491a290c 100644
--- a/libavfilter/vf_paletteuse.c
+++ b/libavfilter/vf_paletteuse.c
@@ -538,7 +538,7 @@ static void disp_node(AVBPrint *buf,
 static int disp_tree(const struct color_node *node, const char *fname)
 {
     AVBPrint buf;
-    FILE *f = av_fopen_utf8(fname, "w");
+    FILE *f = avpriv_fopen_utf8(fname, "w");
 
     if (!f) {
         int ret = AVERROR(errno);
diff --git a/libavformat/ipfsgateway.c b/libavformat/ipfsgateway.c
index 9b0d3dea59..83d52293b4 100644
--- a/libavformat/ipfsgateway.c
+++ b/libavformat/ipfsgateway.c
@@ -139,7 +139,7 @@ static int populate_ipfs_gateway(URLContext *h)
     }
 
     // Get the contents of the gateway file.
-    gateway_file = av_fopen_utf8(ipfs_gateway_file, "r");
+    gateway_file = avpriv_fopen_utf8(ipfs_gateway_file, "r");
     if (!gateway_file) {
         av_log(h, AV_LOG_WARNING,
                "The IPFS gateway file (full uri: %s) doesn't exist. "
-- 
2.32.0 (Apple Git-132)

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
  2022-05-20 21:12 [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8 Martin Storsjö
  2022-05-20 21:12 ` [FFmpeg-devel] [PATCH 2/3] libavutil: Deprecate av_fopen_utf8, provide an avpriv version Martin Storsjö
  2022-05-20 21:12 ` [FFmpeg-devel] [PATCH 3/3] Switch uses of av_fopen_utf8 to avpriv_fopen_utf8 Martin Storsjö
@ 2022-05-21  5:07 ` Soft Works
  2022-05-23 10:53   ` Martin Storsjö
  2 siblings, 1 reply; 15+ messages in thread
From: Soft Works @ 2022-05-21  5:07 UTC (permalink / raw)
  To: FFmpeg development discussions and patches



> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Martin
> Storsjö
> Sent: Friday, May 20, 2022 11:13 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
> 
> Provide a header based inline reimplementation of it.
> 
> Using av_fopen_utf8 doesn't work outside of the libraries when built
> with MSVC as shared libraries (in the default configuration, where
> each DLL gets a separate statically linked CRT).
> ---
>  fftools/ffmpeg_opt.c |  3 +-
>  fftools/fopen_utf8.h | 71 ++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 73 insertions(+), 1 deletion(-)
>  create mode 100644 fftools/fopen_utf8.h
> 
> diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c
> index 47e8b9b7bd..a5cd989d35 100644
> --- a/fftools/ffmpeg_opt.c
> +++ b/fftools/ffmpeg_opt.c
> @@ -28,6 +28,7 @@
>  #endif
> 
>  #include "ffmpeg.h"
> +#include "fopen_utf8.h"
>  #include "cmdutils.h"
>  #include "opt_common.h"
> 
> @@ -1882,7 +1883,7 @@ static OutputStream *new_video_stream(OptionsContext
> *o, AVFormatContext *oc, in
>                      video_enc->stats_in = logbuffer;
>                  }
>                  if (video_enc->flags & AV_CODEC_FLAG_PASS1) {
> -                    f = av_fopen_utf8(logfilename, "wb");
> +                    f = fopen_utf8(logfilename, "wb");
>                      if (!f) {
>                          av_log(NULL, AV_LOG_FATAL,
>                                 "Cannot write log file '%s' for pass-1
> encoding: %s\n",
> diff --git a/fftools/fopen_utf8.h b/fftools/fopen_utf8.h
> new file mode 100644
> index 0000000000..db57fcaec4
> --- /dev/null
> +++ b/fftools/fopen_utf8.h
> @@ -0,0 +1,71 @@
> +/*
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-
> 1301 USA
> + */
> +
> +#ifndef FFTOOLS_FOPEN_UTF8_H
> +#define FFTOOLS_FOPEN_UTF8_H
> +
> +#include <stdio.h>
> +
> +/* The fopen_utf8 function here is essentially equivalent to
> av_fopen_utf8,
> + * except that it doesn't set O_CLOEXEC, and that it isn't exported
> + * from a different library. (On Windows, each DLL might use a different
> + * CRT, and FILE* handles can't be shared across them.) */
> +
> +#ifdef _WIN32
> +#include "libavutil/wchar_filename.h"
> +
> +static inline FILE *fopen_utf8(const char *path_utf8, const char *mode)
> +{
> +    wchar_t *path_w, *mode_w;
> +    FILE *f;
> +
> +    /* convert UTF-8 to wide chars */
> +    if (utf8towchar(path_utf8, &path_w)) /* This sets errno on error. */
> +        return NULL;
> +    if (!path_w)
> +        goto fallback;
> +
> +    if (utf8towchar(mode, &mode_w))
> +        return NULL;
> +    if (!mode_w) {
> +        /* If failing to interpret the mode string as utf8, it is an
> invalid
> +         * parameter. */
> +        av_freep(&path_w);
> +        errno = EINVAL;
> +        return NULL;
> +    }
> +
> +    f = _wfopen(path_w, mode_w);
> +    av_freep(&path_w);
> +    av_freep(&mode_w);
> +
> +    return f;
> +fallback:
> +    /* path may be in CP_ACP */
> +    return fopen(path_utf8, mode);
> +}
> +
> +#else
> +
> +static inline FILE *fopen_utf8(const char *path, const char *mode)
> +{
> +    return fopen(path, mode);
> +}
> +#endif
> +
> +#endif /* FFTOOLS_FOPEN_UTF8_H */
> --

LGTM. (all three)

Tested with VS project build (full static linkage, though).

Thank you,
sw
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
  2022-05-21  5:07 ` [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8 Soft Works
@ 2022-05-23 10:53   ` Martin Storsjö
  2022-05-23 10:55     ` Soft Works
  0 siblings, 1 reply; 15+ messages in thread
From: Martin Storsjö @ 2022-05-23 10:53 UTC (permalink / raw)
  To: FFmpeg development discussions and patches

On Sat, 21 May 2022, Soft Works wrote:

>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Martin
>> Storsjö
>> Sent: Friday, May 20, 2022 11:13 PM
>> To: ffmpeg-devel@ffmpeg.org
>> Subject: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
>>
>> Provide a header based inline reimplementation of it.
>>
>> Using av_fopen_utf8 doesn't work outside of the libraries when built
>> with MSVC as shared libraries (in the default configuration, where
>> each DLL gets a separate statically linked CRT).
>> ---
>>  fftools/ffmpeg_opt.c |  3 +-
>>  fftools/fopen_utf8.h | 71 ++++++++++++++++++++++++++++++++++++++++++++
>>  2 files changed, 73 insertions(+), 1 deletion(-)
>>  create mode 100644 fftools/fopen_utf8.h
>>
>> diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c
>> index 47e8b9b7bd..a5cd989d35 100644
>> --- a/fftools/ffmpeg_opt.c
>> +++ b/fftools/ffmpeg_opt.c
>> @@ -28,6 +28,7 @@
>>  #endif
>>
>>  #include "ffmpeg.h"
>> +#include "fopen_utf8.h"
>>  #include "cmdutils.h"
>>  #include "opt_common.h"
>>
>> @@ -1882,7 +1883,7 @@ static OutputStream *new_video_stream(OptionsContext
>> *o, AVFormatContext *oc, in
>>                      video_enc->stats_in = logbuffer;
>>                  }
>>                  if (video_enc->flags & AV_CODEC_FLAG_PASS1) {
>> -                    f = av_fopen_utf8(logfilename, "wb");
>> +                    f = fopen_utf8(logfilename, "wb");
>>                      if (!f) {
>>                          av_log(NULL, AV_LOG_FATAL,
>>                                 "Cannot write log file '%s' for pass-1
>> encoding: %s\n",
>> diff --git a/fftools/fopen_utf8.h b/fftools/fopen_utf8.h
>> new file mode 100644
>> index 0000000000..db57fcaec4
>> --- /dev/null
>> +++ b/fftools/fopen_utf8.h
>> @@ -0,0 +1,71 @@
>> +/*
>> + * This file is part of FFmpeg.
>> + *
>> + * FFmpeg is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2.1 of the License, or (at your option) any later version.
>> + *
>> + * FFmpeg is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> + * Lesser General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public
>> + * License along with FFmpeg; if not, write to the Free Software
>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-
>> 1301 USA
>> + */
>> +
>> +#ifndef FFTOOLS_FOPEN_UTF8_H
>> +#define FFTOOLS_FOPEN_UTF8_H
>> +
>> +#include <stdio.h>
>> +
>> +/* The fopen_utf8 function here is essentially equivalent to
>> av_fopen_utf8,
>> + * except that it doesn't set O_CLOEXEC, and that it isn't exported
>> + * from a different library. (On Windows, each DLL might use a different
>> + * CRT, and FILE* handles can't be shared across them.) */
>> +
>> +#ifdef _WIN32
>> +#include "libavutil/wchar_filename.h"
>> +
>> +static inline FILE *fopen_utf8(const char *path_utf8, const char *mode)
>> +{
>> +    wchar_t *path_w, *mode_w;
>> +    FILE *f;
>> +
>> +    /* convert UTF-8 to wide chars */
>> +    if (utf8towchar(path_utf8, &path_w)) /* This sets errno on error. */
>> +        return NULL;
>> +    if (!path_w)
>> +        goto fallback;
>> +
>> +    if (utf8towchar(mode, &mode_w))
>> +        return NULL;
>> +    if (!mode_w) {
>> +        /* If failing to interpret the mode string as utf8, it is an
>> invalid
>> +         * parameter. */
>> +        av_freep(&path_w);
>> +        errno = EINVAL;
>> +        return NULL;
>> +    }
>> +
>> +    f = _wfopen(path_w, mode_w);
>> +    av_freep(&path_w);
>> +    av_freep(&mode_w);
>> +
>> +    return f;
>> +fallback:
>> +    /* path may be in CP_ACP */
>> +    return fopen(path_utf8, mode);
>> +}
>> +
>> +#else
>> +
>> +static inline FILE *fopen_utf8(const char *path, const char *mode)
>> +{
>> +    return fopen(path, mode);
>> +}
>> +#endif
>> +
>> +#endif /* FFTOOLS_FOPEN_UTF8_H */
>> --
>
> LGTM. (all three)
>
> Tested with VS project build (full static linkage, though).

I discussed this with Anton on irc, and he was ok with the patchset too, 
so I pushed it now.

// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
  2022-05-23 10:53   ` Martin Storsjö
@ 2022-05-23 10:55     ` Soft Works
  2022-05-23 10:58       ` Martin Storsjö
  0 siblings, 1 reply; 15+ messages in thread
From: Soft Works @ 2022-05-23 10:55 UTC (permalink / raw)
  To: FFmpeg development discussions and patches



> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Martin
> Storsjö
> Sent: Monday, May 23, 2022 12:53 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
> 
> On Sat, 21 May 2022, Soft Works wrote:
> 
> >> -----Original Message-----
> >> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Martin
> >> Storsjö
> >> Sent: Friday, May 20, 2022 11:13 PM
> >> To: ffmpeg-devel@ffmpeg.org
> >> Subject: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
> >>
> >> Provide a header based inline reimplementation of it.
> >>
> >> Using av_fopen_utf8 doesn't work outside of the libraries when built
> >> with MSVC as shared libraries (in the default configuration, where
> >> each DLL gets a separate statically linked CRT).
> >> ---
> >>  fftools/ffmpeg_opt.c |  3 +-
> >>  fftools/fopen_utf8.h | 71 ++++++++++++++++++++++++++++++++++++++++++++
> >>  2 files changed, 73 insertions(+), 1 deletion(-)
> >>  create mode 100644 fftools/fopen_utf8.h
> >>
> >> diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c
> >> index 47e8b9b7bd..a5cd989d35 100644
> >> --- a/fftools/ffmpeg_opt.c
> >> +++ b/fftools/ffmpeg_opt.c
> >> @@ -28,6 +28,7 @@
> >>  #endif
> >>
> >>  #include "ffmpeg.h"
> >> +#include "fopen_utf8.h"
> >>  #include "cmdutils.h"
> >>  #include "opt_common.h"
> >>
> >> @@ -1882,7 +1883,7 @@ static OutputStream
> *new_video_stream(OptionsContext
> >> *o, AVFormatContext *oc, in
> >>                      video_enc->stats_in = logbuffer;
> >>                  }
> >>                  if (video_enc->flags & AV_CODEC_FLAG_PASS1) {
> >> -                    f = av_fopen_utf8(logfilename, "wb");
> >> +                    f = fopen_utf8(logfilename, "wb");
> >>                      if (!f) {
> >>                          av_log(NULL, AV_LOG_FATAL,
> >>                                 "Cannot write log file '%s' for pass-1
> >> encoding: %s\n",
> >> diff --git a/fftools/fopen_utf8.h b/fftools/fopen_utf8.h
> >> new file mode 100644
> >> index 0000000000..db57fcaec4
> >> --- /dev/null
> >> +++ b/fftools/fopen_utf8.h
> >> @@ -0,0 +1,71 @@
> >> +/*
> >> + * This file is part of FFmpeg.
> >> + *
> >> + * FFmpeg is free software; you can redistribute it and/or
> >> + * modify it under the terms of the GNU Lesser General Public
> >> + * License as published by the Free Software Foundation; either
> >> + * version 2.1 of the License, or (at your option) any later version.
> >> + *
> >> + * FFmpeg is distributed in the hope that it will be useful,
> >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> >> + * Lesser General Public License for more details.
> >> + *
> >> + * You should have received a copy of the GNU Lesser General Public
> >> + * License along with FFmpeg; if not, write to the Free Software
> >> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-
> >> 1301 USA
> >> + */
> >> +
> >> +#ifndef FFTOOLS_FOPEN_UTF8_H
> >> +#define FFTOOLS_FOPEN_UTF8_H
> >> +
> >> +#include <stdio.h>
> >> +
> >> +/* The fopen_utf8 function here is essentially equivalent to
> >> av_fopen_utf8,
> >> + * except that it doesn't set O_CLOEXEC, and that it isn't exported
> >> + * from a different library. (On Windows, each DLL might use a
> different
> >> + * CRT, and FILE* handles can't be shared across them.) */
> >> +
> >> +#ifdef _WIN32
> >> +#include "libavutil/wchar_filename.h"
> >> +
> >> +static inline FILE *fopen_utf8(const char *path_utf8, const char
> *mode)
> >> +{
> >> +    wchar_t *path_w, *mode_w;
> >> +    FILE *f;
> >> +
> >> +    /* convert UTF-8 to wide chars */
> >> +    if (utf8towchar(path_utf8, &path_w)) /* This sets errno on error.
> */
> >> +        return NULL;
> >> +    if (!path_w)
> >> +        goto fallback;
> >> +
> >> +    if (utf8towchar(mode, &mode_w))
> >> +        return NULL;
> >> +    if (!mode_w) {
> >> +        /* If failing to interpret the mode string as utf8, it is an
> >> invalid
> >> +         * parameter. */
> >> +        av_freep(&path_w);
> >> +        errno = EINVAL;
> >> +        return NULL;
> >> +    }
> >> +
> >> +    f = _wfopen(path_w, mode_w);
> >> +    av_freep(&path_w);
> >> +    av_freep(&mode_w);
> >> +
> >> +    return f;
> >> +fallback:
> >> +    /* path may be in CP_ACP */
> >> +    return fopen(path_utf8, mode);
> >> +}
> >> +
> >> +#else
> >> +
> >> +static inline FILE *fopen_utf8(const char *path, const char *mode)
> >> +{
> >> +    return fopen(path, mode);
> >> +}
> >> +#endif
> >> +
> >> +#endif /* FFTOOLS_FOPEN_UTF8_H */
> >> --
> >
> > LGTM. (all three)
> >
> > Tested with VS project build (full static linkage, though).
> 
> I discussed this with Anton on irc, and he was ok with the patchset too,
> so I pushed it now.
> 
> // Martin

Great, thanks.

Shall I update mine now to cover the remaining fopen() calls?

sw
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
  2022-05-23 10:55     ` Soft Works
@ 2022-05-23 10:58       ` Martin Storsjö
  2022-05-23 11:06         ` Soft Works
  0 siblings, 1 reply; 15+ messages in thread
From: Martin Storsjö @ 2022-05-23 10:58 UTC (permalink / raw)
  To: FFmpeg development discussions and patches

On Mon, 23 May 2022, Soft Works wrote:

>
>
>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Martin
>> Storsjö
>> Sent: Monday, May 23, 2022 12:53 PM
>> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
>> Subject: Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
>>
>> On Sat, 21 May 2022, Soft Works wrote:
>>
>>> LGTM. (all three)
>>>
>>> Tested with VS project build (full static linkage, though).
>>
>> I discussed this with Anton on irc, and he was ok with the patchset too,
>> so I pushed it now.
>>
>> // Martin
>
> Great, thanks.
>
> Shall I update mine now to cover the remaining fopen() calls?

Yep, that sounds good to me. It should be easier to move forward with the 
uncontroversial parts of the patchsets now.

// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
  2022-05-23 10:58       ` Martin Storsjö
@ 2022-05-23 11:06         ` Soft Works
  2022-05-23 11:11           ` Martin Storsjö
  0 siblings, 1 reply; 15+ messages in thread
From: Soft Works @ 2022-05-23 11:06 UTC (permalink / raw)
  To: FFmpeg development discussions and patches



> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Martin
> Storsjö
> Sent: Monday, May 23, 2022 12:58 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
> 
> On Mon, 23 May 2022, Soft Works wrote:
> 
> >
> >
> >> -----Original Message-----
> >> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Martin
> >> Storsjö
> >> Sent: Monday, May 23, 2022 12:53 PM
> >> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> >> Subject: Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using
> av_fopen_utf8
> >>
> >> On Sat, 21 May 2022, Soft Works wrote:
> >>
> >>> LGTM. (all three)
> >>>
> >>> Tested with VS project build (full static linkage, though).
> >>
> >> I discussed this with Anton on irc, and he was ok with the patchset
> too,
> >> so I pushed it now.
> >>
> >> // Martin
> >
> > Great, thanks.
> >
> > Shall I update mine now to cover the remaining fopen() calls?
> 
> Yep, that sounds good to me. It should be easier to move forward with the
> uncontroversial parts of the patchsets now.
> 
> // Martin

Sure, will do. 

I think I have addressed all concerns from the side of nil-admirari;
the last point was the mapping of the stat() function, which -
even though it works - could use a few more eyes taking a look at.

Any other specific concerns from your side?

Thanks,
sw







_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
  2022-05-23 11:06         ` Soft Works
@ 2022-05-23 11:11           ` Martin Storsjö
  2022-05-23 11:44             ` Soft Works
  0 siblings, 1 reply; 15+ messages in thread
From: Martin Storsjö @ 2022-05-23 11:11 UTC (permalink / raw)
  To: FFmpeg development discussions and patches

On Mon, 23 May 2022, Soft Works wrote:

>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Martin
>> Storsjö
>> Sent: Monday, May 23, 2022 12:58 PM
>> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
>> Subject: Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
>>
>> On Mon, 23 May 2022, Soft Works wrote:
>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
>> Martin
>>>> Storsjö
>>>> Sent: Monday, May 23, 2022 12:53 PM
>>>> To: FFmpeg development discussions and patches <ffmpeg-
>> devel@ffmpeg.org>
>>>> Subject: Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using
>> av_fopen_utf8
>>>>
>>>> On Sat, 21 May 2022, Soft Works wrote:
>>>>
>>>>> LGTM. (all three)
>>>>>
>>>>> Tested with VS project build (full static linkage, though).
>>>>
>>>> I discussed this with Anton on irc, and he was ok with the patchset
>> too,
>>>> so I pushed it now.
>>>>
>>>> // Martin
>>>
>>> Great, thanks.
>>>
>>> Shall I update mine now to cover the remaining fopen() calls?
>>
>> Yep, that sounds good to me. It should be easier to move forward with the
>> uncontroversial parts of the patchsets now.
>>
>> // Martin
>
> Sure, will do.
>
> I think I have addressed all concerns from the side of nil-admirari;
> the last point was the mapping of the stat() function, which -
> even though it works - could use a few more eyes taking a look at.
>
> Any other specific concerns from your side?

I haven't followed your discussion closely (I was waiting for it to 
converge, which it apparently mostly has) so I haven't got anything to add 
offhand right now. I can try to read through the latest iteration (or the 
next one if rebased and reposted) and give a more qualified opinion in a 
day or two.

// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
  2022-05-23 11:11           ` Martin Storsjö
@ 2022-05-23 11:44             ` Soft Works
  2022-05-24  9:29               ` Martin Storsjö
  0 siblings, 1 reply; 15+ messages in thread
From: Soft Works @ 2022-05-23 11:44 UTC (permalink / raw)
  To: FFmpeg development discussions and patches



> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Martin
> Storsjö
> Sent: Monday, May 23, 2022 1:12 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
> 
> On Mon, 23 May 2022, Soft Works wrote:
> 
> >> -----Original Message-----
> >> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Martin
> >> Storsjö
> >> Sent: Monday, May 23, 2022 12:58 PM
> >> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> >> Subject: Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using
> av_fopen_utf8
> >>
> >> On Mon, 23 May 2022, Soft Works wrote:
> >>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> >> Martin
> >>>> Storsjö
> >>>> Sent: Monday, May 23, 2022 12:53 PM
> >>>> To: FFmpeg development discussions and patches <ffmpeg-
> >> devel@ffmpeg.org>
> >>>> Subject: Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using
> >> av_fopen_utf8
> >>>>
> >>>> On Sat, 21 May 2022, Soft Works wrote:
> >>>>
> >>>>> LGTM. (all three)
> >>>>>
> >>>>> Tested with VS project build (full static linkage, though).
> >>>>
> >>>> I discussed this with Anton on irc, and he was ok with the patchset
> >> too,
> >>>> so I pushed it now.
> >>>>
> >>>> // Martin
> >>>
> >>> Great, thanks.
> >>>
> >>> Shall I update mine now to cover the remaining fopen() calls?
> >>
> >> Yep, that sounds good to me. It should be easier to move forward with
> the
> >> uncontroversial parts of the patchsets now.
> >>
> >> // Martin
> >
> > Sure, will do.
> >
> > I think I have addressed all concerns from the side of nil-admirari;
> > the last point was the mapping of the stat() function, which -
> > even though it works - could use a few more eyes taking a look at.
> >
> > Any other specific concerns from your side?
> 
> I haven't followed your discussion closely (I was waiting for it to
> converge, which it apparently mostly has) so I haven't got anything to add
> offhand right now. I can try to read through the latest iteration (or the
> next one if rebased and reposted) and give a more qualified opinion in a
> day or two.

Great. I rebased and resubmitted both patchsets. The primary long-path 
patchset didn't need any change.

Considerations for the latter were:

- Should the file wchar_filename.h be renamed as it is now containing 
  the path prefixing code?

- I have kept the path functions in the same way like .NET does it,
  just for easy reference and following. Compilers will inline 
  them anyway (my pov). Maybe others don't like that. I can change
  if it's got to be.

- The one ugliness is the copied struct for the mapping of the stat
  function. Maybe there's some tricky way to get around doing so,
  I just couldn't find any.

I'm very confident about the core functionality, though (due to it
origin).

Thanks again,
sw




_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
  2022-05-23 11:44             ` Soft Works
@ 2022-05-24  9:29               ` Martin Storsjö
  2022-05-24 19:45                 ` Soft Works
  0 siblings, 1 reply; 15+ messages in thread
From: Martin Storsjö @ 2022-05-24  9:29 UTC (permalink / raw)
  To: FFmpeg development discussions and patches

On Mon, 23 May 2022, Soft Works wrote:

> Great. I rebased and resubmitted both patchsets. The primary long-path
> patchset didn't need any change.
>
> Considerations for the latter were:
>
> - Should the file wchar_filename.h be renamed as it is now containing
>  the path prefixing code?

I guess we could do that later at some point, but I don't see it as 
urgent.

> - I have kept the path functions in the same way like .NET does it,
>  just for easy reference and following. Compilers will inline
>  them anyway (my pov). Maybe others don't like that. I can change
>  if it's got to be.

Having the individual functions inline compared to merging it all in one 
big function doesn't matter to me. But the amount of code in this inline 
header is growing a bit big, to the point that I think it is measurable 
when multiple files within the same library use these functions. Longer 
term, it would probably make sense to make them more shared in some way.

If C would have the C++ style deduplication feature for non-static inline 
functions, this wouldn't be an issue. One could consider making them ff_ 
functions and carry a copy in each library too, maybe. (But that also 
makes it trickier to use in fftools.) Not entirely urgent yet anyway.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
  2022-05-24  9:29               ` Martin Storsjö
@ 2022-05-24 19:45                 ` Soft Works
  2022-05-24 20:21                   ` Martin Storsjö
  0 siblings, 1 reply; 15+ messages in thread
From: Soft Works @ 2022-05-24 19:45 UTC (permalink / raw)
  To: FFmpeg development discussions and patches



> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Martin
> Storsjö
> Sent: Tuesday, May 24, 2022 11:29 AM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
> 
> On Mon, 23 May 2022, Soft Works wrote:
> 
> > Great. I rebased and resubmitted both patchsets. The primary long-path
> > patchset didn't need any change.
> >
> > Considerations for the latter were:
> >
> > - Should the file wchar_filename.h be renamed as it is now containing
> >  the path prefixing code?
> 
> I guess we could do that later at some point, but I don't see it as
> urgent.
> 
> > - I have kept the path functions in the same way like .NET does it,
> >  just for easy reference and following. Compilers will inline
> >  them anyway (my pov). Maybe others don't like that. I can change
> >  if it's got to be.
> 
> Having the individual functions inline compared to merging it all in one
> big function doesn't matter to me. But the amount of code in this inline
> header is growing a bit big, to the point that I think it is measurable
> when multiple files within the same library use these functions. Longer
> term, it would probably make sense to make them more shared in some way.

> If C would have the C++ style deduplication feature for non-static inline
> functions, this wouldn't be an issue. One could consider making them ff_
> functions and carry a copy in each library too, maybe. (But that also
> makes it trickier to use in fftools.) 

A copy in each library - isn't that exactly what's already happening?

The get_extended_win32_path() is used from 2 places:

1. file_open.c

For this, we already have copies in each library. file_open.c includes
wchar_filename.h and the new functions are inlined into file_open.obj.
So, there's only one copy in each library

2. os_support.h

This is used in libavformat exclusively. But in this case, the code gets
duplicated actually for each consumption of one of those file functions.
There aren't many usages in total, though, and I don't see any trend 
on the horizon for sudden increase, so I agree that there's no urgency.


BTW, thanks for your other comments, I have added the missing av_free()
calls and included URLs to the .NET source.
I chose to use permalinks. These are long and ugly, but they have changed
their structure (and even repo) so often during the past years, that
chances are low that the non-versioned links would still be valid in a
few years (I wouldn't even bet on months :-)

softworkz








_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
  2022-05-24 19:45                 ` Soft Works
@ 2022-05-24 20:21                   ` Martin Storsjö
  2022-05-24 20:50                     ` Soft Works
  0 siblings, 1 reply; 15+ messages in thread
From: Martin Storsjö @ 2022-05-24 20:21 UTC (permalink / raw)
  To: FFmpeg development discussions and patches

On Tue, 24 May 2022, Soft Works wrote:

>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Martin
>> Storsjö
>> Sent: Tuesday, May 24, 2022 11:29 AM
>> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
>> Subject: Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
>>
>> On Mon, 23 May 2022, Soft Works wrote:
>>
>>> Great. I rebased and resubmitted both patchsets. The primary long-path
>>> patchset didn't need any change.
>>>
>>> Considerations for the latter were:
>>>
>>> - Should the file wchar_filename.h be renamed as it is now containing
>>>  the path prefixing code?
>>
>> I guess we could do that later at some point, but I don't see it as
>> urgent.
>>
>>> - I have kept the path functions in the same way like .NET does it,
>>>  just for easy reference and following. Compilers will inline
>>>  them anyway (my pov). Maybe others don't like that. I can change
>>>  if it's got to be.
>>
>> Having the individual functions inline compared to merging it all in one
>> big function doesn't matter to me. But the amount of code in this inline
>> header is growing a bit big, to the point that I think it is measurable
>> when multiple files within the same library use these functions. Longer
>> term, it would probably make sense to make them more shared in some way.
>
>> If C would have the C++ style deduplication feature for non-static inline
>> functions, this wouldn't be an issue. One could consider making them ff_
>> functions and carry a copy in each library too, maybe. (But that also
>> makes it trickier to use in fftools.)
>
> A copy in each library - isn't that exactly what's already happening?
>
> The get_extended_win32_path() is used from 2 places:
>
> 2. os_support.h
>
> This is used in libavformat exclusively. But in this case, the code gets
> duplicated actually for each consumption of one of those file functions.
> There aren't many usages in total, though, and I don't see any trend
> on the horizon for sudden increase, so I agree that there's no urgency.

Yes, this is what I was referring to. It's clearly more than one use. When 
counting files that use mkdir, unlink or "struct stat", I find around 9 
individual .c files in libavformat, that would end up with static inline 
copies of all of this.

Compared with the av_fopen_utf8 case, I first tested fixing it by making 
it a static inline function in a libavutil header, but that did increase 
the size of a built DLL by a couple KB. I guess this increases it by a 
bit more than that.

It's still not a lot, and this isn't a blocker (and I probably prefer that 
we don't try to redesign this issue here now, in order not to drag out the 
review further), but compared to how C++ inline methods are deduplicated 
by the linker, it's a slightly annoying inefficiency.

// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
  2022-05-24 20:21                   ` Martin Storsjö
@ 2022-05-24 20:50                     ` Soft Works
  2022-05-24 20:55                       ` Martin Storsjö
  0 siblings, 1 reply; 15+ messages in thread
From: Soft Works @ 2022-05-24 20:50 UTC (permalink / raw)
  To: FFmpeg development discussions and patches



> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Martin
> Storsjö
> Sent: Tuesday, May 24, 2022 10:22 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
> 
> On Tue, 24 May 2022, Soft Works wrote:
> 
> >> -----Original Message-----
> >> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Martin
> >> Storsjö
> >> Sent: Tuesday, May 24, 2022 11:29 AM
> >> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> >> Subject: Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using
> av_fopen_utf8
> >>
> >> On Mon, 23 May 2022, Soft Works wrote:
> >>
> >>> Great. I rebased and resubmitted both patchsets. The primary long-path
> >>> patchset didn't need any change.
> >>>
> >>> Considerations for the latter were:
> >>>
> >>> - Should the file wchar_filename.h be renamed as it is now containing
> >>>  the path prefixing code?
> >>
> >> I guess we could do that later at some point, but I don't see it as
> >> urgent.
> >>
> >>> - I have kept the path functions in the same way like .NET does it,
> >>>  just for easy reference and following. Compilers will inline
> >>>  them anyway (my pov). Maybe others don't like that. I can change
> >>>  if it's got to be.
> >>
> >> Having the individual functions inline compared to merging it all in
> one
> >> big function doesn't matter to me. But the amount of code in this
> inline
> >> header is growing a bit big, to the point that I think it is measurable
> >> when multiple files within the same library use these functions. Longer
> >> term, it would probably make sense to make them more shared in some
> way.
> >
> >> If C would have the C++ style deduplication feature for non-static
> inline
> >> functions, this wouldn't be an issue. One could consider making them
> ff_
> >> functions and carry a copy in each library too, maybe. (But that also
> >> makes it trickier to use in fftools.)
> >
> > A copy in each library - isn't that exactly what's already happening?
> >
> > The get_extended_win32_path() is used from 2 places:
> >
> > 2. os_support.h
> >
> > This is used in libavformat exclusively. But in this case, the code gets
> > duplicated actually for each consumption of one of those file functions.
> > There aren't many usages in total, though, and I don't see any trend
> > on the horizon for sudden increase, so I agree that there's no urgency.
> 
> Yes, this is what I was referring to. It's clearly more than one use. When
> counting files that use mkdir, unlink or "struct stat", I find around 9
> individual .c files in libavformat, that would end up with static inline
> copies of all of this.
> 
> Compared with the av_fopen_utf8 case, I first tested fixing it by making
> it a static inline function in a libavutil header, but that did increase
> the size of a built DLL by a couple KB. I guess this increases it by a
> bit more than that.
> 
> It's still not a lot, and this isn't a blocker (and I probably prefer that
> we don't try to redesign this issue here now, in order not to drag out the
> review further)

I'm glad you're saying that ;-)

It probably won't be difficult to improve this in a future change, making it
a wchar_filename.c plus wchar_filename.h file in libavutil. If I'm not 
mistaken, there shouldn't be an issue with multiple CRTs as memory is 
managed by av_ functions?

Thanks,
sw


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
  2022-05-24 20:50                     ` Soft Works
@ 2022-05-24 20:55                       ` Martin Storsjö
  0 siblings, 0 replies; 15+ messages in thread
From: Martin Storsjö @ 2022-05-24 20:55 UTC (permalink / raw)
  To: FFmpeg development discussions and patches

On Tue, 24 May 2022, Soft Works wrote:

>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Martin
>> Storsjö
>> Sent: Tuesday, May 24, 2022 10:22 PM
>> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
>> Subject: Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8
>>
>> On Tue, 24 May 2022, Soft Works wrote:
>>
>>>> -----Original Message-----
>>>> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
>> Martin
>>>> Storsjö
>>>> Sent: Tuesday, May 24, 2022 11:29 AM
>>>> To: FFmpeg development discussions and patches <ffmpeg-
>> devel@ffmpeg.org>
>>>> Subject: Re: [FFmpeg-devel] [PATCH 1/3] fftools: Stop using
>> av_fopen_utf8
>>>>
>>>> On Mon, 23 May 2022, Soft Works wrote:
>>>>
>>>>> Great. I rebased and resubmitted both patchsets. The primary long-path
>>>>> patchset didn't need any change.
>>>>>
>>>>> Considerations for the latter were:
>>>>>
>>>>> - Should the file wchar_filename.h be renamed as it is now containing
>>>>>  the path prefixing code?
>>>>
>>>> I guess we could do that later at some point, but I don't see it as
>>>> urgent.
>>>>
>>>>> - I have kept the path functions in the same way like .NET does it,
>>>>>  just for easy reference and following. Compilers will inline
>>>>>  them anyway (my pov). Maybe others don't like that. I can change
>>>>>  if it's got to be.
>>>>
>>>> Having the individual functions inline compared to merging it all in
>> one
>>>> big function doesn't matter to me. But the amount of code in this
>> inline
>>>> header is growing a bit big, to the point that I think it is measurable
>>>> when multiple files within the same library use these functions. Longer
>>>> term, it would probably make sense to make them more shared in some
>> way.
>>>
>>>> If C would have the C++ style deduplication feature for non-static
>> inline
>>>> functions, this wouldn't be an issue. One could consider making them
>> ff_
>>>> functions and carry a copy in each library too, maybe. (But that also
>>>> makes it trickier to use in fftools.)
>>>
>>> A copy in each library - isn't that exactly what's already happening?
>>>
>>> The get_extended_win32_path() is used from 2 places:
>>>
>>> 2. os_support.h
>>>
>>> This is used in libavformat exclusively. But in this case, the code gets
>>> duplicated actually for each consumption of one of those file functions.
>>> There aren't many usages in total, though, and I don't see any trend
>>> on the horizon for sudden increase, so I agree that there's no urgency.
>>
>> Yes, this is what I was referring to. It's clearly more than one use. When
>> counting files that use mkdir, unlink or "struct stat", I find around 9
>> individual .c files in libavformat, that would end up with static inline
>> copies of all of this.
>>
>> Compared with the av_fopen_utf8 case, I first tested fixing it by making
>> it a static inline function in a libavutil header, but that did increase
>> the size of a built DLL by a couple KB. I guess this increases it by a
>> bit more than that.
>>
>> It's still not a lot, and this isn't a blocker (and I probably prefer that
>> we don't try to redesign this issue here now, in order not to drag out the
>> review further)
>
> I'm glad you're saying that ;-)
>
> It probably won't be difficult to improve this in a future change, making it
> a wchar_filename.c plus wchar_filename.h file in libavutil. If I'm not
> mistaken, there shouldn't be an issue with multiple CRTs as memory is
> managed by av_ functions?

Indeed, that's not a problem for plain filename conversions. The main 
hesitation is that we'd generally want to avoid adding more avpriv_ APIs 
unless strictly necessary.

// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2022-05-24 20:55 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-20 21:12 [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8 Martin Storsjö
2022-05-20 21:12 ` [FFmpeg-devel] [PATCH 2/3] libavutil: Deprecate av_fopen_utf8, provide an avpriv version Martin Storsjö
2022-05-20 21:12 ` [FFmpeg-devel] [PATCH 3/3] Switch uses of av_fopen_utf8 to avpriv_fopen_utf8 Martin Storsjö
2022-05-21  5:07 ` [FFmpeg-devel] [PATCH 1/3] fftools: Stop using av_fopen_utf8 Soft Works
2022-05-23 10:53   ` Martin Storsjö
2022-05-23 10:55     ` Soft Works
2022-05-23 10:58       ` Martin Storsjö
2022-05-23 11:06         ` Soft Works
2022-05-23 11:11           ` Martin Storsjö
2022-05-23 11:44             ` Soft Works
2022-05-24  9:29               ` Martin Storsjö
2022-05-24 19:45                 ` Soft Works
2022-05-24 20:21                   ` Martin Storsjö
2022-05-24 20:50                     ` Soft Works
2022-05-24 20:55                       ` Martin Storsjö

Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
		ffmpegdev@gitmailbox.com
	public-inbox-index ffmpegdev

Example config snippet for mirrors.


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git