[FFmpeg-devel] [PATCH 0/5] Add IPFS and IPNS protocol support

[FFmpeg-devel] [PATCH 0/5] Add IPFS and IPNS protocol support

[Mark Gaiser]

Hi,

This patch series adds support for IPFS.

I've been adviced to keep the patches as they are (split). If needed, I can 
squash them to a single patch.

The following is a short summary. In the IPFS ecosystem you access it's content
by a "Content IDentifier" (CID). This CID is, in simplified terms, a hash of 
the content. IPFS itself is a distributed network where any user can run a node
to be part of the network and access files by their CID. If any reachable node 
within that network has the CID, you can get it.

IPFS (as a technology) has two protocols, ipfs and ipns.
The ipfs protocol is the immutable way to access content.
The ipns protocol is a mutable layer on top of it. It's essentially a new CID 
that points to a ipfs CID. This "pointer" if you will can be changed to point 
to something else.
Much more information on how this technology works can be found here [1].

This patch series allows to interact natively with IPFS. That means being able
to access files like:
- ffplay ipfs://<cid>
- ffplay ipns://<cid>

There are multiple ways to access files on the IPFS network. This patch series
uses the gateway driven way. An IPFS node - by default - exposes a local 
gateway (say http://localhost:8080) which is then used to get content from IPFS.

Much of the logic in this patch series is to find that gateway and essentially 
rewrite:

"ipfs://<cid>"

to:

"http://localhost:8080/ipfs/<cid>"

Once that's found it's forwared to the protocol handler where eventually the
http protocol is going to handle it. Note that it could also be https. There's 
enough flexibility in the implementation to allow the user to provide a 
gateway. There are also public https gateways which can be used just as well.

After this patch is accepted, I'll work on getting IPFS supported in:
- mpv (requires this ffmpeg patch)
- vlc (prefers this patch but can be made to work without this patch)
- kodi (requires this ffmpeg patch)

Best regards,
Mark Gaiser

[1] https://docs.ipfs.io/concepts/


Mark Gaiser (5):
  Early version of IPFS protocol support.
  Fix up IPNS support.
  Merge IPNS and IPFS handling.
  Implement logic to determine the IPFS gateway.
  Fix review feedback

 configure               |   2 +
 doc/protocols.texi      |  30 +++++
 libavformat/Makefile    |   2 +
 libavformat/ipfs.c      | 283 ++++++++++++++++++++++++++++++++++++++++
 libavformat/protocols.c |   2 +
 5 files changed, 319 insertions(+)
 create mode 100644 libavformat/ipfs.c

-- 
2.35.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 1/5] Early version of IPFS protocol support.

[Mark Gaiser]

Signed-off-by: Mark Gaiser <markg85@gmail.com>
---
 configure               |   1 +
 doc/protocols.texi      |  30 ++++++
 libavformat/Makefile    |   1 +
 libavformat/ipfs.c      | 202 ++++++++++++++++++++++++++++++++++++++++
 libavformat/protocols.c |   2 +
 5 files changed, 236 insertions(+)
 create mode 100644 libavformat/ipfs.c

diff --git a/configure b/configure
index 5b19a35f59..e466f924a3 100755
--- a/configure
+++ b/configure
@@ -3585,6 +3585,7 @@ udp_protocol_select="network"
 udplite_protocol_select="network"
 unix_protocol_deps="sys_un_h"
 unix_protocol_select="network"
+ipfs_protocol_select="https_protocol"
 
 # external library protocols
 libamqp_protocol_deps="librabbitmq"
diff --git a/doc/protocols.texi b/doc/protocols.texi
index d207df0b52..7c9c0a4808 100644
--- a/doc/protocols.texi
+++ b/doc/protocols.texi
@@ -2025,5 +2025,35 @@ decoding errors.
 
 @end table
 
+@section ipfs
+
+InterPlanetary File System (IPFS) protocol support. One can access files stored 
+on the IPFS network through so called gateways. Those are http(s) endpoints.
+This protocol wraps the IPFS native protocols (ipfs:// and ipns://) to be send 
+to such a gateway. Users can (and should) host their own node which means this 
+protocol will use your local machine gateway to access files on the IPFS network.
+
+If a user doesn't have a node of their own then the public gateway dweb.link is 
+used by default.
+
+You can use this protocol in 2 ways. Using IPFS:
+@example
+ffplay ipfs://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
+@end example
+
+Or the IPNS protocol (IPNS is mutable IPFS):
+@example
+ffplay ipns://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
+@end example
+
+You can also change the gateway to be used:
+
+@table @option
+
+@item gateway
+Defines the gateway to use. When nothing is provided the protocol will first try 
+your local gateway. If that fails dweb.link will be used.
+
+@end table
 
 @c man end PROTOCOLS
diff --git a/libavformat/Makefile b/libavformat/Makefile
index 3dc6a479cc..983a77f4f2 100644
--- a/libavformat/Makefile
+++ b/libavformat/Makefile
@@ -656,6 +656,7 @@ OBJS-$(CONFIG_SRTP_PROTOCOL)             += srtpproto.o srtp.o
 OBJS-$(CONFIG_SUBFILE_PROTOCOL)          += subfile.o
 OBJS-$(CONFIG_TEE_PROTOCOL)              += teeproto.o tee_common.o
 OBJS-$(CONFIG_TCP_PROTOCOL)              += tcp.o
+OBJS-$(CONFIG_IPFS_PROTOCOL)             += ipfs.o
 TLS-OBJS-$(CONFIG_GNUTLS)                += tls_gnutls.o
 TLS-OBJS-$(CONFIG_LIBTLS)                += tls_libtls.o
 TLS-OBJS-$(CONFIG_MBEDTLS)               += tls_mbedtls.o
diff --git a/libavformat/ipfs.c b/libavformat/ipfs.c
new file mode 100644
index 0000000000..4cc65750ed
--- /dev/null
+++ b/libavformat/ipfs.c
@@ -0,0 +1,202 @@
+/*
+ * IPFS protocol.
+ * Copyright (c) 2021 Mark Gaiser
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/avassert.h"
+#include "libavutil/avstring.h"
+#include "libavutil/internal.h"
+#include "libavutil/opt.h"
+#include "libavutil/tree.h"
+#include "avformat.h"
+#include <fcntl.h>
+#if HAVE_IO_H
+#include <io.h>
+#endif
+#if HAVE_UNISTD_H
+#include <unistd.h>
+#endif
+#include <sys/stat.h>
+#include <stdlib.h>
+#include "os_support.h"
+#include "url.h"
+
+typedef struct Context {
+    AVClass *class;
+    URLContext *inner;
+    char *fulluri;
+    char *gateway;
+} Context;
+
+static int ipfs_open(URLContext *h, const char *uri, int flags, AVDictionary **options)
+{
+    const char *gatewaysuffix;
+    int ret = 0;
+    Context *c = h->priv_data;
+    
+    if (!av_strstart(uri, "ipfs://", &gatewaysuffix) &&
+        !av_strstart(uri, "ipfs:", &gatewaysuffix)) {
+        av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri);
+        ret = AVERROR(EINVAL);
+        goto err;
+    }
+    
+    char* ipfs_gateway = "https://ipfs.io/ipfs/";
+    
+    c->fulluri = malloc(strlen(ipfs_gateway)+strlen(gatewaysuffix) + 1);
+    
+    strcpy(c->fulluri, ipfs_gateway);
+    strcat(c->fulluri, gatewaysuffix);
+    
+    if ((ret = ffurl_open_whitelist(&c->inner, c->fulluri, flags,
+                                    &h->interrupt_callback, options,
+                                    h->protocol_whitelist, h->protocol_blacklist, h)) < 0) {
+        av_log(h, AV_LOG_ERROR, "Unable to open resource: %s\n", c->fulluri);
+        goto err;
+    }
+    
+err:
+    return ret;
+}
+
+static int ipfs_read(URLContext *h, unsigned char *buf, int size)
+{
+    Context *c = h->priv_data;
+    int ret;
+
+    ret = ffurl_read(c->inner, buf, size);
+
+    return ret;
+}
+
+static int64_t ipfs_seek(URLContext *h, int64_t pos, int whence)
+{
+    Context *c = h->priv_data;
+    int64_t ret;
+
+    ret = ffurl_seek(c->inner, pos, whence);
+
+    return ret;
+}
+
+static int ipfs_close(URLContext *h)
+{
+    Context *c = h->priv_data;
+    int ret;
+
+    ret = ffurl_closep(&c->inner);
+
+    return ret;
+}
+
+static int ipns_open(URLContext *h, const char *uri, int flags, AVDictionary **options)
+{
+    const char *gatewaysuffix;
+    int ret = 0;
+    Context *c = h->priv_data;
+    
+    if (!av_strstart(uri, "ipns://", &gatewaysuffix) &&
+        !av_strstart(uri, "ipns:", &gatewaysuffix)) {
+        av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri);
+        ret = AVERROR(EINVAL);
+        goto err;
+    }
+    
+    char* ipfs_gateway = "https://ipfs.io/ipns/";
+    
+    c->fulluri = malloc(strlen(ipfs_gateway)+strlen(gatewaysuffix) + 1);
+    
+    strcpy(c->fulluri, ipfs_gateway);
+    strcat(c->fulluri, gatewaysuffix);
+    
+    if ((ret = ffurl_open_whitelist(&c->inner, c->fulluri, flags,
+                                    &h->interrupt_callback, options,
+                                    h->protocol_whitelist, h->protocol_blacklist, h)) < 0) {
+        av_log(h, AV_LOG_ERROR, "Unable to open resource: %s\n", c->fulluri);
+        goto err;
+    }
+    
+err:
+    return ret;
+}
+
+static int ipns_read(URLContext *h, unsigned char *buf, int size)
+{
+    Context *c = h->priv_data;
+    int ret;
+
+    ret = ffurl_read(c->inner, buf, size);
+
+    return ret;
+}
+
+static int64_t ipns_seek(URLContext *h, int64_t pos, int whence)
+{
+    Context *c = h->priv_data;
+    int64_t ret;
+
+    ret = ffurl_seek(c->inner, pos, whence);
+
+    return ret;
+}
+
+static int ipns_close(URLContext *h)
+{
+    Context *c = h->priv_data;
+    int ret;
+
+    ret = ffurl_closep(&c->inner);
+
+    return ret;
+}
+
+#define OFFSET(x) offsetof(Context, x)
+#define D AV_OPT_FLAG_DECODING_PARAM
+
+static const AVOption options[] = {
+    {"gateway", "The gateway to ask for IPFS data.", OFFSET(gateway), AV_OPT_TYPE_BINARY, .flags = D },
+    {NULL},
+};
+
+static const AVClass ipfs_context_class = {
+    .class_name = "IPFS",
+    .item_name  = av_default_item_name,
+    .option     = options,
+    .version    = LIBAVUTIL_VERSION_INT,
+};
+
+const URLProtocol ff_ipfs_protocol = {
+    .name                = "ipfs",
+    .url_open2           = ipfs_open,
+    .url_read            = ipfs_read,
+    .url_seek            = ipfs_seek,
+    .url_close           = ipfs_close,
+    .priv_data_size      = sizeof(Context),
+    .priv_data_class     = &ipfs_context_class,
+};
+
+const URLProtocol ff_infs_protocol = {
+    .name                = "ipns",
+    .url_open2           = ipfs_open,
+    .url_read            = ipfs_read,
+    .url_seek            = ipfs_seek,
+    .url_close           = ipfs_close,
+    .priv_data_size      = sizeof(Context),
+    .priv_data_class     = &ipfs_context_class,
+};
diff --git a/libavformat/protocols.c b/libavformat/protocols.c
index 948fae411f..675b684bd3 100644
--- a/libavformat/protocols.c
+++ b/libavformat/protocols.c
@@ -73,6 +73,8 @@ extern const URLProtocol ff_libsrt_protocol;
 extern const URLProtocol ff_libssh_protocol;
 extern const URLProtocol ff_libsmbclient_protocol;
 extern const URLProtocol ff_libzmq_protocol;
+extern const URLProtocol ff_ipfs_protocol;
+extern const URLProtocol ff_ipns_protocol;
 
 #include "libavformat/protocol_list.c"
 
-- 
2.35.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/5] Fix up IPNS support.

[Mark Gaiser]

Signed-off-by: Mark Gaiser <markg85@gmail.com>
---
 configure            |  1 +
 libavformat/Makefile |  1 +
 libavformat/ipfs.c   | 12 ++++++------
 3 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/configure b/configure
index e466f924a3..6ff09e7974 100755
--- a/configure
+++ b/configure
@@ -3586,6 +3586,7 @@ udplite_protocol_select="network"
 unix_protocol_deps="sys_un_h"
 unix_protocol_select="network"
 ipfs_protocol_select="https_protocol"
+ipns_protocol_select="https_protocol"
 
 # external library protocols
 libamqp_protocol_deps="librabbitmq"
diff --git a/libavformat/Makefile b/libavformat/Makefile
index 983a77f4f2..ed5a7ffc98 100644
--- a/libavformat/Makefile
+++ b/libavformat/Makefile
@@ -657,6 +657,7 @@ OBJS-$(CONFIG_SUBFILE_PROTOCOL)          += subfile.o
 OBJS-$(CONFIG_TEE_PROTOCOL)              += teeproto.o tee_common.o
 OBJS-$(CONFIG_TCP_PROTOCOL)              += tcp.o
 OBJS-$(CONFIG_IPFS_PROTOCOL)             += ipfs.o
+OBJS-$(CONFIG_IPNS_PROTOCOL)             += ipfs.o
 TLS-OBJS-$(CONFIG_GNUTLS)                += tls_gnutls.o
 TLS-OBJS-$(CONFIG_LIBTLS)                += tls_libtls.o
 TLS-OBJS-$(CONFIG_MBEDTLS)               += tls_mbedtls.o
diff --git a/libavformat/ipfs.c b/libavformat/ipfs.c
index 4cc65750ed..8daf032dd2 100644
--- a/libavformat/ipfs.c
+++ b/libavformat/ipfs.c
@@ -57,7 +57,7 @@ static int ipfs_open(URLContext *h, const char *uri, int flags, AVDictionary **o
         goto err;
     }
     
-    char* ipfs_gateway = "https://ipfs.io/ipfs/";
+    char* ipfs_gateway = "http://localhost:8080/";
     
     c->fulluri = malloc(strlen(ipfs_gateway)+strlen(gatewaysuffix) + 1);
     
@@ -191,12 +191,12 @@ const URLProtocol ff_ipfs_protocol = {
     .priv_data_class     = &ipfs_context_class,
 };
 
-const URLProtocol ff_infs_protocol = {
+const URLProtocol ff_ipns_protocol = {
     .name                = "ipns",
-    .url_open2           = ipfs_open,
-    .url_read            = ipfs_read,
-    .url_seek            = ipfs_seek,
-    .url_close           = ipfs_close,
+    .url_open2           = ipns_open,
+    .url_read            = ipns_read,
+    .url_seek            = ipns_seek,
+    .url_close           = ipns_close,
     .priv_data_size      = sizeof(Context),
     .priv_data_class     = &ipfs_context_class,
 };
-- 
2.35.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 3/5] Merge IPNS and IPFS handling.

[Mark Gaiser]

Only the open function needs to detect which one is used.

Signed-off-by: Mark Gaiser <markg85@gmail.com>
---
 libavformat/ipfs.c | 99 +++++++++++-----------------------------------
 1 file changed, 22 insertions(+), 77 deletions(-)

diff --git a/libavformat/ipfs.c b/libavformat/ipfs.c
index 8daf032dd2..22487c6cc8 100644
--- a/libavformat/ipfs.c
+++ b/libavformat/ipfs.c
@@ -1,6 +1,6 @@
 /*
  * IPFS protocol.
- * Copyright (c) 2021 Mark Gaiser
+ * Copyright (c) 2022 Mark Gaiser
  *
  * This file is part of FFmpeg.
  *
@@ -47,23 +47,30 @@ typedef struct Context {
 static int ipfs_open(URLContext *h, const char *uri, int flags, AVDictionary **options)
 {
     const char *gatewaysuffix;
+    const char *protocolPathSuffix = "ipfs/";
     int ret = 0;
     Context *c = h->priv_data;
+    int isIpfs = (av_strstart(uri, "ipfs://", &gatewaysuffix) || av_strstart(uri, "ipfs:", &gatewaysuffix));
+    int isIpns = (av_strstart(uri, "ipns://", &gatewaysuffix) || av_strstart(uri, "ipns:", &gatewaysuffix));
     
-    if (!av_strstart(uri, "ipfs://", &gatewaysuffix) &&
-        !av_strstart(uri, "ipfs:", &gatewaysuffix)) {
-        av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri);
+    if (!isIpfs && !isIpns) {
         ret = AVERROR(EINVAL);
+        av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri);
         goto err;
     }
-    
-    char* ipfs_gateway = "http://localhost:8080/";
-    
-    c->fulluri = malloc(strlen(ipfs_gateway)+strlen(gatewaysuffix) + 1);
-    
-    strcpy(c->fulluri, ipfs_gateway);
+
+    // If we have IPNS, update the protocol
+    if (isIpns) {
+        protocolPathSuffix = "ipns/";
+    }
+
+    // Concatenate the url. This ends up with something like: http://localhost:8080/ipfs/Qm.....
+    c->fulluri = malloc(strlen(c->gateway) + strlen(protocolPathSuffix) + strlen(gatewaysuffix) + 1);
+    strcpy(c->fulluri, c->gateway);
+    strcat(c->fulluri, protocolPathSuffix);
     strcat(c->fulluri, gatewaysuffix);
     
+    // Pass the URL back to FFMpeg's protocol handler.
     if ((ret = ffurl_open_whitelist(&c->inner, c->fulluri, flags,
                                     &h->interrupt_callback, options,
                                     h->protocol_whitelist, h->protocol_blacklist, h)) < 0) {
@@ -105,72 +112,10 @@ static int ipfs_close(URLContext *h)
     return ret;
 }
 
-static int ipns_open(URLContext *h, const char *uri, int flags, AVDictionary **options)
-{
-    const char *gatewaysuffix;
-    int ret = 0;
-    Context *c = h->priv_data;
-    
-    if (!av_strstart(uri, "ipns://", &gatewaysuffix) &&
-        !av_strstart(uri, "ipns:", &gatewaysuffix)) {
-        av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri);
-        ret = AVERROR(EINVAL);
-        goto err;
-    }
-    
-    char* ipfs_gateway = "https://ipfs.io/ipns/";
-    
-    c->fulluri = malloc(strlen(ipfs_gateway)+strlen(gatewaysuffix) + 1);
-    
-    strcpy(c->fulluri, ipfs_gateway);
-    strcat(c->fulluri, gatewaysuffix);
-    
-    if ((ret = ffurl_open_whitelist(&c->inner, c->fulluri, flags,
-                                    &h->interrupt_callback, options,
-                                    h->protocol_whitelist, h->protocol_blacklist, h)) < 0) {
-        av_log(h, AV_LOG_ERROR, "Unable to open resource: %s\n", c->fulluri);
-        goto err;
-    }
-    
-err:
-    return ret;
-}
-
-static int ipns_read(URLContext *h, unsigned char *buf, int size)
-{
-    Context *c = h->priv_data;
-    int ret;
-
-    ret = ffurl_read(c->inner, buf, size);
-
-    return ret;
-}
-
-static int64_t ipns_seek(URLContext *h, int64_t pos, int whence)
-{
-    Context *c = h->priv_data;
-    int64_t ret;
-
-    ret = ffurl_seek(c->inner, pos, whence);
-
-    return ret;
-}
-
-static int ipns_close(URLContext *h)
-{
-    Context *c = h->priv_data;
-    int ret;
-
-    ret = ffurl_closep(&c->inner);
-
-    return ret;
-}
-
 #define OFFSET(x) offsetof(Context, x)
-#define D AV_OPT_FLAG_DECODING_PARAM
 
 static const AVOption options[] = {
-    {"gateway", "The gateway to ask for IPFS data.", OFFSET(gateway), AV_OPT_TYPE_BINARY, .flags = D },
+    {"gateway", "The gateway to ask for IPFS data.", OFFSET(gateway), AV_OPT_TYPE_STRING, {.str = "http://localhost:8080/"}, 0, 0, AV_OPT_FLAG_DECODING_PARAM },
     {NULL},
 };
 
@@ -193,10 +138,10 @@ const URLProtocol ff_ipfs_protocol = {
 
 const URLProtocol ff_ipns_protocol = {
     .name                = "ipns",
-    .url_open2           = ipns_open,
-    .url_read            = ipns_read,
-    .url_seek            = ipns_seek,
-    .url_close           = ipns_close,
+    .url_open2           = ipfs_open,
+    .url_read            = ipfs_read,
+    .url_seek            = ipfs_seek,
+    .url_close           = ipfs_close,
     .priv_data_size      = sizeof(Context),
     .priv_data_class     = &ipfs_context_class,
 };
-- 
2.35.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 4/5] Implement logic to determine the IPFS gateway.

[Mark Gaiser]

Inform the user about how to set a gateway if none could be found.

Signed-off-by: Mark Gaiser <markg85@gmail.com>
---
 libavformat/ipfs.c | 159 +++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 147 insertions(+), 12 deletions(-)

diff --git a/libavformat/ipfs.c b/libavformat/ipfs.c
index 22487c6cc8..8f6f91d85d 100644
--- a/libavformat/ipfs.c
+++ b/libavformat/ipfs.c
@@ -44,31 +44,166 @@ typedef struct Context {
     char *gateway;
 } Context;
 
+// A best-effort way to find the IPFS gateway.
+// Only the most appropiate gateway is set. It's not actually requested (http call) to prevent
+// potential slowdown in startup. A potential timeout is handled by the HTTP protocol.
+//
+// When done and the return value is 1, a potential IPFS Gateway is set in the gateway
+// variable in the inner URLContext.
+static int populate_ipfs_gateway(URLContext *h)
+{
+    Context *c = h->priv_data;
+    char *home_folder;
+    const char *ipfs_data_folder = "/.ipfs/";
+    char *ipfs_full_data_folder;
+    char *ipfs_gateway_file;
+    struct stat st;
+    int stat_ret = 0;
+    FILE *gateway_file;
+    char gateway_file_data[1000];
+
+    // First, test if there already is a path in c->gateway. If it is then it was provided as
+    // cli arument and should be used. It takes precdence.
+    if (c->gateway != NULL) {
+        return 1;
+    }
+
+    // Test $IPFS_GATEWAY
+    c->gateway = getenv("IPFS_GATEWAY");
+    if (c->gateway) {
+        return 1;
+    } else {
+        av_log(h, AV_LOG_DEBUG, "$IPFS_GATEWAY is empty.\n");
+    }
+
+    // We need to know the IPFS folder to - eventually - read the contents of the "gateway" file which would tell us the gateway to use.
+    ipfs_full_data_folder = getenv("IPFS_PATH");
+
+    if (!ipfs_full_data_folder) {
+        av_log(h, AV_LOG_DEBUG, "$IPFS_PATH is empty.\n");
+
+        // Try via the home folder
+        home_folder = getenv("HOME");
+        ipfs_full_data_folder = malloc(strlen(home_folder) + strlen(ipfs_data_folder) + 1);
+        strcpy(ipfs_full_data_folder, home_folder);
+        strcat(ipfs_full_data_folder, ipfs_data_folder);
+
+        // Stat the folder. It should exist in a default IPFS setup when run as local user.
+#ifndef _WIN32
+        stat_ret = stat(ipfs_full_data_folder, &st);
+#else
+        stat_ret = win32_stat(ipfs_full_data_folder, &st);
+#endif
+        if (stat_ret < 0) {
+            av_log(h, AV_LOG_DEBUG, "Unable to find IPFS folder. We tried:\n");
+            av_log(h, AV_LOG_DEBUG, "- $IPFS_PATH, which was empty.\n");
+            av_log(h, AV_LOG_DEBUG, "- $HOME/.ipfs (full uri: %s) which doesn't exist.\n", ipfs_full_data_folder);
+            return -1;
+        }
+    }
+
+    ipfs_gateway_file = malloc(strlen(ipfs_full_data_folder) + strlen("gateway") + 1);
+    strcpy(ipfs_gateway_file, ipfs_full_data_folder);
+    strcat(ipfs_gateway_file, "gateway");
+
+    // Stat the gateway file. If it doesn't exist we have no gateway. If it does, we might have one.
+#ifndef _WIN32
+    stat_ret = stat(ipfs_gateway_file, &st);
+#else
+    stat_ret = win32_stat(ipfs_gateway_file, &st);
+#endif
+    if (stat_ret < 0) {
+        av_log(h, AV_LOG_ERROR, "The IPFS gateway file (full uri: %s) doesn't exist. Is the gateway enabled?\n", ipfs_gateway_file);
+        return -1;
+    }
+
+    // Get the contents of the gateway file
+    gateway_file = av_fopen_utf8(ipfs_gateway_file, "r");
+    if (!gateway_file) {
+        av_log(h, AV_LOG_ERROR, "Unable to open the IPFS gateway file (full uri: %s).\n", ipfs_gateway_file);
+        return -1;
+    }
+
+    fscanf(gateway_file, "%[^\n]", gateway_file_data);
+    fclose(gateway_file);
+
+    if ((st.st_size < 1) || (strlen(gateway_file_data) < 1)) {
+        av_log(h, AV_LOG_ERROR, "The IPFS gateway file (full uri: %s) appears to be empty. Is the gateway started?\n", ipfs_gateway_file);
+        return -1;
+    }
+
+    // Copy the gateway url into c->gateway
+    c->gateway = malloc(strlen(gateway_file_data) + 1);
+    strcpy(c->gateway, gateway_file_data);
+    if (c->gateway) {
+        return 1;
+    } else {
+        av_log(h, AV_LOG_DEBUG, "Unknown error in the IPFS gateway file.\n");
+    }
+
+    return -1;
+}
+
+// For now just makes sure that the gateway ends in url we expect. Like http://localhost:8080/.
+// Explicitly with the traling slash.
+static void sanitize_ipfs_gateway(URLContext *h)
+{
+    Context *c = h->priv_data;
+    const char last_gateway_char = c->gateway[strlen(c->gateway) - 1];
+    char *sanitized_gateway = c->gateway;
+
+    if (last_gateway_char != '/') {
+        sanitized_gateway = malloc(strlen(c->gateway) + 2);
+        strcpy(sanitized_gateway, c->gateway);
+        strcat(sanitized_gateway, "/");
+    }
+
+    c->gateway = sanitized_gateway;
+}
+
 static int ipfs_open(URLContext *h, const char *uri, int flags, AVDictionary **options)
 {
-    const char *gatewaysuffix;
-    const char *protocolPathSuffix = "ipfs/";
+    const char *gateway_suffix;
+    const char *protocol_path_suffix = "ipfs/";
     int ret = 0;
     Context *c = h->priv_data;
-    int isIpfs = (av_strstart(uri, "ipfs://", &gatewaysuffix) || av_strstart(uri, "ipfs:", &gatewaysuffix));
-    int isIpns = (av_strstart(uri, "ipns://", &gatewaysuffix) || av_strstart(uri, "ipns:", &gatewaysuffix));
-    
-    if (!isIpfs && !isIpns) {
+    int is_ipfs = (av_strstart(uri, "ipfs://", &gateway_suffix) || av_strstart(uri, "ipfs:", &gateway_suffix));
+    int is_ipns = (av_strstart(uri, "ipns://", &gateway_suffix) || av_strstart(uri, "ipns:", &gateway_suffix));
+
+    // Populate the IPFS gateway if we have any.
+    // If not, inform the user how to properly set one.
+    if (populate_ipfs_gateway(h) < 0) {
+        av_log(h, AV_LOG_ERROR, "No IPFS gateway was set. Make sure a local IPFS instance is running.\n");
+        av_log(h, AV_LOG_INFO, "There are multiple options to define this gateway. The below options are in order of precedence:\n");
+        av_log(h, AV_LOG_INFO, "1. Define a -gateway <url> to the gateway without trailing forward slash.\n");
+        av_log(h, AV_LOG_INFO, "2. Define $IPFS_GATEWAY with the full http link to the gateway without trailing forward slash.\n");
+        av_log(h, AV_LOG_INFO, "3. Define $IPFS_PATH and point it to the IPFS data path.\n");
+        av_log(h, AV_LOG_INFO, "4. Have IPFS running in your local user folder (under $HOME/.ipfs).\n");
+        av_log(h, AV_LOG_INFO, "In all path cases, a file named gateway is expected. See https://github.com/ipfs/specs/issues/261 for more information.\n");
+        ret = AVERROR(EINVAL);
+        goto err;
+    }
+
+    // Sanitize the gateway to a format we expect.
+    sanitize_ipfs_gateway(h);
+
+    // We must have either ipns or ipfs.
+    if (!is_ipfs && !is_ipns) {
         ret = AVERROR(EINVAL);
         av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri);
         goto err;
     }
 
     // If we have IPNS, update the protocol
-    if (isIpns) {
-        protocolPathSuffix = "ipns/";
+    if (is_ipns) {
+        protocol_path_suffix = "ipns/";
     }
 
     // Concatenate the url. This ends up with something like: http://localhost:8080/ipfs/Qm.....
-    c->fulluri = malloc(strlen(c->gateway) + strlen(protocolPathSuffix) + strlen(gatewaysuffix) + 1);
+    c->fulluri = malloc(strlen(c->gateway) + strlen(protocol_path_suffix) + strlen(gateway_suffix) + 1);
     strcpy(c->fulluri, c->gateway);
-    strcat(c->fulluri, protocolPathSuffix);
-    strcat(c->fulluri, gatewaysuffix);
+    strcat(c->fulluri, protocol_path_suffix);
+    strcat(c->fulluri, gateway_suffix);
     
     // Pass the URL back to FFMpeg's protocol handler.
     if ((ret = ffurl_open_whitelist(&c->inner, c->fulluri, flags,
@@ -115,7 +250,7 @@ static int ipfs_close(URLContext *h)
 #define OFFSET(x) offsetof(Context, x)
 
 static const AVOption options[] = {
-    {"gateway", "The gateway to ask for IPFS data.", OFFSET(gateway), AV_OPT_TYPE_STRING, {.str = "http://localhost:8080/"}, 0, 0, AV_OPT_FLAG_DECODING_PARAM },
+    {"gateway", "The gateway to ask for IPFS data.", OFFSET(gateway), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, AV_OPT_FLAG_DECODING_PARAM },
     {NULL},
 };
 
-- 
2.35.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 5/5] Fix review feedback

[Mark Gaiser]

Little bit of consistency in punctuations.

Signed-off-by: Mark Gaiser <markg85@gmail.com>
---
 libavformat/ipfs.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/libavformat/ipfs.c b/libavformat/ipfs.c
index 8f6f91d85d..1644c74eac 100644
--- a/libavformat/ipfs.c
+++ b/libavformat/ipfs.c
@@ -46,7 +46,7 @@ typedef struct Context {
 
 // A best-effort way to find the IPFS gateway.
 // Only the most appropiate gateway is set. It's not actually requested (http call) to prevent
-// potential slowdown in startup. A potential timeout is handled by the HTTP protocol.
+// a potential slowdown in startup. A potential timeout is handled by the HTTP protocol.
 //
 // When done and the return value is 1, a potential IPFS Gateway is set in the gateway
 // variable in the inner URLContext.
@@ -68,7 +68,7 @@ static int populate_ipfs_gateway(URLContext *h)
         return 1;
     }
 
-    // Test $IPFS_GATEWAY
+    // Test $IPFS_GATEWAY.
     c->gateway = getenv("IPFS_GATEWAY");
     if (c->gateway) {
         return 1;
@@ -82,7 +82,7 @@ static int populate_ipfs_gateway(URLContext *h)
     if (!ipfs_full_data_folder) {
         av_log(h, AV_LOG_DEBUG, "$IPFS_PATH is empty.\n");
 
-        // Try via the home folder
+        // Try via the home folder.
         home_folder = getenv("HOME");
         ipfs_full_data_folder = malloc(strlen(home_folder) + strlen(ipfs_data_folder) + 1);
         strcpy(ipfs_full_data_folder, home_folder);
@@ -117,7 +117,7 @@ static int populate_ipfs_gateway(URLContext *h)
         return -1;
     }
 
-    // Get the contents of the gateway file
+    // Get the contents of the gateway file.
     gateway_file = av_fopen_utf8(ipfs_gateway_file, "r");
     if (!gateway_file) {
         av_log(h, AV_LOG_ERROR, "Unable to open the IPFS gateway file (full uri: %s).\n", ipfs_gateway_file);
@@ -132,7 +132,7 @@ static int populate_ipfs_gateway(URLContext *h)
         return -1;
     }
 
-    // Copy the gateway url into c->gateway
+    // Copy the gateway url into c->gateway.
     c->gateway = malloc(strlen(gateway_file_data) + 1);
     strcpy(c->gateway, gateway_file_data);
     if (c->gateway) {
@@ -194,7 +194,7 @@ static int ipfs_open(URLContext *h, const char *uri, int flags, AVDictionary **o
         goto err;
     }
 
-    // If we have IPNS, update the protocol
+    // If we have IPNS, update the protocol.
     if (is_ipns) {
         protocol_path_suffix = "ipns/";
     }
-- 
2.35.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 5/5] Fix review feedback

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 759 bytes --]

On Mon, Jan 31, 2022 at 02:51:16PM +0100, Mark Gaiser wrote:
> Little bit of consistency in punctuations.
> 
> Signed-off-by: Mark Gaiser <markg85@gmail.com>
> ---
>  libavformat/ipfs.c | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)

Review fixes should be integrated into the patches when they are not applied
yet. not done as a seperate patch on top

also there are some tabs and trailing whitespace in the earlier patches
which should be fixed too

thx


[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Into a blind darkness they enter who follow after the Ignorance,
they as if into a greater darkness enter who devote themselves
to the Knowledge alone. -- Isha Upanishad

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/5] Add IPFS and IPNS protocol support

[Tomas Härdin]

mån 2022-01-31 klockan 14:51 +0100 skrev Mark Gaiser:
> 
> There are multiple ways to access files on the IPFS network. This
> patch series
> uses the gateway driven way. An IPFS node - by default - exposes a
> local 
> gateway (say http://localhost:8080) which is then used to get content
> from IPFS.


Perhaps the protocol should be called something other than just ipfs if
it doesn't actually implement IPFS. Like ipfsgateway. It could still be
registered to ipfs:// of course, until someone writes a wrapper for
libipfs.

/Tomas

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/5] Early version of IPFS protocol support.

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 5705 bytes --]

On Mon, Jan 31, 2022 at 02:51:12PM +0100, Mark Gaiser wrote:
> Signed-off-by: Mark Gaiser <markg85@gmail.com>
> ---
>  configure               |   1 +
>  doc/protocols.texi      |  30 ++++++
>  libavformat/Makefile    |   1 +
>  libavformat/ipfs.c      | 202 ++++++++++++++++++++++++++++++++++++++++
>  libavformat/protocols.c |   2 +
>  5 files changed, 236 insertions(+)
>  create mode 100644 libavformat/ipfs.c
> 
> diff --git a/configure b/configure
> index 5b19a35f59..e466f924a3 100755
> --- a/configure
> +++ b/configure
> @@ -3585,6 +3585,7 @@ udp_protocol_select="network"
>  udplite_protocol_select="network"
>  unix_protocol_deps="sys_un_h"
>  unix_protocol_select="network"
> +ipfs_protocol_select="https_protocol"
>  
>  # external library protocols
>  libamqp_protocol_deps="librabbitmq"
> diff --git a/doc/protocols.texi b/doc/protocols.texi
> index d207df0b52..7c9c0a4808 100644
> --- a/doc/protocols.texi
> +++ b/doc/protocols.texi
> @@ -2025,5 +2025,35 @@ decoding errors.
>  
>  @end table
>  
> +@section ipfs
> +
> +InterPlanetary File System (IPFS) protocol support. One can access files stored 
> +on the IPFS network through so called gateways. Those are http(s) endpoints.
> +This protocol wraps the IPFS native protocols (ipfs:// and ipns://) to be send 
> +to such a gateway. Users can (and should) host their own node which means this 
> +protocol will use your local machine gateway to access files on the IPFS network.
> +
> +If a user doesn't have a node of their own then the public gateway dweb.link is 
> +used by default.
> +
> +You can use this protocol in 2 ways. Using IPFS:
> +@example
> +ffplay ipfs://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> +@end example
> +
> +Or the IPNS protocol (IPNS is mutable IPFS):
> +@example
> +ffplay ipns://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> +@end example
> +
> +You can also change the gateway to be used:
> +
> +@table @option
> +
> +@item gateway
> +Defines the gateway to use. When nothing is provided the protocol will first try 
> +your local gateway. If that fails dweb.link will be used.
> +
> +@end table
>  
>  @c man end PROTOCOLS
> diff --git a/libavformat/Makefile b/libavformat/Makefile
> index 3dc6a479cc..983a77f4f2 100644
> --- a/libavformat/Makefile
> +++ b/libavformat/Makefile
> @@ -656,6 +656,7 @@ OBJS-$(CONFIG_SRTP_PROTOCOL)             += srtpproto.o srtp.o
>  OBJS-$(CONFIG_SUBFILE_PROTOCOL)          += subfile.o
>  OBJS-$(CONFIG_TEE_PROTOCOL)              += teeproto.o tee_common.o
>  OBJS-$(CONFIG_TCP_PROTOCOL)              += tcp.o
> +OBJS-$(CONFIG_IPFS_PROTOCOL)             += ipfs.o
>  TLS-OBJS-$(CONFIG_GNUTLS)                += tls_gnutls.o
>  TLS-OBJS-$(CONFIG_LIBTLS)                += tls_libtls.o
>  TLS-OBJS-$(CONFIG_MBEDTLS)               += tls_mbedtls.o
> diff --git a/libavformat/ipfs.c b/libavformat/ipfs.c
> new file mode 100644
> index 0000000000..4cc65750ed
> --- /dev/null
> +++ b/libavformat/ipfs.c
> @@ -0,0 +1,202 @@
> +/*
> + * IPFS protocol.
> + * Copyright (c) 2021 Mark Gaiser
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +#include "libavutil/avassert.h"
> +#include "libavutil/avstring.h"
> +#include "libavutil/internal.h"
> +#include "libavutil/opt.h"
> +#include "libavutil/tree.h"
> +#include "avformat.h"
> +#include <fcntl.h>
> +#if HAVE_IO_H
> +#include <io.h>
> +#endif
> +#if HAVE_UNISTD_H
> +#include <unistd.h>
> +#endif
> +#include <sys/stat.h>
> +#include <stdlib.h>
> +#include "os_support.h"
> +#include "url.h"
> +
> +typedef struct Context {
> +    AVClass *class;
> +    URLContext *inner;
> +    char *fulluri;
> +    char *gateway;
> +} Context;
> +
> +static int ipfs_open(URLContext *h, const char *uri, int flags, AVDictionary **options)
> +{
> +    const char *gatewaysuffix;

> +    int ret = 0;

the initializuation seems redundant


> +    Context *c = h->priv_data;
> +    
> +    if (!av_strstart(uri, "ipfs://", &gatewaysuffix) &&
> +        !av_strstart(uri, "ipfs:", &gatewaysuffix)) {
> +        av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri);
> +        ret = AVERROR(EINVAL);
> +        goto err;
> +    }
> +    
> +    char* ipfs_gateway = "https://ipfs.io/ipfs/";
> +    
> +    c->fulluri = malloc(strlen(ipfs_gateway)+strlen(gatewaysuffix) + 1);
> +    
> +    strcpy(c->fulluri, ipfs_gateway);
> +    strcat(c->fulluri, gatewaysuffix);

malloc() should be av_malloc() unless there is some API interaction requiring otherwise
also all the  cpy/cat stuff should use "secure" size checking variants
but maybe something like av_asprintf() could simplify this all

[...]

thx


-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Elect your leaders based on what they did after the last election, not
based on what they say before an election.


[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/5] Fix up IPNS support.

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 601 bytes --]

On Mon, Jan 31, 2022 at 02:51:13PM +0100, Mark Gaiser wrote:
> Signed-off-by: Mark Gaiser <markg85@gmail.com>
> ---
>  configure            |  1 +
>  libavformat/Makefile |  1 +
>  libavformat/ipfs.c   | 12 ++++++------
>  3 files changed, 8 insertions(+), 6 deletions(-)

as with other fixes they should be stashed in the patch that contains the bug

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Complexity theory is the science of finding the exact solution to an
approximation. Benchmarking OTOH is finding an approximation of the exact

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/5] Early version of IPFS protocol support.

[James Almer]

On 1/31/2022 10:51 AM, Mark Gaiser wrote:
> Signed-off-by: Mark Gaiser <markg85@gmail.com>
> ---
>   configure               |   1 +
>   doc/protocols.texi      |  30 ++++++
>   libavformat/Makefile    |   1 +
>   libavformat/ipfs.c      | 202 ++++++++++++++++++++++++++++++++++++++++
>   libavformat/protocols.c |   2 +
>   5 files changed, 236 insertions(+)
>   create mode 100644 libavformat/ipfs.c
> 
> diff --git a/configure b/configure
> index 5b19a35f59..e466f924a3 100755
> --- a/configure
> +++ b/configure
> @@ -3585,6 +3585,7 @@ udp_protocol_select="network"
>   udplite_protocol_select="network"
>   unix_protocol_deps="sys_un_h"
>   unix_protocol_select="network"
> +ipfs_protocol_select="https_protocol"
>   
>   # external library protocols
>   libamqp_protocol_deps="librabbitmq"
> diff --git a/doc/protocols.texi b/doc/protocols.texi
> index d207df0b52..7c9c0a4808 100644
> --- a/doc/protocols.texi
> +++ b/doc/protocols.texi
> @@ -2025,5 +2025,35 @@ decoding errors.
>   
>   @end table
>   
> +@section ipfs
> +
> +InterPlanetary File System (IPFS) protocol support. One can access files stored
> +on the IPFS network through so called gateways. Those are http(s) endpoints.
> +This protocol wraps the IPFS native protocols (ipfs:// and ipns://) to be send
> +to such a gateway. Users can (and should) host their own node which means this
> +protocol will use your local machine gateway to access files on the IPFS network.
> +
> +If a user doesn't have a node of their own then the public gateway dweb.link is
> +used by default.
> +
> +You can use this protocol in 2 ways. Using IPFS:
> +@example
> +ffplay ipfs://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> +@end example
> +
> +Or the IPNS protocol (IPNS is mutable IPFS):
> +@example
> +ffplay ipns://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> +@end example
> +
> +You can also change the gateway to be used:
> +
> +@table @option
> +
> +@item gateway
> +Defines the gateway to use. When nothing is provided the protocol will first try
> +your local gateway. If that fails dweb.link will be used.
> +
> +@end table
>   
>   @c man end PROTOCOLS
> diff --git a/libavformat/Makefile b/libavformat/Makefile
> index 3dc6a479cc..983a77f4f2 100644
> --- a/libavformat/Makefile
> +++ b/libavformat/Makefile
> @@ -656,6 +656,7 @@ OBJS-$(CONFIG_SRTP_PROTOCOL)             += srtpproto.o srtp.o
>   OBJS-$(CONFIG_SUBFILE_PROTOCOL)          += subfile.o
>   OBJS-$(CONFIG_TEE_PROTOCOL)              += teeproto.o tee_common.o
>   OBJS-$(CONFIG_TCP_PROTOCOL)              += tcp.o
> +OBJS-$(CONFIG_IPFS_PROTOCOL)             += ipfs.o
>   TLS-OBJS-$(CONFIG_GNUTLS)                += tls_gnutls.o
>   TLS-OBJS-$(CONFIG_LIBTLS)                += tls_libtls.o
>   TLS-OBJS-$(CONFIG_MBEDTLS)               += tls_mbedtls.o
> diff --git a/libavformat/ipfs.c b/libavformat/ipfs.c
> new file mode 100644
> index 0000000000..4cc65750ed
> --- /dev/null
> +++ b/libavformat/ipfs.c
> @@ -0,0 +1,202 @@
> +/*
> + * IPFS protocol.
> + * Copyright (c) 2021 Mark Gaiser
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +#include "libavutil/avassert.h"
> +#include "libavutil/avstring.h"
> +#include "libavutil/internal.h"
> +#include "libavutil/opt.h"
> +#include "libavutil/tree.h"
> +#include "avformat.h"
> +#include <fcntl.h>
> +#if HAVE_IO_H
> +#include <io.h>
> +#endif
> +#if HAVE_UNISTD_H
> +#include <unistd.h>
> +#endif
> +#include <sys/stat.h>
> +#include <stdlib.h>
> +#include "os_support.h"
> +#include "url.h"
> +
> +typedef struct Context {
> +    AVClass *class;
> +    URLContext *inner;
> +    char *fulluri;
> +    char *gateway;
> +} Context;

A more descriptive name would be nice.

> +
> +static int ipfs_open(URLContext *h, const char *uri, int flags, AVDictionary **options)
> +{
> +    const char *gatewaysuffix;
> +    int ret = 0;
> +    Context *c = h->priv_data;
> +
> +    if (!av_strstart(uri, "ipfs://", &gatewaysuffix) &&
> +        !av_strstart(uri, "ipfs:", &gatewaysuffix)) {
> +        av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri);
> +        ret = AVERROR(EINVAL);
> +        goto err;

Just return ret. No need for this goto.

> +    }
> +
> +    char* ipfs_gateway = "https://ipfs.io/ipfs/";

const char*

> +
> +    c->fulluri = malloc(strlen(ipfs_gateway)+strlen(gatewaysuffix) + 1);

av_malloc()

Also, check the return value for allocation failure.

> +
> +    strcpy(c->fulluri, ipfs_gateway);
> +    strcat(c->fulluri, gatewaysuffix);
> +
> +    if ((ret = ffurl_open_whitelist(&c->inner, c->fulluri, flags,

Is fulluri going to be used after this call? If not, then it should be 
local to this function as there's no need to have it in h->priv_data 
(And for that matter, you're not freeing fulluri anywhere).

> +                                    &h->interrupt_callback, options,
> +                                    h->protocol_whitelist, h->protocol_blacklist, h)) < 0) {

You need to define a whitelist including https in both URLProtocol below.

> +        av_log(h, AV_LOG_ERROR, "Unable to open resource: %s\n", c->fulluri);
> +        goto err;
> +    }
> +
> +err:
> +    return ret;
> +}
> +
> +static int ipfs_read(URLContext *h, unsigned char *buf, int size)
> +{
> +    Context *c = h->priv_data;
> +    int ret;
> +
> +    ret = ffurl_read(c->inner, buf, size);
> +
> +    return ret;
> +}
> +
> +static int64_t ipfs_seek(URLContext *h, int64_t pos, int whence)
> +{
> +    Context *c = h->priv_data;
> +    int64_t ret;
> +
> +    ret = ffurl_seek(c->inner, pos, whence);
> +
> +    return ret;
> +}
> +
> +static int ipfs_close(URLContext *h)
> +{
> +    Context *c = h->priv_data;
> +    int ret;
> +
> +    ret = ffurl_closep(&c->inner);
> +
> +    return ret;
> +}
> +
> +static int ipns_open(URLContext *h, const char *uri, int flags, AVDictionary **options)
> +{
> +    const char *gatewaysuffix;
> +    int ret = 0;
> +    Context *c = h->priv_data;
> +
> +    if (!av_strstart(uri, "ipns://", &gatewaysuffix) &&
> +        !av_strstart(uri, "ipns:", &gatewaysuffix)) {
> +        av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri);
> +        ret = AVERROR(EINVAL);
> +        goto err;
> +    }
> +
> +    char* ipfs_gateway = "https://ipfs.io/ipns/";
> +
> +    c->fulluri = malloc(strlen(ipfs_gateway)+strlen(gatewaysuffix) + 1);
> +
> +    strcpy(c->fulluri, ipfs_gateway);
> +    strcat(c->fulluri, gatewaysuffix);
> +
> +    if ((ret = ffurl_open_whitelist(&c->inner, c->fulluri, flags,
> +                                    &h->interrupt_callback, options,
> +                                    h->protocol_whitelist, h->protocol_blacklist, h)) < 0) {
> +        av_log(h, AV_LOG_ERROR, "Unable to open resource: %s\n", c->fulluri);
> +        goto err;
> +    }
> +
> +err:
> +    return ret;

Same comments as above apply here.

> +}
> +
> +static int ipns_read(URLContext *h, unsigned char *buf, int size)
> +{
> +    Context *c = h->priv_data;
> +    int ret;
> +
> +    ret = ffurl_read(c->inner, buf, size);
> +
> +    return ret;
> +}
> +
> +static int64_t ipns_seek(URLContext *h, int64_t pos, int whence)
> +{
> +    Context *c = h->priv_data;
> +    int64_t ret;
> +
> +    ret = ffurl_seek(c->inner, pos, whence);
> +
> +    return ret;
> +}
> +
> +static int ipns_close(URLContext *h)
> +{
> +    Context *c = h->priv_data;
> +    int ret;
> +
> +    ret = ffurl_closep(&c->inner);
> +
> +    return ret;
> +}
> +
> +#define OFFSET(x) offsetof(Context, x)
> +#define D AV_OPT_FLAG_DECODING_PARAM
> +
> +static const AVOption options[] = {
> +    {"gateway", "The gateway to ask for IPFS data.", OFFSET(gateway), AV_OPT_TYPE_BINARY, .flags = D },
> +    {NULL},
> +};
> +
> +static const AVClass ipfs_context_class = {
> +    .class_name = "IPFS",
> +    .item_name  = av_default_item_name,
> +    .option     = options,
> +    .version    = LIBAVUTIL_VERSION_INT,
> +};
> +
> +const URLProtocol ff_ipfs_protocol = {
> +    .name                = "ipfs",
> +    .url_open2           = ipfs_open,
> +    .url_read            = ipfs_read,
> +    .url_seek            = ipfs_seek,
> +    .url_close           = ipfs_close,
> +    .priv_data_size      = sizeof(Context),
> +    .priv_data_class     = &ipfs_context_class,
> +};
> +
> +const URLProtocol ff_infs_protocol = {
> +    .name                = "ipns",
> +    .url_open2           = ipfs_open,
> +    .url_read            = ipfs_read,
> +    .url_seek            = ipfs_seek,
> +    .url_close           = ipfs_close,
> +    .priv_data_size      = sizeof(Context),
> +    .priv_data_class     = &ipfs_context_class,
> +};
> diff --git a/libavformat/protocols.c b/libavformat/protocols.c
> index 948fae411f..675b684bd3 100644
> --- a/libavformat/protocols.c
> +++ b/libavformat/protocols.c
> @@ -73,6 +73,8 @@ extern const URLProtocol ff_libsrt_protocol;
>   extern const URLProtocol ff_libssh_protocol;
>   extern const URLProtocol ff_libsmbclient_protocol;
>   extern const URLProtocol ff_libzmq_protocol;
> +extern const URLProtocol ff_ipfs_protocol;
> +extern const URLProtocol ff_ipns_protocol;
>   
>   #include "libavformat/protocol_list.c"
>   
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/5] Add IPFS and IPNS protocol support

[Mark Gaiser]

On Mon, Jan 31, 2022 at 4:52 PM Tomas Härdin <tjoppen@acc.umu.se> wrote:

> mån 2022-01-31 klockan 14:51 +0100 skrev Mark Gaiser:
> >
> > There are multiple ways to access files on the IPFS network. This
> > patch series
> > uses the gateway driven way. An IPFS node - by default - exposes a
> > local
> > gateway (say http://localhost:8080) which is then used to get content
> > from IPFS.
>
>
> Perhaps the protocol should be called something other than just ipfs if
> it doesn't actually implement IPFS. Like ipfsgateway. It could still be
> registered to ipfs:// of course, until someone writes a wrapper for
> libipfs.
>

Do you mean to have it named like "ipfsgateway" as files (and library) but
keep the protocol registration of ipfs and ipns?
I'm fine with that. The name is only artificial in code anyhow, all that
matters are the protocol names.

Question though. In a V2 patch, would it make sense to squash everything in
one commit? It's not that much anyhow.
I have a feeling the current spliced patches give much appreciated feedback
but also on parts that a later commit removed :)

>
> /Tomas
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 5/5] Fix review feedback

[Mark Gaiser]

On Mon, Jan 31, 2022 at 4:46 PM Michael Niedermayer <michael@niedermayer.cc>
wrote:

> On Mon, Jan 31, 2022 at 02:51:16PM +0100, Mark Gaiser wrote:
> > Little bit of consistency in punctuations.
> >
> > Signed-off-by: Mark Gaiser <markg85@gmail.com>
> > ---
> >  libavformat/ipfs.c | 12 ++++++------
> >  1 file changed, 6 insertions(+), 6 deletions(-)
>
> Review fixes should be integrated into the patches when they are not
> applied
> yet. not done as a seperate patch on top
>
> also there are some tabs and trailing whitespace in the earlier patches
> which should be fixed too
>
> Thank you, Michael, that's much appreciated! I will fix those in V2.


> thx
>
>
> [...]
>
> --
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> Into a blind darkness they enter who follow after the Ignorance,
> they as if into a greater darkness enter who devote themselves
> to the Knowledge alone. -- Isha Upanishad
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/5] Early version of IPFS protocol support.

[Mark Gaiser]

On Mon, Jan 31, 2022 at 5:06 PM James Almer <jamrial@gmail.com> wrote:

>
>
> On 1/31/2022 10:51 AM, Mark Gaiser wrote:
> > Signed-off-by: Mark Gaiser <markg85@gmail.com>
> > ---
> >   configure               |   1 +
> >   doc/protocols.texi      |  30 ++++++
> >   libavformat/Makefile    |   1 +
> >   libavformat/ipfs.c      | 202 ++++++++++++++++++++++++++++++++++++++++
> >   libavformat/protocols.c |   2 +
> >   5 files changed, 236 insertions(+)
> >   create mode 100644 libavformat/ipfs.c
> >
> > diff --git a/configure b/configure
> > index 5b19a35f59..e466f924a3 100755
> > --- a/configure
> > +++ b/configure
> > @@ -3585,6 +3585,7 @@ udp_protocol_select="network"
> >   udplite_protocol_select="network"
> >   unix_protocol_deps="sys_un_h"
> >   unix_protocol_select="network"
> > +ipfs_protocol_select="https_protocol"
> >
> >   # external library protocols
> >   libamqp_protocol_deps="librabbitmq"
> > diff --git a/doc/protocols.texi b/doc/protocols.texi
> > index d207df0b52..7c9c0a4808 100644
> > --- a/doc/protocols.texi
> > +++ b/doc/protocols.texi
> > @@ -2025,5 +2025,35 @@ decoding errors.
> >
> >   @end table
> >
> > +@section ipfs
> > +
> > +InterPlanetary File System (IPFS) protocol support. One can access
> files stored
> > +on the IPFS network through so called gateways. Those are http(s)
> endpoints.
> > +This protocol wraps the IPFS native protocols (ipfs:// and ipns://) to
> be send
> > +to such a gateway. Users can (and should) host their own node which
> means this
> > +protocol will use your local machine gateway to access files on the
> IPFS network.
> > +
> > +If a user doesn't have a node of their own then the public gateway
> dweb.link is
> > +used by default.
> > +
> > +You can use this protocol in 2 ways. Using IPFS:
> > +@example
> > +ffplay ipfs://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> > +@end example
> > +
> > +Or the IPNS protocol (IPNS is mutable IPFS):
> > +@example
> > +ffplay ipns://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> > +@end example
> > +
> > +You can also change the gateway to be used:
> > +
> > +@table @option
> > +
> > +@item gateway
> > +Defines the gateway to use. When nothing is provided the protocol will
> first try
> > +your local gateway. If that fails dweb.link will be used.
> > +
> > +@end table
> >
> >   @c man end PROTOCOLS
> > diff --git a/libavformat/Makefile b/libavformat/Makefile
> > index 3dc6a479cc..983a77f4f2 100644
> > --- a/libavformat/Makefile
> > +++ b/libavformat/Makefile
> > @@ -656,6 +656,7 @@ OBJS-$(CONFIG_SRTP_PROTOCOL)             +=
> srtpproto.o srtp.o
> >   OBJS-$(CONFIG_SUBFILE_PROTOCOL)          += subfile.o
> >   OBJS-$(CONFIG_TEE_PROTOCOL)              += teeproto.o tee_common.o
> >   OBJS-$(CONFIG_TCP_PROTOCOL)              += tcp.o
> > +OBJS-$(CONFIG_IPFS_PROTOCOL)             += ipfs.o
> >   TLS-OBJS-$(CONFIG_GNUTLS)                += tls_gnutls.o
> >   TLS-OBJS-$(CONFIG_LIBTLS)                += tls_libtls.o
> >   TLS-OBJS-$(CONFIG_MBEDTLS)               += tls_mbedtls.o
> > diff --git a/libavformat/ipfs.c b/libavformat/ipfs.c
> > new file mode 100644
> > index 0000000000..4cc65750ed
> > --- /dev/null
> > +++ b/libavformat/ipfs.c
> > @@ -0,0 +1,202 @@
> > +/*
> > + * IPFS protocol.
> > + * Copyright (c) 2021 Mark Gaiser
> > + *
> > + * This file is part of FFmpeg.
> > + *
> > + * FFmpeg is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU Lesser General Public
> > + * License as published by the Free Software Foundation; either
> > + * version 2.1 of the License, or (at your option) any later version.
> > + *
> > + * FFmpeg is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > + * Lesser General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU Lesser General Public
> > + * License along with FFmpeg; if not, write to the Free Software
> > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA
> > + */
> > +
> > +#include "libavutil/avassert.h"
> > +#include "libavutil/avstring.h"
> > +#include "libavutil/internal.h"
> > +#include "libavutil/opt.h"
> > +#include "libavutil/tree.h"
> > +#include "avformat.h"
> > +#include <fcntl.h>
> > +#if HAVE_IO_H
> > +#include <io.h>
> > +#endif
> > +#if HAVE_UNISTD_H
> > +#include <unistd.h>
> > +#endif
> > +#include <sys/stat.h>
> > +#include <stdlib.h>
> > +#include "os_support.h"
> > +#include "url.h"
> > +
> > +typedef struct Context {
> > +    AVClass *class;
> > +    URLContext *inner;
> > +    char *fulluri;
> > +    char *gateway;
> > +} Context;
>
> A more descriptive name would be nice.
>
> > +
> > +static int ipfs_open(URLContext *h, const char *uri, int flags,
> AVDictionary **options)
> > +{
> > +    const char *gatewaysuffix;
> > +    int ret = 0;
> > +    Context *c = h->priv_data;
> > +
> > +    if (!av_strstart(uri, "ipfs://", &gatewaysuffix) &&
> > +        !av_strstart(uri, "ipfs:", &gatewaysuffix)) {
> > +        av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri);
> > +        ret = AVERROR(EINVAL);
> > +        goto err;
>
> Just return ret. No need for this goto.
>
> > +    }
> > +
> > +    char* ipfs_gateway = "https://ipfs.io/ipfs/";
>
> const char*
>
> > +
> > +    c->fulluri = malloc(strlen(ipfs_gateway)+strlen(gatewaysuffix) + 1);
>
> av_malloc()
>
> Also, check the return value for allocation failure.
>
> > +
> > +    strcpy(c->fulluri, ipfs_gateway);
> > +    strcat(c->fulluri, gatewaysuffix);
> > +
> > +    if ((ret = ffurl_open_whitelist(&c->inner, c->fulluri, flags,
>
> Is fulluri going to be used after this call? If not, then it should be
> local to this function as there's no need to have it in h->priv_data
> (And for that matter, you're not freeing fulluri anywhere).
>
> > +                                    &h->interrupt_callback, options,
> > +                                    h->protocol_whitelist,
> h->protocol_blacklist, h)) < 0) {
>
> You need to define a whitelist including https in both URLProtocol below.
>
> > +        av_log(h, AV_LOG_ERROR, "Unable to open resource: %s\n",
> c->fulluri);
> > +        goto err;
> > +    }
> > +
> > +err:
> > +    return ret;
> > +}
> > +
> > +static int ipfs_read(URLContext *h, unsigned char *buf, int size)
> > +{
> > +    Context *c = h->priv_data;
> > +    int ret;
> > +
> > +    ret = ffurl_read(c->inner, buf, size);
> > +
> > +    return ret;
> > +}
> > +
> > +static int64_t ipfs_seek(URLContext *h, int64_t pos, int whence)
> > +{
> > +    Context *c = h->priv_data;
> > +    int64_t ret;
> > +
> > +    ret = ffurl_seek(c->inner, pos, whence);
> > +
> > +    return ret;
> > +}
> > +
> > +static int ipfs_close(URLContext *h)
> > +{
> > +    Context *c = h->priv_data;
> > +    int ret;
> > +
> > +    ret = ffurl_closep(&c->inner);
> > +
> > +    return ret;
> > +}
> > +
> > +static int ipns_open(URLContext *h, const char *uri, int flags,
> AVDictionary **options)
> > +{
> > +    const char *gatewaysuffix;
> > +    int ret = 0;
> > +    Context *c = h->priv_data;
> > +
> > +    if (!av_strstart(uri, "ipns://", &gatewaysuffix) &&
> > +        !av_strstart(uri, "ipns:", &gatewaysuffix)) {
> > +        av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri);
> > +        ret = AVERROR(EINVAL);
> > +        goto err;
> > +    }
> > +
> > +    char* ipfs_gateway = "https://ipfs.io/ipns/";
> > +
> > +    c->fulluri = malloc(strlen(ipfs_gateway)+strlen(gatewaysuffix) + 1);
> > +
> > +    strcpy(c->fulluri, ipfs_gateway);
> > +    strcat(c->fulluri, gatewaysuffix);
> > +
> > +    if ((ret = ffurl_open_whitelist(&c->inner, c->fulluri, flags,
> > +                                    &h->interrupt_callback, options,
> > +                                    h->protocol_whitelist,
> h->protocol_blacklist, h)) < 0) {
> > +        av_log(h, AV_LOG_ERROR, "Unable to open resource: %s\n",
> c->fulluri);
> > +        goto err;
> > +    }
> > +
> > +err:
> > +    return ret;
>
> Same comments as above apply here.
>

Awesome, thank you for your review time!
Will fix all those in V2.

>
> > +}
> > +
> > +static int ipns_read(URLContext *h, unsigned char *buf, int size)
> > +{
> > +    Context *c = h->priv_data;
> > +    int ret;
> > +
> > +    ret = ffurl_read(c->inner, buf, size);
> > +
> > +    return ret;
> > +}
> > +
> > +static int64_t ipns_seek(URLContext *h, int64_t pos, int whence)
> > +{
> > +    Context *c = h->priv_data;
> > +    int64_t ret;
> > +
> > +    ret = ffurl_seek(c->inner, pos, whence);
> > +
> > +    return ret;
> > +}
> > +
> > +static int ipns_close(URLContext *h)
> > +{
> > +    Context *c = h->priv_data;
> > +    int ret;
> > +
> > +    ret = ffurl_closep(&c->inner);
> > +
> > +    return ret;
> > +}
> > +
> > +#define OFFSET(x) offsetof(Context, x)
> > +#define D AV_OPT_FLAG_DECODING_PARAM
> > +
> > +static const AVOption options[] = {
> > +    {"gateway", "The gateway to ask for IPFS data.", OFFSET(gateway),
> AV_OPT_TYPE_BINARY, .flags = D },
> > +    {NULL},
> > +};
> > +
> > +static const AVClass ipfs_context_class = {
> > +    .class_name = "IPFS",
> > +    .item_name  = av_default_item_name,
> > +    .option     = options,
> > +    .version    = LIBAVUTIL_VERSION_INT,
> > +};
> > +
> > +const URLProtocol ff_ipfs_protocol = {
> > +    .name                = "ipfs",
> > +    .url_open2           = ipfs_open,
> > +    .url_read            = ipfs_read,
> > +    .url_seek            = ipfs_seek,
> > +    .url_close           = ipfs_close,
> > +    .priv_data_size      = sizeof(Context),
> > +    .priv_data_class     = &ipfs_context_class,
> > +};
> > +
> > +const URLProtocol ff_infs_protocol = {
> > +    .name                = "ipns",
> > +    .url_open2           = ipfs_open,
> > +    .url_read            = ipfs_read,
> > +    .url_seek            = ipfs_seek,
> > +    .url_close           = ipfs_close,
> > +    .priv_data_size      = sizeof(Context),
> > +    .priv_data_class     = &ipfs_context_class,
> > +};
> > diff --git a/libavformat/protocols.c b/libavformat/protocols.c
> > index 948fae411f..675b684bd3 100644
> > --- a/libavformat/protocols.c
> > +++ b/libavformat/protocols.c
> > @@ -73,6 +73,8 @@ extern const URLProtocol ff_libsrt_protocol;
> >   extern const URLProtocol ff_libssh_protocol;
> >   extern const URLProtocol ff_libsmbclient_protocol;
> >   extern const URLProtocol ff_libzmq_protocol;
> > +extern const URLProtocol ff_ipfs_protocol;
> > +extern const URLProtocol ff_ipns_protocol;
> >
> >   #include "libavformat/protocol_list.c"
> >
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/5] Add IPFS and IPNS protocol support

[Tomas Härdin]

mån 2022-01-31 klockan 17:31 +0100 skrev Mark Gaiser:
> On Mon, Jan 31, 2022 at 4:52 PM Tomas Härdin <tjoppen@acc.umu.se>
> wrote:
> 
> > mån 2022-01-31 klockan 14:51 +0100 skrev Mark Gaiser:
> > > 
> > > There are multiple ways to access files on the IPFS network. This
> > > patch series
> > > uses the gateway driven way. An IPFS node - by default - exposes
> > > a
> > > local
> > > gateway (say http://localhost:8080) which is then used to get
> > > content
> > > from IPFS.
> > 
> > 
> > Perhaps the protocol should be called something other than just
> > ipfs if
> > it doesn't actually implement IPFS. Like ipfsgateway. It could
> > still be
> > registered to ipfs:// of course, until someone writes a wrapper for
> > libipfs.
> > 
> 
> Do you mean to have it named like "ipfsgateway" as files (and
> library) but
> keep the protocol registration of ipfs and ipns?
> I'm fine with that. The name is only artificial in code anyhow, all
> that
> matters are the protocol names.

What I'm really after is if other devs think there might be an issue
once someone goes an implements proper IPFS support

It strikes me that this borders on incorporating business logic within
lavf. A user could achieve the same thing with a small shell script.
For example adding an alias that inspects calls to ffmpeg and sed:s
ipfs:// URLs accordingly

> 
> Question though. In a V2 patch, would it make sense to squash
> everything in
> one commit? 

Just look at what other patch series look like. This is small enough
that a single patch is likely enough yes

/Tomas

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/5] Early version of IPFS protocol support.

[Lynne]

Jan 31, 2022, 14:51 by markg85@gmail.com:

> Signed-off-by: Mark Gaiser <markg85@gmail.com>
> ---
>  configure               |   1 +
>  doc/protocols.texi      |  30 ++++++
>  libavformat/Makefile    |   1 +
>  libavformat/ipfs.c      | 202 ++++++++++++++++++++++++++++++++++++++++
>  libavformat/protocols.c |   2 +
>  5 files changed, 236 insertions(+)
>  create mode 100644 libavformat/ipfs.c
>
> +
> +static int ipfs_open(URLContext *h, const char *uri, int flags, AVDictionary **options)
> +{
> +    const char *gatewaysuffix;
> +    int ret = 0;
> +    Context *c = h->priv_data;
> + 
> +    if (!av_strstart(uri, "ipfs://", &gatewaysuffix) &&
> +        !av_strstart(uri, "ipfs:", &gatewaysuffix)) {
> +        av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri);
> +        ret = AVERROR(EINVAL);
> +        goto err;
> +    }
> + 
> +    char* ipfs_gateway = "https://ipfs.io/ipfs/";
>

That's a no from me. I'd rather have native support rather
than depend on some third party service. Users can just convert
the link themselves if they want to. Surely the IPFS project
has libraries one could use instead.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/5] Add IPFS and IPNS protocol support

[Mark Gaiser]

On Mon, Jan 31, 2022 at 9:23 PM Tomas Härdin <tjoppen@acc.umu.se> wrote:

> mån 2022-01-31 klockan 17:31 +0100 skrev Mark Gaiser:
> > On Mon, Jan 31, 2022 at 4:52 PM Tomas Härdin <tjoppen@acc.umu.se>
> > wrote:
> >
> > > mån 2022-01-31 klockan 14:51 +0100 skrev Mark Gaiser:
> > > >
> > > > There are multiple ways to access files on the IPFS network. This
> > > > patch series
> > > > uses the gateway driven way. An IPFS node - by default - exposes
> > > > a
> > > > local
> > > > gateway (say http://localhost:8080) which is then used to get
> > > > content
> > > > from IPFS.
> > >
> > >
> > > Perhaps the protocol should be called something other than just
> > > ipfs if
> > > it doesn't actually implement IPFS. Like ipfsgateway. It could
> > > still be
> > > registered to ipfs:// of course, until someone writes a wrapper for
> > > libipfs.
> > >
> >
> > Do you mean to have it named like "ipfsgateway" as files (and
> > library) but
> > keep the protocol registration of ipfs and ipns?
> > I'm fine with that. The name is only artificial in code anyhow, all
> > that
> > matters are the protocol names.
>
> What I'm really after is if other devs think there might be an issue
> once someone goes an implements proper IPFS support
>

A "proper" implementation is unfeasible for ffmpeg purposes because a
proper implementation would act as an IPFS node.
That means it would:
- spin up
- do it's bootstrapping
- connect to nodes and find new nodes to connect to
- find the CID on the network
- etc...

This all adds a lot of startup time making it very unfriendly to users.
In this scenario it could take up to minutes before your video starts
playing if it doesn't time out.

Rather, the gateway approach uses an already running IPFS instance.
Users are meant to run a local IPFS instance but it doesn't *have* to be
local.
Sure, a user could point it to another local network hosted IPFS node or
some online public gateway.
But the idea is for users to host a node themselves.


>
> It strikes me that this borders on incorporating business logic within
> lavf. A user could achieve the same thing with a small shell script.
> For example adding an alias that inspects calls to ffmpeg and sed:s
> ipfs:// URLs accordingly
>

That might work but isn't really user friendly. It also doesn't help for
tools/applications that incorporate ffmpeg to potentially use IPFS
resources.
KODI (when IPFS is merged into ffmpeg) is one such application where I'll
be adding support for IPFS.
But there are more that could potentially benefit. Think for example of OBS
studio and blender (i haven't been in contact with them, all i know is that
they use ffmpeg).

>
> >
> > Question though. In a V2 patch, would it make sense to squash
> > everything in
> > one commit?
>
> Just look at what other patch series look like. This is small enough
> that a single patch is likely enough yes
>
> /Tomas
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/5] Early version of IPFS protocol support.

[Mark Gaiser]

On Mon, Jan 31, 2022 at 9:26 PM Lynne <dev@lynne.ee> wrote:

> Jan 31, 2022, 14:51 by markg85@gmail.com:
>
> > Signed-off-by: Mark Gaiser <markg85@gmail.com>
> > ---
> >  configure               |   1 +
> >  doc/protocols.texi      |  30 ++++++
> >  libavformat/Makefile    |   1 +
> >  libavformat/ipfs.c      | 202 ++++++++++++++++++++++++++++++++++++++++
> >  libavformat/protocols.c |   2 +
> >  5 files changed, 236 insertions(+)
> >  create mode 100644 libavformat/ipfs.c
> >
> > +
> > +static int ipfs_open(URLContext *h, const char *uri, int flags,
> AVDictionary **options)
> > +{
> > +    const char *gatewaysuffix;
> > +    int ret = 0;
> > +    Context *c = h->priv_data;
> > +
> > +    if (!av_strstart(uri, "ipfs://", &gatewaysuffix) &&
> > +        !av_strstart(uri, "ipfs:", &gatewaysuffix)) {
> > +        av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri);
> > +        ret = AVERROR(EINVAL);
> > +        goto err;
> > +    }
> > +
> > +    char* ipfs_gateway = "https://ipfs.io/ipfs/";
> >
>
> That's a no from me. I'd rather have native support rather
> than depend on some third party service. Users can just convert
> the link themselves if they want to. Surely the IPFS project
> has libraries one could use instead.
>

I'm sorry, that part isn't in the final version.
I had sent the patches as I made the implementation. This kinda made this
part (and a couple others) weird because a later patch gets rid of it.

I'll send the v2 revision as a single patch which will give a clear picture
of the changes. No fixed url is in it.

Just a little oops from a first time contributor :)


> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/5] Add IPFS and IPNS protocol support

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 822 bytes --]

On Mon, Jan 31, 2022 at 09:22:52PM +0100, Tomas Härdin wrote:
[...]
> It strikes me that this borders on incorporating business logic within
> lavf. A user could achieve the same thing with a small shell script.
> For example adding an alias that inspects calls to ffmpeg and sed:s
> ipfs:// URLs accordingly

That sounds like a security nightmare
the ffmpeg command line is complex, there are filters, strings for thinsg like
drawtext, there is metadata possibly and links can occur at different places
some of this may be controlled by a user (= attacker)
seperating the actual links out to modify them and just them is no easy feat

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

There will always be a question for which you do not know the correct answer.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/5] Add IPFS and IPNS protocol support

[Tomas Härdin]

mån 2022-01-31 klockan 23:00 +0100 skrev Mark Gaiser:
> On Mon, Jan 31, 2022 at 9:23 PM Tomas Härdin <tjoppen@acc.umu.se>
> wrote:
> 
> > mån 2022-01-31 klockan 17:31 +0100 skrev Mark Gaiser:
> > > On Mon, Jan 31, 2022 at 4:52 PM Tomas Härdin <tjoppen@acc.umu.se>
> > > wrote:
> > > 
> > > > mån 2022-01-31 klockan 14:51 +0100 skrev Mark Gaiser:
> > > > > 
> > > > > There are multiple ways to access files on the IPFS network.
> > > > > This
> > > > > patch series
> > > > > uses the gateway driven way. An IPFS node - by default -
> > > > > exposes
> > > > > a
> > > > > local
> > > > > gateway (say http://localhost:8080) which is then used to get
> > > > > content
> > > > > from IPFS.
> > > > 
> > > > 
> > > > Perhaps the protocol should be called something other than just
> > > > ipfs if
> > > > it doesn't actually implement IPFS. Like ipfsgateway. It could
> > > > still be
> > > > registered to ipfs:// of course, until someone writes a wrapper
> > > > for
> > > > libipfs.
> > > > 
> > > 
> > > Do you mean to have it named like "ipfsgateway" as files (and
> > > library) but
> > > keep the protocol registration of ipfs and ipns?
> > > I'm fine with that. The name is only artificial in code anyhow,
> > > all
> > > that
> > > matters are the protocol names.
> > 
> > What I'm really after is if other devs think there might be an
> > issue
> > once someone goes an implements proper IPFS support
> > 
> 
> A "proper" implementation is unfeasible for ffmpeg purposes because a
> proper implementation would act as an IPFS node.
> That means it would:
> - spin up
> - do it's bootstrapping
> - connect to nodes and find new nodes to connect to
> - find the CID on the network
> - etc...

This sounds similar to Tor, except Tor has a much more elegant way of
dealing with this stuff by just wrapping ffmpeg with torify. It takes
care of all the IPC stuff with tor.service

> > 
> > It strikes me that this borders on incorporating business logic
> > within
> > lavf. A user could achieve the same thing with a small shell
> > script.
> > For example adding an alias that inspects calls to ffmpeg and sed:s
> > ipfs:// URLs accordingly
> > 
> 
> That might work but isn't really user friendly. It also doesn't help
> for
> tools/applications that incorporate ffmpeg to potentially use IPFS
> resources.
> KODI (when IPFS is merged into ffmpeg) is one such application where
> I'll
> be adding support for IPFS.
> But there are more that could potentially benefit. Think for example
> of OBS
> studio and blender (i haven't been in contact with them, all i know
> is that
> they use ffmpeg).

This sounds like business logic that should live in Kodi, OBS and
Blender respectively. Or better yet, in the operating system itself
(read: systemd). How does Kodi handle browsing media stored in IPFS in
this case? Does it not already have to be IPFS aware for this to work
reasonably well?

Come to think of it, why aren't URLs openable as files on Linux?

/Tomas

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/5] Add IPFS and IPNS protocol support

[Tomas Härdin]

tis 2022-02-01 klockan 11:06 +0100 skrev Michael Niedermayer:
> On Mon, Jan 31, 2022 at 09:22:52PM +0100, Tomas Härdin wrote:
> [...]
> > It strikes me that this borders on incorporating business logic
> > within
> > lavf. A user could achieve the same thing with a small shell
> > script.
> > For example adding an alias that inspects calls to ffmpeg and sed:s
> > ipfs:// URLs accordingly
> 
> That sounds like a security nightmare

Parsing shit in C is a far bigger nightmare I can assure you. The
command line can leverage sed and the regex in the URL RFC.

/Tomas

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/5] Add IPFS and IPNS protocol support

[Mark Gaiser]

On Tue, Feb 1, 2022 at 5:40 PM Tomas Härdin <tjoppen@acc.umu.se> wrote:

> mån 2022-01-31 klockan 23:00 +0100 skrev Mark Gaiser:
> > On Mon, Jan 31, 2022 at 9:23 PM Tomas Härdin <tjoppen@acc.umu.se>
> > wrote:
> >
> > > mån 2022-01-31 klockan 17:31 +0100 skrev Mark Gaiser:
> > > > On Mon, Jan 31, 2022 at 4:52 PM Tomas Härdin <tjoppen@acc.umu.se>
> > > > wrote:
> > > >
> > > > > mån 2022-01-31 klockan 14:51 +0100 skrev Mark Gaiser:
> > > > > >
> > > > > > There are multiple ways to access files on the IPFS network.
> > > > > > This
> > > > > > patch series
> > > > > > uses the gateway driven way. An IPFS node - by default -
> > > > > > exposes
> > > > > > a
> > > > > > local
> > > > > > gateway (say http://localhost:8080) which is then used to get
> > > > > > content
> > > > > > from IPFS.
> > > > >
> > > > >
> > > > > Perhaps the protocol should be called something other than just
> > > > > ipfs if
> > > > > it doesn't actually implement IPFS. Like ipfsgateway. It could
> > > > > still be
> > > > > registered to ipfs:// of course, until someone writes a wrapper
> > > > > for
> > > > > libipfs.
> > > > >
> > > >
> > > > Do you mean to have it named like "ipfsgateway" as files (and
> > > > library) but
> > > > keep the protocol registration of ipfs and ipns?
> > > > I'm fine with that. The name is only artificial in code anyhow,
> > > > all
> > > > that
> > > > matters are the protocol names.
> > >
> > > What I'm really after is if other devs think there might be an
> > > issue
> > > once someone goes an implements proper IPFS support
> > >
> >
> > A "proper" implementation is unfeasible for ffmpeg purposes because a
> > proper implementation would act as an IPFS node.
> > That means it would:
> > - spin up
> > - do it's bootstrapping
> > - connect to nodes and find new nodes to connect to
> > - find the CID on the network
> > - etc...
>
> This sounds similar to Tor, except Tor has a much more elegant way of
> dealing with this stuff by just wrapping ffmpeg with torify. It takes
> care of all the IPC stuff with tor.service
>
> > >
> > > It strikes me that this borders on incorporating business logic
> > > within
> > > lavf. A user could achieve the same thing with a small shell
> > > script.
> > > For example adding an alias that inspects calls to ffmpeg and sed:s
> > > ipfs:// URLs accordingly
> > >
> >
> > That might work but isn't really user friendly. It also doesn't help
> > for
> > tools/applications that incorporate ffmpeg to potentially use IPFS
> > resources.
> > KODI (when IPFS is merged into ffmpeg) is one such application where
> > I'll
> > be adding support for IPFS.
> > But there are more that could potentially benefit. Think for example
> > of OBS
> > studio and blender (i haven't been in contact with them, all i know
> > is that
> > they use ffmpeg).
>
> This sounds like business logic that should live in Kodi, OBS and
> Blender respectively. Or better yet, in the operating system itself
> (read: systemd). How does Kodi handle browsing media stored in IPFS in
> this case? Does it not already have to be IPFS aware for this to work
> reasonably well?
>

I don't entirely agree that this logic should live in kodi. I think ffmpeg
should provide the base essentials (which this patch is doing).
Kodi and other applications should then provide an input field or whatever
to potentially say "i know the gateway you need to use, pick this one".
They would then either pass "-gateway <url>" with the setting they deem
correct or by filling in the $IPFS_GATEWAY environment variable.

Kodi currently has no way to play IPFS data.
Unless one goes through the hassle of translating it manually to a gateway
url.

To give you an idea of how it looks. Kodi has so called strm (stream)
files. They can contain an url that can be played.

Without this patch an ipfs file would look like this:
http://localhost:8080/ipfs/QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T

With this patch it becomes possible to patch kodi to accept:
ipfs://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T

In the former case it's gateway specific. In the latter case it's gateway
agnostic.

The gateway specific way has a problem. If i translate it to a gateway then
that url i picked becomes the point to access the file.
If that url goes down, the file isn't playable. Even if the file might
still be online just fine on the IPFS network.
Imagine you get a file from me and have ipfs running locally. I'm guessing
you don't like to edit the file you received to fix the gateway part of the
url, do you? I certainly don't.

The idea here is that a user sets the gateway that needs to be used to
play. The ideal place to do that is ffmpeg because the gateway logic can
then be in one application. Others (kodi, vlc, ...) can either rely on
ffmpeg doing the right thing or overrule the gateway with their own
setting. In either case, ffmpeg needs the option to set the gateway. This
all changes if the logic can't go in ffmpeg. Then each application gets its
own configuration to set the ipfs gateway. That's a situation I actively
try to prevent because it makes accessing files on ipfs just more
complicated then need be.

Also, please forget the systemd and linux specifics here. ffmpeg isn't
linux only neither is ipfs.

Anyhow, V2 of this patch series will be here shortly.


> Come to think of it, why aren't URLs openable as files on Linux?
>
> /Tomas
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/5] Add IPFS and IPNS protocol support

[Tomas Härdin]

tis 2022-02-01 klockan 22:18 +0100 skrev Mark Gaiser:
> 
> To give you an idea of how it looks. Kodi has so called strm (stream)
> files. They can contain an url that can be played.
> 
> Without this patch an ipfs file would look like this:
>  
> http://localhost:8080/ipfs/QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> 
> With this patch it becomes possible to patch kodi to accept:
> ipfs://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> 
> In the former case it's gateway specific. In the latter case it's
> gateway
> agnostic.
> 
> The gateway specific way has a problem. If i translate it to a
> gateway then
> that url i picked becomes the point to access the file.
> If that url goes down, the file isn't playable. Even if the file
> might
> still be online just fine on the IPFS network.
> Imagine you get a file from me and have ipfs running locally. I'm
> guessing
> you don't like to edit the file you received to fix the gateway part
> of the
> url, do you? I certainly don't.

You translate the URLs on the fly of course, and don't store the
translated URLs in the strm. I can almost guarantee you you will need
to do this inside Kodi anyway. What if you want to play a playlist
stored in IPFS?

/Tomas

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/5] Add IPFS and IPNS protocol support

[Mark Gaiser]

On Wed, Feb 2, 2022 at 1:52 PM Tomas Härdin <tjoppen@acc.umu.se> wrote:

> tis 2022-02-01 klockan 22:18 +0100 skrev Mark Gaiser:
> >
> > To give you an idea of how it looks. Kodi has so called strm (stream)
> > files. They can contain an url that can be played.
> >
> > Without this patch an ipfs file would look like this:
> >
> >
> http://localhost:8080/ipfs/QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> >
> > With this patch it becomes possible to patch kodi to accept:
> > ipfs://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> >
> > In the former case it's gateway specific. In the latter case it's
> > gateway
> > agnostic.
> >
> > The gateway specific way has a problem. If i translate it to a
> > gateway then
> > that url i picked becomes the point to access the file.
> > If that url goes down, the file isn't playable. Even if the file
> > might
> > still be online just fine on the IPFS network.
> > Imagine you get a file from me and have ipfs running locally. I'm
> > guessing
> > you don't like to edit the file you received to fix the gateway part
> > of the
> > url, do you? I certainly don't.
>
> You translate the URLs on the fly of course, and don't store the
> translated URLs in the strm. I can almost guarantee you you will need
> to do this inside Kodi anyway. What if you want to play a playlist
> stored in IPFS?
>

I can guarantee Kodi won't need to do that translation if it's in ffmpeg.
This is because kodi passes media on to ffmpeg. If ipfs and ipns are in
it's whitelisted protocols then ffmpeg is relied upon to handle it.
It's not "exactly" working like that (there are many layers there) but it's
roughly how it works.

We are getting out of the ffmpeg scope here, but here's how I envision it
to work in kodi.
I'm quite sure it's going to work this way because I'm making the patches
for it :)

How I envision it is as follows:
1. Kodi will, somewhere in it's settings, get an option to define an IPFS
gateway. If the user specifies this, it's used.
2. ffmpeg will just honor what was set (kodi will likely pass -gateway
<url>) or will try to auto detect if nothing is given.
3. strm files will be modified to work with ipfs://<cid> url's
4. playlist files. These are tricky as they are not specific to kodi. I
will TRY to let those work if they contain ipfs://<cid>

The playlist case is a bit tricky as m3u8 (i assume you mean that) isn't a
kodi specific thing. strm is.
The eventual endgoal with regards to kodi is for ipfs:// to be a
first-class citizen and to just work. But there are lots of places in kodi
that i'll need to adjust for that to become a reality.
The first step is to get it working in the strm file and I'll go from there.



>
> /Tomas
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/5] Add IPFS and IPNS protocol support

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 2150 bytes --]

On Tue, Feb 01, 2022 at 05:43:24PM +0100, Tomas Härdin wrote:
> tis 2022-02-01 klockan 11:06 +0100 skrev Michael Niedermayer:
> > On Mon, Jan 31, 2022 at 09:22:52PM +0100, Tomas Härdin wrote:
> > [...]
> > > It strikes me that this borders on incorporating business logic
> > > within
> > > lavf. A user could achieve the same thing with a small shell
> > > script.
> > > For example adding an alias that inspects calls to ffmpeg and sed:s
> > > ipfs:// URLs accordingly
> > 
> > That sounds like a security nightmare
> 
> Parsing shit in C is a far bigger nightmare I can assure you. The
> command line can leverage sed and the regex in the URL RFC.

the problem is if you parse it on the command line and change it
before passing it to ffmpeg. You really need to have ffmpeg and
the command line parse it 100% exactly the same. If theres a
difference you can introduce security issues. because things in the
commend line that are not intended to be changed by an attacker
could become changeable by parser differences, this involves also
issues with artgument seperators being parsed or not parsed as urls

for example a filter argument could be a URL or it could be a generic
string the command line wraper would have to know this difference.
metadata and drawtext will behave interresting if they display
ipfs: and keep in mind on top here the : is a argument seperator
for filters so a real url ipfs link as filter argument would have the :
escaped in some way. The command line tool would need to fully
handle all escaping and unescping we use in every argument and it would
need to have a list of what holds an url and what doesnt.
Noone will implement this in a 100% correct form with sed and regex
anyone trying will long before run naked through the streets and flap
her/his hands as if (s)he is a chicken
feel free to proof me wrong of course 

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

In fact, the RIAA has been known to suggest that students drop out
of college or go to community college in order to be able to afford
settlements. -- The RIAA

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/5] Add IPFS and IPNS protocol support

[Tomas Härdin]

ons 2022-02-02 klockan 14:48 +0100 skrev Michael Niedermayer:
> On Tue, Feb 01, 2022 at 05:43:24PM +0100, Tomas Härdin wrote:
> > tis 2022-02-01 klockan 11:06 +0100 skrev Michael Niedermayer:
> > > On Mon, Jan 31, 2022 at 09:22:52PM +0100, Tomas Härdin wrote:
> > > [...]
> > > > It strikes me that this borders on incorporating business logic
> > > > within
> > > > lavf. A user could achieve the same thing with a small shell
> > > > script.
> > > > For example adding an alias that inspects calls to ffmpeg and
> > > > sed:s
> > > > ipfs:// URLs accordingly
> > > 
> > > That sounds like a security nightmare
> > 
> > Parsing shit in C is a far bigger nightmare I can assure you. The
> > command line can leverage sed and the regex in the URL RFC.
> 
> the problem is if you parse it on the command line and change it
> before passing it to ffmpeg. You really need to have ffmpeg and
> the command line parse it 100% exactly the same.

The word you're looking for is langsec. It is why I harp on about being
very strict with what we accept.

/Tomas

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".