From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 252CE4082D for ; Mon, 31 Jan 2022 16:36:21 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B6BC368B1D6; Mon, 31 Jan 2022 18:36:19 +0200 (EET) Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B4FBE68B0EE for ; Mon, 31 Jan 2022 18:36:13 +0200 (EET) Received: by mail-wr1-f52.google.com with SMTP id w11so26525330wra.4 for ; Mon, 31 Jan 2022 08:36:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=rKUiM6UYJE6BzpijtN1yzs76qagQDeWSsBWYaNoE9zk=; b=mFZGa1Md+pj1S/5n5S0JibbdEL0+h8nuj8P5kpzlZONvWvc/I7EhmYRg/Gw1C7rVo6 v1ProQh5TZy9pFpMYVbBtp+KI9RclLi4vbdFqGzC/4+xp373NOAFSjVYMsKhY9F+RxZr R/yCPaxuDOXPa6v7K8KW/y4aNFXEYG2Y3cC9fq26zZiDqMaISdB/nhljF9FLJ67YzMLk GMZ2IVXYZtm6A7r8gHZyk3NgHsMPZmXmFUtNwmpCv+ZeVCOp4+vTTLlIy0Kcgk1jZlhw aR3DYiEp8Grt18PdUrDQhuo+YKukSgVHV4WMJLx/bhgqsTvd6tpAQpdJvzsSaG8zAMbh uWOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=rKUiM6UYJE6BzpijtN1yzs76qagQDeWSsBWYaNoE9zk=; b=toMFHKdINjaIbegJVJxTKEnXc1EaBe18GdwVCFGylWBu/kHOT34vdHCN67XX7kEhdC 6sxym7mKss68ktW/F+35fIJ2n0QOF1XsRv6oCqdje9VWjJ42tu0N0a5pn9UgTcbi24Pm P4y2e2fh1tUy9VgFZHf/AO+kwcK7GSARyPkkvuKNmoCOojnA2PGpFWqTCzm0beNvuWMq sOrq3bl2Wx1XDMOKurfIrhroxGPNI7LMjS9M588A8/+oPuIYLGxSzyWyWywazLet1HZW /ppcbfWbW++oXJOTvRC18DOnJDv4ondJPaWYONUF3CWBGA2B9r2BOHfX4x0gpE2mdLeT ibdQ== X-Gm-Message-State: AOAM531a6dsZRS4ZGsLTJBSqGcPVXyfhapfS2BBCIEhZFqYW4M+wjVww BqAkjC0Fmhf5WiX/BN/4x5TU/A5jcwi2x7OKxU7DDObRc+A= X-Google-Smtp-Source: ABdhPJzZAY9UdbhhLUjLGz99BQXK8y5wzJDjqSftOu79dFMLyEL+WfSY8f58e7EeVhkWnZE0dsLyDjikU5qj05gd3+I= X-Received: by 2002:adf:fb0f:: with SMTP id c15mr18876156wrr.42.1643646972647; Mon, 31 Jan 2022 08:36:12 -0800 (PST) MIME-Version: 1.0 References: <20220131135116.14035-1-markg85@gmail.com> <20220131135116.14035-2-markg85@gmail.com> <0a7f2b72-ddfa-2854-b87d-def414df5b85@gmail.com> In-Reply-To: <0a7f2b72-ddfa-2854-b87d-def414df5b85@gmail.com> From: Mark Gaiser Date: Mon, 31 Jan 2022 17:34:57 +0100 Message-ID: To: FFmpeg development discussions and patches X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [FFmpeg-devel] [PATCH 1/5] Early version of IPFS protocol support. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On Mon, Jan 31, 2022 at 5:06 PM James Almer wrote: > > > On 1/31/2022 10:51 AM, Mark Gaiser wrote: > > Signed-off-by: Mark Gaiser > > --- > > configure | 1 + > > doc/protocols.texi | 30 ++++++ > > libavformat/Makefile | 1 + > > libavformat/ipfs.c | 202 ++++++++++++++++++++++++++++++++++++++++ > > libavformat/protocols.c | 2 + > > 5 files changed, 236 insertions(+) > > create mode 100644 libavformat/ipfs.c > > > > diff --git a/configure b/configure > > index 5b19a35f59..e466f924a3 100755 > > --- a/configure > > +++ b/configure > > @@ -3585,6 +3585,7 @@ udp_protocol_select="network" > > udplite_protocol_select="network" > > unix_protocol_deps="sys_un_h" > > unix_protocol_select="network" > > +ipfs_protocol_select="https_protocol" > > > > # external library protocols > > libamqp_protocol_deps="librabbitmq" > > diff --git a/doc/protocols.texi b/doc/protocols.texi > > index d207df0b52..7c9c0a4808 100644 > > --- a/doc/protocols.texi > > +++ b/doc/protocols.texi > > @@ -2025,5 +2025,35 @@ decoding errors. > > > > @end table > > > > +@section ipfs > > + > > +InterPlanetary File System (IPFS) protocol support. One can access > files stored > > +on the IPFS network through so called gateways. Those are http(s) > endpoints. > > +This protocol wraps the IPFS native protocols (ipfs:// and ipns://) to > be send > > +to such a gateway. Users can (and should) host their own node which > means this > > +protocol will use your local machine gateway to access files on the > IPFS network. > > + > > +If a user doesn't have a node of their own then the public gateway > dweb.link is > > +used by default. > > + > > +You can use this protocol in 2 ways. Using IPFS: > > +@example > > +ffplay ipfs://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T > > +@end example > > + > > +Or the IPNS protocol (IPNS is mutable IPFS): > > +@example > > +ffplay ipns://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T > > +@end example > > + > > +You can also change the gateway to be used: > > + > > +@table @option > > + > > +@item gateway > > +Defines the gateway to use. When nothing is provided the protocol will > first try > > +your local gateway. If that fails dweb.link will be used. > > + > > +@end table > > > > @c man end PROTOCOLS > > diff --git a/libavformat/Makefile b/libavformat/Makefile > > index 3dc6a479cc..983a77f4f2 100644 > > --- a/libavformat/Makefile > > +++ b/libavformat/Makefile > > @@ -656,6 +656,7 @@ OBJS-$(CONFIG_SRTP_PROTOCOL) += > srtpproto.o srtp.o > > OBJS-$(CONFIG_SUBFILE_PROTOCOL) += subfile.o > > OBJS-$(CONFIG_TEE_PROTOCOL) += teeproto.o tee_common.o > > OBJS-$(CONFIG_TCP_PROTOCOL) += tcp.o > > +OBJS-$(CONFIG_IPFS_PROTOCOL) += ipfs.o > > TLS-OBJS-$(CONFIG_GNUTLS) += tls_gnutls.o > > TLS-OBJS-$(CONFIG_LIBTLS) += tls_libtls.o > > TLS-OBJS-$(CONFIG_MBEDTLS) += tls_mbedtls.o > > diff --git a/libavformat/ipfs.c b/libavformat/ipfs.c > > new file mode 100644 > > index 0000000000..4cc65750ed > > --- /dev/null > > +++ b/libavformat/ipfs.c > > @@ -0,0 +1,202 @@ > > +/* > > + * IPFS protocol. > > + * Copyright (c) 2021 Mark Gaiser > > + * > > + * This file is part of FFmpeg. > > + * > > + * FFmpeg is free software; you can redistribute it and/or > > + * modify it under the terms of the GNU Lesser General Public > > + * License as published by the Free Software Foundation; either > > + * version 2.1 of the License, or (at your option) any later version. > > + * > > + * FFmpeg is distributed in the hope that it will be useful, > > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + * Lesser General Public License for more details. > > + * > > + * You should have received a copy of the GNU Lesser General Public > > + * License along with FFmpeg; if not, write to the Free Software > > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA > > + */ > > + > > +#include "libavutil/avassert.h" > > +#include "libavutil/avstring.h" > > +#include "libavutil/internal.h" > > +#include "libavutil/opt.h" > > +#include "libavutil/tree.h" > > +#include "avformat.h" > > +#include > > +#if HAVE_IO_H > > +#include > > +#endif > > +#if HAVE_UNISTD_H > > +#include > > +#endif > > +#include > > +#include > > +#include "os_support.h" > > +#include "url.h" > > + > > +typedef struct Context { > > + AVClass *class; > > + URLContext *inner; > > + char *fulluri; > > + char *gateway; > > +} Context; > > A more descriptive name would be nice. > > > + > > +static int ipfs_open(URLContext *h, const char *uri, int flags, > AVDictionary **options) > > +{ > > + const char *gatewaysuffix; > > + int ret = 0; > > + Context *c = h->priv_data; > > + > > + if (!av_strstart(uri, "ipfs://", &gatewaysuffix) && > > + !av_strstart(uri, "ipfs:", &gatewaysuffix)) { > > + av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri); > > + ret = AVERROR(EINVAL); > > + goto err; > > Just return ret. No need for this goto. > > > + } > > + > > + char* ipfs_gateway = "https://ipfs.io/ipfs/"; > > const char* > > > + > > + c->fulluri = malloc(strlen(ipfs_gateway)+strlen(gatewaysuffix) + 1); > > av_malloc() > > Also, check the return value for allocation failure. > > > + > > + strcpy(c->fulluri, ipfs_gateway); > > + strcat(c->fulluri, gatewaysuffix); > > + > > + if ((ret = ffurl_open_whitelist(&c->inner, c->fulluri, flags, > > Is fulluri going to be used after this call? If not, then it should be > local to this function as there's no need to have it in h->priv_data > (And for that matter, you're not freeing fulluri anywhere). > > > + &h->interrupt_callback, options, > > + h->protocol_whitelist, > h->protocol_blacklist, h)) < 0) { > > You need to define a whitelist including https in both URLProtocol below. > > > + av_log(h, AV_LOG_ERROR, "Unable to open resource: %s\n", > c->fulluri); > > + goto err; > > + } > > + > > +err: > > + return ret; > > +} > > + > > +static int ipfs_read(URLContext *h, unsigned char *buf, int size) > > +{ > > + Context *c = h->priv_data; > > + int ret; > > + > > + ret = ffurl_read(c->inner, buf, size); > > + > > + return ret; > > +} > > + > > +static int64_t ipfs_seek(URLContext *h, int64_t pos, int whence) > > +{ > > + Context *c = h->priv_data; > > + int64_t ret; > > + > > + ret = ffurl_seek(c->inner, pos, whence); > > + > > + return ret; > > +} > > + > > +static int ipfs_close(URLContext *h) > > +{ > > + Context *c = h->priv_data; > > + int ret; > > + > > + ret = ffurl_closep(&c->inner); > > + > > + return ret; > > +} > > + > > +static int ipns_open(URLContext *h, const char *uri, int flags, > AVDictionary **options) > > +{ > > + const char *gatewaysuffix; > > + int ret = 0; > > + Context *c = h->priv_data; > > + > > + if (!av_strstart(uri, "ipns://", &gatewaysuffix) && > > + !av_strstart(uri, "ipns:", &gatewaysuffix)) { > > + av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri); > > + ret = AVERROR(EINVAL); > > + goto err; > > + } > > + > > + char* ipfs_gateway = "https://ipfs.io/ipns/"; > > + > > + c->fulluri = malloc(strlen(ipfs_gateway)+strlen(gatewaysuffix) + 1); > > + > > + strcpy(c->fulluri, ipfs_gateway); > > + strcat(c->fulluri, gatewaysuffix); > > + > > + if ((ret = ffurl_open_whitelist(&c->inner, c->fulluri, flags, > > + &h->interrupt_callback, options, > > + h->protocol_whitelist, > h->protocol_blacklist, h)) < 0) { > > + av_log(h, AV_LOG_ERROR, "Unable to open resource: %s\n", > c->fulluri); > > + goto err; > > + } > > + > > +err: > > + return ret; > > Same comments as above apply here. > Awesome, thank you for your review time! Will fix all those in V2. > > > +} > > + > > +static int ipns_read(URLContext *h, unsigned char *buf, int size) > > +{ > > + Context *c = h->priv_data; > > + int ret; > > + > > + ret = ffurl_read(c->inner, buf, size); > > + > > + return ret; > > +} > > + > > +static int64_t ipns_seek(URLContext *h, int64_t pos, int whence) > > +{ > > + Context *c = h->priv_data; > > + int64_t ret; > > + > > + ret = ffurl_seek(c->inner, pos, whence); > > + > > + return ret; > > +} > > + > > +static int ipns_close(URLContext *h) > > +{ > > + Context *c = h->priv_data; > > + int ret; > > + > > + ret = ffurl_closep(&c->inner); > > + > > + return ret; > > +} > > + > > +#define OFFSET(x) offsetof(Context, x) > > +#define D AV_OPT_FLAG_DECODING_PARAM > > + > > +static const AVOption options[] = { > > + {"gateway", "The gateway to ask for IPFS data.", OFFSET(gateway), > AV_OPT_TYPE_BINARY, .flags = D }, > > + {NULL}, > > +}; > > + > > +static const AVClass ipfs_context_class = { > > + .class_name = "IPFS", > > + .item_name = av_default_item_name, > > + .option = options, > > + .version = LIBAVUTIL_VERSION_INT, > > +}; > > + > > +const URLProtocol ff_ipfs_protocol = { > > + .name = "ipfs", > > + .url_open2 = ipfs_open, > > + .url_read = ipfs_read, > > + .url_seek = ipfs_seek, > > + .url_close = ipfs_close, > > + .priv_data_size = sizeof(Context), > > + .priv_data_class = &ipfs_context_class, > > +}; > > + > > +const URLProtocol ff_infs_protocol = { > > + .name = "ipns", > > + .url_open2 = ipfs_open, > > + .url_read = ipfs_read, > > + .url_seek = ipfs_seek, > > + .url_close = ipfs_close, > > + .priv_data_size = sizeof(Context), > > + .priv_data_class = &ipfs_context_class, > > +}; > > diff --git a/libavformat/protocols.c b/libavformat/protocols.c > > index 948fae411f..675b684bd3 100644 > > --- a/libavformat/protocols.c > > +++ b/libavformat/protocols.c > > @@ -73,6 +73,8 @@ extern const URLProtocol ff_libsrt_protocol; > > extern const URLProtocol ff_libssh_protocol; > > extern const URLProtocol ff_libsmbclient_protocol; > > extern const URLProtocol ff_libzmq_protocol; > > +extern const URLProtocol ff_ipfs_protocol; > > +extern const URLProtocol ff_ipns_protocol; > > > > #include "libavformat/protocol_list.c" > > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".