From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 55681407F0 for ; Mon, 31 Jan 2022 13:53:33 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1150868B2C9; Mon, 31 Jan 2022 15:52:51 +0200 (EET) Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id ECF7368B2A5 for ; Mon, 31 Jan 2022 15:52:40 +0200 (EET) Received: by mail-ed1-f45.google.com with SMTP id b13so27095210edn.0 for ; Mon, 31 Jan 2022 05:52:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fHOy8q97dike5Sf75OgESduAeMMM/OZo7T1z2Tj7Tb4=; b=SzMrrdarpOBiEAsnCjAPYdK3UTd6WhYEGjDZsKhRzJAiSu12uR1ZcEIGzZS3I41u15 N09KiARERKOWAGaeYR5eny4xPXh+CURYeWK1/zm7k6jGmH2sp6+aDWfKyykfapMMObIT CP/dhrKpePqXI0Xz1hrM8q4buBA7hoxV9+b20DppffcDtLUu/BAO+WEKfH8xFDWFt8Ng HHItZ+KmiSatOgSa7VKAi1Hjh520NJf8yB50nu5W79kbsRKx6LZSWlvSP2JxJX83KHqb aEVgjhHK16Fizn67YxQ95u+Hy4pdopNuYUmECHG04KlWoeab7nO48HNXFJ7CQHL46SON EN8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fHOy8q97dike5Sf75OgESduAeMMM/OZo7T1z2Tj7Tb4=; b=2DouvBnEOzgl1BHqr1MQrFmUz2le8mPxwPhT+dKlE2TTQi1dkJ5a3GJ4jwQW0E6Has Gf3LTf1asEXlUYYBek2KcvCQuYl2/ih82rLEq3MOBPKto6z8YF4tDPGym99c6SKEFHH9 i6uqQU38LAPp/pQ3YcUkCKFv6hlRcHBT0VPA9bfqjUazAzdwgbWEkZnAPvTw7rVSymaX Xf3E/Z31WEgTesOmNyAMWDasXNp/1e//K/FoeOre8OHtkKQplnc4s42oFWxS2pLApMfr 0Feh/81ckk/4VH8Bz3lN5eqecFX53297yVgGYRrR0WbKlPbdhUouMp5qh3Stp86eumYG YiDQ== X-Gm-Message-State: AOAM533lD205zxtLha4z0IImugXfNsIf4gEPq2xqhclGZgQwxH+/0Sjq Cw51zmELiQKbiPfcqh48NUOIuN8yNKeCEw== X-Google-Smtp-Source: ABdhPJzvVZ6c3u8hE1MvnR3sQCYu/FAhFEd5NhHkTnJv4J3yqBXqDqZT1aUWrr7QWG4UwwBovY6XXA== X-Received: by 2002:a05:6402:11c7:: with SMTP id j7mr20373598edw.139.1643637160105; Mon, 31 Jan 2022 05:52:40 -0800 (PST) Received: from localhost.localdomain (ip60-253-211-87.adsl2.static.versatel.nl. [87.211.253.60]) by smtp.gmail.com with ESMTPSA id l19sm5431885ejq.139.2022.01.31.05.52.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Jan 2022 05:52:39 -0800 (PST) From: Mark Gaiser To: ffmpeg-devel@ffmpeg.org Date: Mon, 31 Jan 2022 14:51:15 +0100 Message-Id: <20220131135116.14035-5-markg85@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220131135116.14035-1-markg85@gmail.com> References: <20220131135116.14035-1-markg85@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/5] Implement logic to determine the IPFS gateway. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Mark Gaiser Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Inform the user about how to set a gateway if none could be found. Signed-off-by: Mark Gaiser --- libavformat/ipfs.c | 159 +++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 147 insertions(+), 12 deletions(-) diff --git a/libavformat/ipfs.c b/libavformat/ipfs.c index 22487c6cc8..8f6f91d85d 100644 --- a/libavformat/ipfs.c +++ b/libavformat/ipfs.c @@ -44,31 +44,166 @@ typedef struct Context { char *gateway; } Context; +// A best-effort way to find the IPFS gateway. +// Only the most appropiate gateway is set. It's not actually requested (http call) to prevent +// potential slowdown in startup. A potential timeout is handled by the HTTP protocol. +// +// When done and the return value is 1, a potential IPFS Gateway is set in the gateway +// variable in the inner URLContext. +static int populate_ipfs_gateway(URLContext *h) +{ + Context *c = h->priv_data; + char *home_folder; + const char *ipfs_data_folder = "/.ipfs/"; + char *ipfs_full_data_folder; + char *ipfs_gateway_file; + struct stat st; + int stat_ret = 0; + FILE *gateway_file; + char gateway_file_data[1000]; + + // First, test if there already is a path in c->gateway. If it is then it was provided as + // cli arument and should be used. It takes precdence. + if (c->gateway != NULL) { + return 1; + } + + // Test $IPFS_GATEWAY + c->gateway = getenv("IPFS_GATEWAY"); + if (c->gateway) { + return 1; + } else { + av_log(h, AV_LOG_DEBUG, "$IPFS_GATEWAY is empty.\n"); + } + + // We need to know the IPFS folder to - eventually - read the contents of the "gateway" file which would tell us the gateway to use. + ipfs_full_data_folder = getenv("IPFS_PATH"); + + if (!ipfs_full_data_folder) { + av_log(h, AV_LOG_DEBUG, "$IPFS_PATH is empty.\n"); + + // Try via the home folder + home_folder = getenv("HOME"); + ipfs_full_data_folder = malloc(strlen(home_folder) + strlen(ipfs_data_folder) + 1); + strcpy(ipfs_full_data_folder, home_folder); + strcat(ipfs_full_data_folder, ipfs_data_folder); + + // Stat the folder. It should exist in a default IPFS setup when run as local user. +#ifndef _WIN32 + stat_ret = stat(ipfs_full_data_folder, &st); +#else + stat_ret = win32_stat(ipfs_full_data_folder, &st); +#endif + if (stat_ret < 0) { + av_log(h, AV_LOG_DEBUG, "Unable to find IPFS folder. We tried:\n"); + av_log(h, AV_LOG_DEBUG, "- $IPFS_PATH, which was empty.\n"); + av_log(h, AV_LOG_DEBUG, "- $HOME/.ipfs (full uri: %s) which doesn't exist.\n", ipfs_full_data_folder); + return -1; + } + } + + ipfs_gateway_file = malloc(strlen(ipfs_full_data_folder) + strlen("gateway") + 1); + strcpy(ipfs_gateway_file, ipfs_full_data_folder); + strcat(ipfs_gateway_file, "gateway"); + + // Stat the gateway file. If it doesn't exist we have no gateway. If it does, we might have one. +#ifndef _WIN32 + stat_ret = stat(ipfs_gateway_file, &st); +#else + stat_ret = win32_stat(ipfs_gateway_file, &st); +#endif + if (stat_ret < 0) { + av_log(h, AV_LOG_ERROR, "The IPFS gateway file (full uri: %s) doesn't exist. Is the gateway enabled?\n", ipfs_gateway_file); + return -1; + } + + // Get the contents of the gateway file + gateway_file = av_fopen_utf8(ipfs_gateway_file, "r"); + if (!gateway_file) { + av_log(h, AV_LOG_ERROR, "Unable to open the IPFS gateway file (full uri: %s).\n", ipfs_gateway_file); + return -1; + } + + fscanf(gateway_file, "%[^\n]", gateway_file_data); + fclose(gateway_file); + + if ((st.st_size < 1) || (strlen(gateway_file_data) < 1)) { + av_log(h, AV_LOG_ERROR, "The IPFS gateway file (full uri: %s) appears to be empty. Is the gateway started?\n", ipfs_gateway_file); + return -1; + } + + // Copy the gateway url into c->gateway + c->gateway = malloc(strlen(gateway_file_data) + 1); + strcpy(c->gateway, gateway_file_data); + if (c->gateway) { + return 1; + } else { + av_log(h, AV_LOG_DEBUG, "Unknown error in the IPFS gateway file.\n"); + } + + return -1; +} + +// For now just makes sure that the gateway ends in url we expect. Like http://localhost:8080/. +// Explicitly with the traling slash. +static void sanitize_ipfs_gateway(URLContext *h) +{ + Context *c = h->priv_data; + const char last_gateway_char = c->gateway[strlen(c->gateway) - 1]; + char *sanitized_gateway = c->gateway; + + if (last_gateway_char != '/') { + sanitized_gateway = malloc(strlen(c->gateway) + 2); + strcpy(sanitized_gateway, c->gateway); + strcat(sanitized_gateway, "/"); + } + + c->gateway = sanitized_gateway; +} + static int ipfs_open(URLContext *h, const char *uri, int flags, AVDictionary **options) { - const char *gatewaysuffix; - const char *protocolPathSuffix = "ipfs/"; + const char *gateway_suffix; + const char *protocol_path_suffix = "ipfs/"; int ret = 0; Context *c = h->priv_data; - int isIpfs = (av_strstart(uri, "ipfs://", &gatewaysuffix) || av_strstart(uri, "ipfs:", &gatewaysuffix)); - int isIpns = (av_strstart(uri, "ipns://", &gatewaysuffix) || av_strstart(uri, "ipns:", &gatewaysuffix)); - - if (!isIpfs && !isIpns) { + int is_ipfs = (av_strstart(uri, "ipfs://", &gateway_suffix) || av_strstart(uri, "ipfs:", &gateway_suffix)); + int is_ipns = (av_strstart(uri, "ipns://", &gateway_suffix) || av_strstart(uri, "ipns:", &gateway_suffix)); + + // Populate the IPFS gateway if we have any. + // If not, inform the user how to properly set one. + if (populate_ipfs_gateway(h) < 0) { + av_log(h, AV_LOG_ERROR, "No IPFS gateway was set. Make sure a local IPFS instance is running.\n"); + av_log(h, AV_LOG_INFO, "There are multiple options to define this gateway. The below options are in order of precedence:\n"); + av_log(h, AV_LOG_INFO, "1. Define a -gateway to the gateway without trailing forward slash.\n"); + av_log(h, AV_LOG_INFO, "2. Define $IPFS_GATEWAY with the full http link to the gateway without trailing forward slash.\n"); + av_log(h, AV_LOG_INFO, "3. Define $IPFS_PATH and point it to the IPFS data path.\n"); + av_log(h, AV_LOG_INFO, "4. Have IPFS running in your local user folder (under $HOME/.ipfs).\n"); + av_log(h, AV_LOG_INFO, "In all path cases, a file named gateway is expected. See https://github.com/ipfs/specs/issues/261 for more information.\n"); + ret = AVERROR(EINVAL); + goto err; + } + + // Sanitize the gateway to a format we expect. + sanitize_ipfs_gateway(h); + + // We must have either ipns or ipfs. + if (!is_ipfs && !is_ipns) { ret = AVERROR(EINVAL); av_log(h, AV_LOG_ERROR, "Unsupported url %s\n", uri); goto err; } // If we have IPNS, update the protocol - if (isIpns) { - protocolPathSuffix = "ipns/"; + if (is_ipns) { + protocol_path_suffix = "ipns/"; } // Concatenate the url. This ends up with something like: http://localhost:8080/ipfs/Qm..... - c->fulluri = malloc(strlen(c->gateway) + strlen(protocolPathSuffix) + strlen(gatewaysuffix) + 1); + c->fulluri = malloc(strlen(c->gateway) + strlen(protocol_path_suffix) + strlen(gateway_suffix) + 1); strcpy(c->fulluri, c->gateway); - strcat(c->fulluri, protocolPathSuffix); - strcat(c->fulluri, gatewaysuffix); + strcat(c->fulluri, protocol_path_suffix); + strcat(c->fulluri, gateway_suffix); // Pass the URL back to FFMpeg's protocol handler. if ((ret = ffurl_open_whitelist(&c->inner, c->fulluri, flags, @@ -115,7 +250,7 @@ static int ipfs_close(URLContext *h) #define OFFSET(x) offsetof(Context, x) static const AVOption options[] = { - {"gateway", "The gateway to ask for IPFS data.", OFFSET(gateway), AV_OPT_TYPE_STRING, {.str = "http://localhost:8080/"}, 0, 0, AV_OPT_FLAG_DECODING_PARAM }, + {"gateway", "The gateway to ask for IPFS data.", OFFSET(gateway), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, AV_OPT_FLAG_DECODING_PARAM }, {NULL}, }; -- 2.35.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".