From patchwork Tue Feb 1 21:58:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Gaiser X-Patchwork-Id: 34053 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2c4e:0:0:0:0 with SMTP id x14csp1006785iov; Tue, 1 Feb 2022 13:59:51 -0800 (PST) X-Google-Smtp-Source: ABdhPJxTf8hWx1X4zncUIW4VRTs7fULjTOjpaHHpGxjzRphPKI2S26SdOvBLGm22ZLf8lYaXftbp X-Received: by 2002:a17:906:4695:: with SMTP id a21mr5604506ejr.709.1643752791528; Tue, 01 Feb 2022 13:59:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643752791; cv=none; d=google.com; s=arc-20160816; b=XRTUxR2NkvVea7Y3BEMRzAsEPDcNPmyZXEhszVuZ48rnmKD+UYH2fGZfvmHP7gydIx mR3lbiegwWUmcbZjfeWlds42q7YDcHPcNhoLyr4yIiX4gpCB22/sepGO92cBaFqCrp5m iB08VW9/IVkSfyQrDbQtHJPDpKldtTpu60CkpJyzjM95BEIim5HDl+c7PeVlniRbfS/P fj3moNPBM9x7Jbt1wsYgNgqsf82L/2olczuYScsuufMb2B1VCukNdZdkk79TbSofNwBt dM/LPcgH+qMNqBUoQ6J8QEpYtNEtswgnzxo6SVX+Fnz6bt7KqcezttL1ris8uiy4uknD D8fg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=lqIn2Yozt1E63aJvXRKh9cWX5BX6NXOqqdKS5R7LaBw=; b=J0p8kwcaTxfPrposdNFXB4ZGT0KNxgRtW84ZB7KOR88yiNE+jlHnLniy1DJRLp7HFF p8SnyB33a/3QWlKUFMMW2syrcd1qr8N7an8SefskYjrVU59HId6IHAiJ4nbAi0dQC4Wh t3YR6ePTcrg2p7jyHqbJ7JCNNsKu61kb1uLEo5buS7w4J2Ft6pBIEXqEEGluBF89sV4e CQ/XiQohvRqPERk9aCIIyt5tfsG11T8auyK0Xskt+T0NO7Fj2qz9Nid5MtyOPSJwg8yi s9KFh+gKLrBeSr0qXB+EcM1GqMgldZyBDzA32CaTKYaL1u4UxDs3GoIGGY9AH/T+o666 MhbA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=fDipC15g; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ej25si9487490edb.612.2022.02.01.13.59.50; Tue, 01 Feb 2022 13:59:51 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=fDipC15g; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E722568B155; Tue, 1 Feb 2022 23:59:46 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f46.google.com (mail-ej1-f46.google.com [209.85.218.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EE7886804B9 for ; Tue, 1 Feb 2022 23:59:40 +0200 (EET) Received: by mail-ej1-f46.google.com with SMTP id d10so58548337eje.10 for ; Tue, 01 Feb 2022 13:59:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=sl32u4tqGl6RfFT3caw9l0QvjMZYSbbxDuAF6Pu/Yms=; b=fDipC15g0Vl1y6oB3Te4u2XKrEg8OTq5LvAL7FUPvknDt4m4xa42H15SQr9XPWqz+B sODlkObPLBlJWAUKr+KcTleGQqOQH8Ov651mMpyDNoMLtO+dtGlnZFi0FzIwZBnx6tv1 1kR8v/ikrCxXArLlPP/pCAbWtsxnhlI8cVjYbobJSrz/UR3zNCcYiW/dcY9+yDvoPJ64 5MI11WTc+1xBrixk2sRltJIjDmKDY11BzCiK1pj9G4ne57a0phZ5vy7p/ah6WJ4cbOFV OFaZtXIvKhOW7m/oSYN0qh9hTXzx5dbuL2B25Ad5ZuR78fNA+5FeGNJmJmXwOr1wtX/6 vBLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=sl32u4tqGl6RfFT3caw9l0QvjMZYSbbxDuAF6Pu/Yms=; b=284l2Ac/aV95NCSgyQzsPqgteOEUThxLaleYAFoihqiIFD4eHSSTxGwOxL4ZB24Ug2 0GcA6+STDSHrLS4dGEl/7LHSEZxU3Ah1TvP1D8TOe8smbMk0Z+EWWOFo2HU8OpXNSXOa 9ABV44TtWai+isfnYGsn1xH6kLiUz934aqT+wetRJ+5Vtnyhle/GXvaueiTbOSqn7btx VcZm3WhUSxrTbeCFKCetKKtSlmraFEsmxdBSGEKKRHTeOzlT89iIcr0Ikjar7ZqRDqsB T7PVK58atHQsdmoReujmIXI61DpYd07tNQ2I1LLn1hkYEJ0TtY+dEPzCHE40NvWWgwea Q7Tw== X-Gm-Message-State: AOAM532JKbMw44HGbAk1hs2qVLp2CDseylrxS6NUGrIu/T/Ym5UMhIJv XfbM8CU3EvW2leObeJNoLsrpil+hogDKqQ== X-Received: by 2002:a17:906:48d0:: with SMTP id d16mr22641329ejt.734.1643752780018; Tue, 01 Feb 2022 13:59:40 -0800 (PST) Received: from localhost.localdomain (ip60-253-211-87.adsl2.static.versatel.nl. [87.211.253.60]) by smtp.gmail.com with ESMTPSA id lt23sm15361515ejb.173.2022.02.01.13.59.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Feb 2022 13:59:39 -0800 (PST) From: Mark Gaiser To: ffmpeg-devel@ffmpeg.org Date: Tue, 1 Feb 2022 22:58:29 +0100 Message-Id: <20220201215830.65635-1-markg85@gmail.com> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 0/1] Add IPFS protocol support. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Mark Gaiser Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 3+hVa6O9LTN6 Hi, This patch series adds support for IPFS. This is v2. Everything changed and it's now 1 commit. The following is a short summary. In the IPFS ecosystem you access it's content by a "Content IDentifier" (CID). This CID is, in simplified terms, a hash of the content. IPFS itself is a distributed network where any user can run a node to be part of the network and access files by their CID. If any reachable node within that network has the CID, you can get it. IPFS (as a technology) has two protocols, ipfs and ipns. The ipfs protocol is the immutable way to access content. The ipns protocol is a mutable layer on top of it. It's essentially a new CID that points to a ipfs CID. This "pointer" if you will can be changed to point to something else. Much more information on how this technology works can be found here [1]. This patch series allows to interact natively with IPFS. That means being able to access files like: - ffplay ipfs:// - ffplay ipns:// There are multiple ways to access files on the IPFS network. This patch series uses the gateway driven way. An IPFS node - by default - exposes a local gateway (say http://localhost:8080) which is then used to get content from IPFS. The gateway functionality on the IPFS side contains optimizations to be as ideal to streaming data as it can be. Optimizations that the http protocol in ffmpeg also has and are thus reused for free in this approach. A note on other "more appropiate" ways, as I received some feedback on that. For ffmpeg purposes the gateway approach is ideal! There is a "libipfs" but that would spin up an ipfs node with the overhead of: - bootstrapping - connecting to nodes - finding other nodes to connect too - finally finding your file This alternative approach could take minutes before a file is played. The gateway approach immediately connects to an already running node thus gives the file the fastest. Much of the logic in this patch series is to find that gateway and essentially rewrite: "ipfs://" to: "http://localhost:8080/ipfs/" Once that's found it's forwared to the protocol handler where eventually the http protocol is going to handle it. Note that it could also be https. There's enough flexibility in the implementation to allow the user to provide a gateway. There are also public https gateways which can be used just as well. After this patch is accepted, I'll work on getting IPFS supported in: - mpv (requires this ffmpeg patch) - vlc (prefers this patch but can be made to work without this patch) - kodi (requires this ffmpeg patch) Best regards, Mark Gaiser [1] https://docs.ipfs.io/concepts/ Mark Gaiser (1): avformat: Add IPFS protocol support. configure | 2 + doc/protocols.texi | 30 +++++ libavformat/Makefile | 2 + libavformat/ipfsgateway.c | 267 ++++++++++++++++++++++++++++++++++++++ libavformat/protocols.c | 2 + 5 files changed, 303 insertions(+) create mode 100644 libavformat/ipfsgateway.c