From patchwork Thu Feb 3 17:25:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Gaiser X-Patchwork-Id: 34094 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2c4e:0:0:0:0 with SMTP id x14csp2679519iov; Thu, 3 Feb 2022 09:26:34 -0800 (PST) X-Google-Smtp-Source: ABdhPJwy8ZA2PwbO8C1ihutgFtLX/B+rNHtuU1g4QmHPcZQ5/9cfP5YShnGoloWTsLrmndUz4D24 X-Received: by 2002:a17:906:9514:: with SMTP id u20mr25753496ejx.227.1643909193968; Thu, 03 Feb 2022 09:26:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643909193; cv=none; d=google.com; s=arc-20160816; b=RDfnfP041FVm9Wq/IBeLXLiC6yY3MUVaJNqbXwb0eWcaTactuvEEGL8I8s6kL0mlC2 WQG+yWWcwMaq0k+3T6NgzxqU+Vp4xsxvqehsYH0wOfib/frlGxtbxNwemBGpM2CHhQpb /b0/aUGN8ZVeGY4kf0o4qHTnv6RIR+x2MkwG+cTR+Pwamzr1mkcmfASq3hpFRbIYQ8k3 zQUZwF7C+AqdB42HTV41F/e/CXA4dl64LWIfSQ+c2/FaCvGSx0KtDIUE5isCaIZyLi6A SQnnZYHRRJZqOfp8pWbYj9Z9zAWgAQ21/9hJ4EaJ2epQ6jRdqqnbbhZOfU6wiq/yDSHQ bKAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=i8LQHsyGJmyI56w/kQNutCp+Lp1nbm1SlhzQl8SGIYM=; b=HEdfDHZRIqXc0ico9veh9a+XRndEGvq0+ohwZD8NEY616PxoQpWVG2qY7xYT0jblkw 1D/ycHiPnn71IsMtF9/q1u8X1O3Aw3eZCVblMU/3oL7/il3LDhUQXGcHRpKzAP3PwVRU apnsK7jw6VAJ+ar5lfuIWyeYi84y09BRSKsupLzexwgLRVVftXYPx+MnGP9NmOOC1csB 5RxTwBN+C5SVBNVfC/mPhUc2+p1/7bxtxp1mZy2pXK9UZgZoPESS1KE9Fjz9MOjZ7c/H TvGVsok3u6AUUB5RbuRsR61GKVyRb+cG8ewhTI66FpJ8YfdpnXbsjQiexNycBaW3lHF9 vzaA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=hMXxfVRy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id nc13si1626196ejc.529.2022.02.03.09.26.33; Thu, 03 Feb 2022 09:26:33 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=hMXxfVRy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0CA7D68B064; Thu, 3 Feb 2022 19:26:29 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3B84F68A58C for ; Thu, 3 Feb 2022 19:26:22 +0200 (EET) Received: by mail-ej1-f48.google.com with SMTP id h7so11005509ejf.1 for ; Thu, 03 Feb 2022 09:26:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=m6DTDGPp9Ix+HUuBj7vtbdS6dxAj2/aqjdYQ0ZmPVMg=; b=hMXxfVRy33VgiyNREedo0i2Y0WSLfaFnLpuzNeB8adSgqmMGiiRvQ6ZqMQvO5fQPrn PUKo0hF6XDuaOeeZxmlBUdrv7TbsMzhebLRr42uLtsLaAYyoc+sm1R/IGpwxNsrS2vS9 4SMQhgkvSb9b+6QOMR3Pos9Ey82F1PdgovLVRH1QudFC5h9T9Fz+I6C2huTqs+sdKDfV TbWLSgEBgrbC+8iFFAP7fC1rC428m1BKO9a0i+R4iSN9W0VxHAhdQtIM/MTW5dbftB1J nbC4NYYuanJYS2CaOVn8v2Vt9FP9pSaGZ8MOj1KDrJOCy5bhwYGvwFUGk5bzkqKhEtvT JCHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=m6DTDGPp9Ix+HUuBj7vtbdS6dxAj2/aqjdYQ0ZmPVMg=; b=EHfZPiuM1ku8llf68u01d9qg0SiGsG+fCVXqoA9VScCJfUndBuFFKxTPPMSuIU4na7 rOMFg19r3j5KxNL4FhHRlT+F4QXXu3bioCu0eiiLuZtXogdV7j3iu+TqNsLk1IjXIvCr RuNbiBPZEV0jMLqF+3cftP0ZSKhBTxgOVKUHHUA62RFGBi4/CAvLzf2u5h+zcCWhkfX5 PMR1bIRrag0gYa7C0ofJkekXSc0gHpqg2R2DINRjCivVIfFqG51rE3gMEaqNrtl2oIHE 8MJsaYdMKbaKjgLUVjreHBGlu6p5rgAMcDLrnC/ozwaHBnVHz0YfM7VBtaMSkrthNbiD vHsQ== X-Gm-Message-State: AOAM533dnUi3bJYKoULJudxSkCh//7Axx+Oo1RElBDVIn8NbzNwIgOd2 9GSxhqWIu0dQemr9CH+5H+6x8IgVwX116Q== X-Received: by 2002:a17:907:8694:: with SMTP id qa20mr30602209ejc.530.1643909180468; Thu, 03 Feb 2022 09:26:20 -0800 (PST) Received: from localhost.localdomain (ip60-253-211-87.adsl2.static.versatel.nl. [87.211.253.60]) by smtp.gmail.com with ESMTPSA id d23sm16630157eja.86.2022.02.03.09.26.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Feb 2022 09:26:19 -0800 (PST) From: Mark Gaiser To: ffmpeg-devel@ffmpeg.org Date: Thu, 3 Feb 2022 18:25:06 +0100 Message-Id: <20220203172507.21121-1-markg85@gmail.com> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 0/1] Add IPFS protocol support. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Mark Gaiser Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: SkbySZ1e+V1k Hi, This patch series adds support for IPFS. V3: - A lot of style changes - Made url checks a lot more strict - av_asprintf leak fixes - So many changes that a diff to v2 is again not sensible. V2: - Squashed and changed so much that a diff to v1 was not sensible. The following is a short summary. In the IPFS ecosystem you access it's content by a "Content IDentifier" (CID). This CID is, in simplified terms, a hash of the content. IPFS itself is a distributed network where any user can run a node to be part of the network and access files by their CID. If any reachable node within that network has the CID, you can get it. IPFS (as a technology) has two protocols, ipfs and ipns. The ipfs protocol is the immutable way to access content. The ipns protocol is a mutable layer on top of it. It's essentially a new CID that points to a ipfs CID. This "pointer" if you will can be changed to point to something else. Much more information on how this technology works can be found here [1]. This patch series allows to interact natively with IPFS. That means being able to access files like: - ffplay ipfs:// - ffplay ipns:// There are multiple ways to access files on the IPFS network. This patch series uses the gateway driven way. An IPFS node - by default - exposes a local gateway (say http://localhost:8080) which is then used to get content from IPFS. The gateway functionality on the IPFS side contains optimizations to be as ideal to streaming data as it can be. Optimizations that the http protocol in ffmpeg also has and are thus reused for free in this approach. A note on other "more appropiate" ways, as I received some feedback on that. For ffmpeg purposes the gateway approach is ideal! There is a "libipfs" but that would spin up an ipfs node with the overhead of: - bootstrapping - connecting to nodes - finding other nodes to connect too - finally finding your file This alternative approach could take minutes before a file is played. The gateway approach immediately connects to an already running node thus gives the file the fastest. Much of the logic in this patch series is to find that gateway and essentially rewrite: "ipfs://" to: "http://localhost:8080/ipfs/" Once that's found it's forwared to the protocol handler where eventually the http protocol is going to handle it. Note that it could also be https. There's enough flexibility in the implementation to allow the user to provide a gateway. There are also public https gateways which can be used just as well. After this patch is accepted, I'll work on getting IPFS supported in: - mpv (requires this ffmpeg patch) - vlc (prefers this patch but can be made to work without this patch) - kodi (requires this ffmpeg patch) Best regards, Mark Gaiser [1] https://docs.ipfs.io/concepts/ Mark Gaiser (1): avformat: Add IPFS protocol support. configure | 2 + doc/protocols.texi | 30 ++++ libavformat/Makefile | 2 + libavformat/ipfsgateway.c | 329 ++++++++++++++++++++++++++++++++++++++ libavformat/protocols.c | 2 + 5 files changed, 365 insertions(+) create mode 100644 libavformat/ipfsgateway.c