mbox

[FFmpeg-devel,v2,0/1] Add IPFS protocol support.

Message ID 20220203172507.21121-1-markg85@gmail.com
Headers show

Message

Mark Gaiser Feb. 3, 2022, 5:25 p.m. UTC 
Hi,

This patch series adds support for IPFS.
V3:
- A lot of style changes
- Made url checks a lot more strict
- av_asprintf leak fixes
- So many changes that a diff to v2 is again not sensible.
V2:
- Squashed and changed so much that a diff to v1 was not sensible.

The following is a short summary. In the IPFS ecosystem you access it's content
by a "Content IDentifier" (CID). This CID is, in simplified terms, a hash of 
the content. IPFS itself is a distributed network where any user can run a node
to be part of the network and access files by their CID. If any reachable node 
within that network has the CID, you can get it.

IPFS (as a technology) has two protocols, ipfs and ipns.
The ipfs protocol is the immutable way to access content.
The ipns protocol is a mutable layer on top of it. It's essentially a new CID 
that points to a ipfs CID. This "pointer" if you will can be changed to point 
to something else.
Much more information on how this technology works can be found here [1].

This patch series allows to interact natively with IPFS. That means being able
to access files like:
- ffplay ipfs://<cid>
- ffplay ipns://<cid>

There are multiple ways to access files on the IPFS network. This patch series
uses the gateway driven way. An IPFS node - by default - exposes a local 
gateway (say http://localhost:8080) which is then used to get content from IPFS.
The gateway functionality on the IPFS side contains optimizations to
be as ideal to streaming data as it can be. Optimizations that the http protocol
in ffmpeg also has and are thus reused for free in this approach.

A note on other "more appropiate" ways, as I received some feedback on that.
For ffmpeg purposes the gateway approach is ideal! There is a "libipfs" but
that would spin up an ipfs node with the overhead of:
- bootstrapping
- connecting to nodes
- finding other nodes to connect too
- finally finding your file

This alternative approach could take minutes before a file is played. The
gateway approach immediately connects to an already running node thus gives
the file the fastest.

Much of the logic in this patch series is to find that gateway and essentially 
rewrite:

"ipfs://<cid>"

to:

"http://localhost:8080/ipfs/<cid>"

Once that's found it's forwared to the protocol handler where eventually the
http protocol is going to handle it. Note that it could also be https. There's 
enough flexibility in the implementation to allow the user to provide a 
gateway. There are also public https gateways which can be used just as well.

After this patch is accepted, I'll work on getting IPFS supported in:
- mpv (requires this ffmpeg patch)
- vlc (prefers this patch but can be made to work without this patch)
- kodi (requires this ffmpeg patch)

Best regards,
Mark Gaiser

[1] https://docs.ipfs.io/concepts/

Mark Gaiser (1):
  avformat: Add IPFS protocol support.

 configure                 |   2 +
 doc/protocols.texi        |  30 ++++
 libavformat/Makefile      |   2 +
 libavformat/ipfsgateway.c | 329 ++++++++++++++++++++++++++++++++++++++
 libavformat/protocols.c   |   2 +
 5 files changed, 365 insertions(+)
 create mode 100644 libavformat/ipfsgateway.c