[FFmpeg-devel] avformat/avio: Add Metacube support

Support wrapping the output format in Metacube, as used by the Cubemap
stream reflector (originally designed for VLC). This is nominally meant
to run over HTTP, but longer-term would probably also be useful for
pipe output as a subprocess of Cubemap.

Integrating with Cubemap instantly gives FFmpeg access to a high-performance
(40Gbit++/sec on a regular quadcore, with TLS), multi-user, proven HTTP
reflector -- FFmpeg's own HTTP server still only has experimental multi-user
support. However, Cubemap is deliberately dumb and thus does not understand any
muxes; it is dependent on the origin (e.g. FFmpeg) to mark which bytes are
headers, and at which bytes new clients can start the stream, which is why
it needs its input wrapped in Metacube.

Metacube output is activated by -fflags metacube. E.g., to create streamable
MP4 on port 9095 that Cubemap can pick up and reflect:

  ffmpeg -i <input> -f mp4 -movflags empty_moov+frag_keyframe+default_base_moof+skip_trailer \
    -frag_duration 125000 -fflags metacube -listen 1 'http://[::]:9095'

Tested with MP4 and Matroska.

The metacube2.h header comes from the Cubemap distribution, and is nominally
public domain. It can be considered to be licensed under the LGPL 2.1, like
the rest of FFmpeg.

Signed-off-by: Steinar H. Gunderson <steinar+ffmpeg@gunderson.no>
---
 fftools/ffmpeg_opt.c        |   6 ++-
 libavformat/avformat.h      |   1 +
 libavformat/avio.h          |  30 +++++++++++
 libavformat/aviobuf.c       | 103 +++++++++++++++++++++++++++++++++---
 libavformat/http.c          |  14 ++++-
 libavformat/metacube2.h     |  35 ++++++++++++
 libavformat/movenc.c        |   4 +-
 libavformat/options_table.h |   1 +
 8 files changed, 184 insertions(+), 10 deletions(-)
 create mode 100644 libavformat/metacube2.h

On Mon, 3 May 2021, Steinar H. Gunderson wrote:

> Support wrapping the output format in Metacube, as used by the Cubemap
> stream reflector (originally designed for VLC). This is nominally meant
> to run over HTTP, but longer-term would probably also be useful for
> pipe output as a subprocess of Cubemap.
>
> Integrating with Cubemap instantly gives FFmpeg access to a high-performance
> (40Gbit++/sec on a regular quadcore, with TLS), multi-user, proven HTTP
> reflector -- FFmpeg's own HTTP server still only has experimental multi-user
> support. However, Cubemap is deliberately dumb and thus does not understand any
> muxes; it is dependent on the origin (e.g. FFmpeg) to mark which bytes are
> headers, and at which bytes new clients can start the stream, which is why
> it needs its input wrapped in Metacube.
>
> Metacube output is activated by -fflags metacube. E.g., to create streamable
> MP4 on port 9095 that Cubemap can pick up and reflect:
>
>  ffmpeg -i <input> -f mp4 -movflags empty_moov+frag_keyframe+default_base_moof+skip_trailer \
>    -frag_duration 125000 -fflags metacube -listen 1 'http://[::]:9095'
>
> Tested with MP4 and Matroska.
>
> The metacube2.h header comes from the Cubemap distribution, and is nominally
> public domain. It can be considered to be licensed under the LGPL 2.1, like
> the rest of FFmpeg.
>
> Signed-off-by: Steinar H. Gunderson <steinar+ffmpeg@gunderson.no>
> ---
> fftools/ffmpeg_opt.c        |   6 ++-
> libavformat/avformat.h      |   1 +
> libavformat/avio.h          |  30 +++++++++++
> libavformat/aviobuf.c       | 103 +++++++++++++++++++++++++++++++++---
> libavformat/http.c          |  14 ++++-
> libavformat/metacube2.h     |  35 ++++++++++++
> libavformat/movenc.c        |   4 +-
> libavformat/options_table.h |   1 +
> 8 files changed, 184 insertions(+), 10 deletions(-)
> create mode 100644 libavformat/metacube2.h

It is quite ugly that you are introducing this in *avio*. Why is this not 
an option of HTTP?

Thanks,
Marton

>
> diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c
> index c0b9f023bd6..f6b1c6d632a 100644
> --- a/fftools/ffmpeg_opt.c
> +++ b/fftools/ffmpeg_opt.c
> @@ -2588,11 +2588,15 @@ loop_end:
>     }
>
>     if (!(oc->oformat->flags & AVFMT_NOFILE)) {
> +        int flags = AVIO_FLAG_WRITE;
> +        if (format_flags & AVFMT_FLAG_METACUBE)
> +            flags |= AVIO_FLAG_METACUBE;
> +
>         /* test if it already exists to avoid losing precious files */
>         assert_file_overwrite(filename);
>
>         /* open the file */
> -        if ((err = avio_open2(&oc->pb, filename, AVIO_FLAG_WRITE,
> +        if ((err = avio_open2(&oc->pb, filename, flags,
>                               &oc->interrupt_callback,
>                               &of->opts)) < 0) {
>             print_error(filename, err);
> diff --git a/libavformat/avformat.h b/libavformat/avformat.h
> index 624d2dae2c2..434ec215753 100644
> --- a/libavformat/avformat.h
> +++ b/libavformat/avformat.h
> @@ -1271,6 +1271,7 @@ typedef struct AVFormatContext {
> #define AVFMT_FLAG_FAST_SEEK   0x80000 ///< Enable fast, but inaccurate seeks for some formats
> #define AVFMT_FLAG_SHORTEST   0x100000 ///< Stop muxing when the shortest stream stops.
> #define AVFMT_FLAG_AUTO_BSF   0x200000 ///< Add bitstream filters as requested by the muxer
> +#define AVFMT_FLAG_METACUBE   0x400000 ///< Wrap output bitstream in Metacube (for Cubemap)
>
>     /**
>      * Maximum size of the data read from input for determining
> diff --git a/libavformat/avio.h b/libavformat/avio.h
> index d022820a6ed..bf74f2bbbe4 100644
> --- a/libavformat/avio.h
> +++ b/libavformat/avio.h
> @@ -349,6 +349,30 @@ typedef struct AVIOContext {
>      * Try to buffer at least this amount of data before flushing it
>      */
>     int min_packet_size;
> +
> +    /**
> +     * If set, all output will be wrapped in the Metacube format,
> +     * for consumption by the Cubemap reflector. This is so that Cubemap
> +     * can know what the header is, and where it is possible to start
> +     * the stream (ie., from keyframes) without actually parsing and
> +     * understanding the mux. Only relevant if write_flag is set.
> +     *
> +     * When wrapping in Metacube, s->buffer will have room for a 16-byte
> +     * Metacube header while writing, which is constructed in avio_flush()
> +     * before sending. This header is invisible to the calling code;
> +     * e.g., it will not be counted in seeks and similar.
> +     */
> +    int metacube;
> +
> +    /**
> +     * If the metacube flag is set, we need to know whether we've seen
> +     * at least one sync point marker (AVIO_DATA_MARKER_SYNC_POINT),
> +     * as many muxes don't send them out at all. If we haven't seen any sync
> +     * point markers, we assume that all packets (in particular
> +     * AVIO_DATA_MARKER_UNKNOWN) are valid sync start points.
> +     * (This may not hold for all codecs in practice.)
> +     */
> +    int seen_sync_point;
> } AVIOContext;
>
> /**
> @@ -692,6 +716,12 @@ int avio_get_str16be(AVIOContext *pb, int maxlen, char *buf, int buflen);
>  */
> #define AVIO_FLAG_NONBLOCK 8
>
> +/**
> + * If set, all output will be wrapped in the Metacube format.
> + * See AVIOContext::metacube for more information.
> + */
> +#define AVIO_FLAG_METACUBE 16
> +
> /**
>  * Use direct mode.
>  * avio_read and avio_write should if possible be satisfied directly
> diff --git a/libavformat/aviobuf.c b/libavformat/aviobuf.c
> index ddfa4ecbf1c..dd7b58ff21f 100644
> --- a/libavformat/aviobuf.c
> +++ b/libavformat/aviobuf.c
> @@ -26,10 +26,12 @@
> #include "libavutil/log.h"
> #include "libavutil/opt.h"
> #include "libavutil/avassert.h"
> +#include "libavutil/thread.h"
> #include "avformat.h"
> #include "avio.h"
> #include "avio_internal.h"
> #include "internal.h"
> +#include "metacube2.h"
> #include "url.h"
> #include <stdarg.h>
>
> @@ -105,6 +107,7 @@ int ffio_init_context(AVIOContext *s,
>     s->seekable        = seek ? AVIO_SEEKABLE_NORMAL : 0;
>     s->min_packet_size = 0;
>     s->max_packet_size = 0;
> +    s->metacube        = 0;
>     s->update_checksum = NULL;
>     s->short_seek_threshold = SHORT_SEEK_THRESHOLD;
>
> @@ -174,10 +177,66 @@ static void writeout(AVIOContext *s, const uint8_t *data, int len)
>     s->pos += len;
> }
>
> +static AVOnce metacube2_crc_once_control = AV_ONCE_INIT;
> +static AVCRC metacube2_crc_table[257];
> +
> +static void metacube2_crc_init_table_once(void)
> +{
> +    av_assert0(av_crc_init(metacube2_crc_table, 0, 16, 0x8fdb, sizeof(metacube2_crc_table)) >= 0);
> +}
> +
> +static uint16_t metacube2_compute_crc(const struct metacube2_block_header *hdr)
> +{
> +    static const int data_len = sizeof(hdr->size) + sizeof(hdr->flags);
> +    const uint8_t *data = (uint8_t *)&hdr->size;
> +    uint16_t crc;
> +
> +    ff_thread_once(&metacube2_crc_once_control, metacube2_crc_init_table_once);
> +
> +    // Metacube2 specifies a CRC start of 0x1234, but its pycrc-derived CRC
> +    // includes a finalization step that is done somewhat differently in av_crc().
> +    // 0x1234 alone sent through that finalization becomes 0x394a, and then we
> +    // need a byte-swap of the CRC value (both on input and output) to account for
> +    // differing conventions.
> +    crc = av_crc(metacube2_crc_table, 0x4a39, data, data_len);
> +    return av_bswap16(crc);
> +}
> +
> +static void finalize_metacube_block_header(AVIOContext *s)
> +{
> +    struct metacube2_block_header hdr;
> +    int len = s->buf_ptr_max - s->buffer;
> +    int flags = 0;
> +
> +    if (s->current_type == AVIO_DATA_MARKER_SYNC_POINT)
> +        s->seen_sync_point = 1;
> +    else if (s->current_type == AVIO_DATA_MARKER_HEADER)
> +        // NOTE: If there are multiple blocks marked METACUBE_FLAGS_HEADER,
> +        // only the last one will count. This may become a problem if the
> +        // mux flushes halfway through the stream header; if so, we would
> +        // need to keep track of and concatenate the different parts.
> +        flags |= METACUBE_FLAGS_HEADER;
> +    else if (s->seen_sync_point)
> +        flags |= METACUBE_FLAGS_NOT_SUITABLE_FOR_STREAM_START;
> +
> +    memcpy(hdr.sync, METACUBE2_SYNC, sizeof(hdr.sync));
> +    AV_WB32(&hdr.size, len - sizeof(hdr));
> +    AV_WB16(&hdr.flags, flags);
> +    AV_WB16(&hdr.csum, metacube2_compute_crc(&hdr));
> +    memcpy(s->buffer, &hdr, sizeof(hdr));
> +}
> +
> static void flush_buffer(AVIOContext *s)
> {
> +    int buffer_empty;
>     s->buf_ptr_max = FFMAX(s->buf_ptr, s->buf_ptr_max);
> -    if (s->write_flag && s->buf_ptr_max > s->buffer) {
> +    if (s->metacube)
> +       buffer_empty = s->buf_ptr_max <= s->buffer + sizeof(struct metacube2_block_header);
> +    else
> +       buffer_empty = s->buf_ptr_max <= s->buffer;
> +    if (s->write_flag && !buffer_empty) {
> +        if (s->metacube)
> +            finalize_metacube_block_header(s);
>         writeout(s, s->buffer, s->buf_ptr_max - s->buffer);
>         if (s->update_checksum) {
>             s->checksum     = s->update_checksum(s->checksum, s->checksum_ptr,
> @@ -186,6 +245,10 @@ static void flush_buffer(AVIOContext *s)
>         }
>     }
>     s->buf_ptr = s->buf_ptr_max = s->buffer;
> +
> +    // Add space for Metacube header.
> +    if (s->write_flag && s->metacube)
> +        s->buf_ptr += sizeof(struct metacube2_block_header);
>     if (!s->write_flag)
>         s->buf_end = s->buffer;
> }
> @@ -214,7 +277,7 @@ void ffio_fill(AVIOContext *s, int b, int count)
>
> void avio_write(AVIOContext *s, const unsigned char *buf, int size)
> {
> -    if (s->direct && !s->update_checksum) {
> +    if (s->direct && !s->update_checksum && !s->metacube) {
>         avio_flush(s);
>         writeout(s, buf, size);
>         return;
> @@ -264,11 +327,17 @@ int64_t avio_seek(AVIOContext *s, int64_t offset, int whence)
>
>     if (whence == SEEK_CUR) {
>         offset1 = pos + (s->buf_ptr - s->buffer);
> -        if (offset == 0)
> -            return offset1;
> +        if (offset == 0) {
> +            if (s->metacube && s->write_flag)
> +                return offset1 - sizeof(struct metacube2_block_header);
> +            else
> +                return offset1;
> +        }
>         if (offset > INT64_MAX - offset1)
>             return AVERROR(EINVAL);
>         offset += offset1;
> +    } else if (s->metacube && s->write_flag) {
> +        offset += sizeof(struct metacube2_block_header);
>     }
>     if (offset < 0)
>         return AVERROR(EINVAL);
> @@ -321,7 +390,10 @@ int64_t avio_seek(AVIOContext *s, int64_t offset, int whence)
>         s->pos = offset;
>     }
>     s->eof_reached = 0;
> -    return offset;
> +    if (s->metacube && s->write_flag)
> +        return offset - sizeof(struct metacube2_block_header);
> +    else
> +        return offset;
> }
>
> int64_t avio_skip(AVIOContext *s, int64_t offset)
> @@ -473,7 +545,7 @@ void avio_write_marker(AVIOContext *s, int64_t time, enum AVIODataMarkerType typ
>             avio_flush(s);
>         return;
>     }
> -    if (!s->write_data_type)
> +    if (!s->write_data_type && !s->metacube)
>         return;
>     // If ignoring boundary points, just treat it as unknown
>     if (type == AVIO_DATA_MARKER_BOUNDARY_POINT && s->ignore_boundary_point)
> @@ -953,6 +1025,8 @@ int ffio_fdopen(AVIOContext **s, URLContext *h)
>     }
>     (*s)->short_seek_get = (int (*)(void *))ffurl_get_short_seek;
>     (*s)->av_class = &ff_avio_class;
> +    (*s)->metacube = h->flags & AVIO_FLAG_METACUBE;
> +    (*s)->seen_sync_point = 0;
>     return 0;
> fail:
>     av_freep(&buffer);
> @@ -1016,6 +1090,10 @@ int ffio_ensure_seekback(AVIOContext *s, int64_t buf_size)
> int ffio_set_buf_size(AVIOContext *s, int buf_size)
> {
>     uint8_t *buffer;
> +
> +    if (s->metacube)
> +        buf_size += sizeof(struct metacube2_block_header);
> +
>     buffer = av_malloc(buf_size);
>     if (!buffer)
>         return AVERROR(ENOMEM);
> @@ -1025,6 +1103,11 @@ int ffio_set_buf_size(AVIOContext *s, int buf_size)
>     s->orig_buffer_size =
>     s->buffer_size = buf_size;
>     s->buf_ptr = s->buf_ptr_max = buffer;
> +
> +    // Add space for Metacube header.
> +    if (s->metacube)
> +        s->buf_ptr += sizeof(struct metacube2_block_header);
> +
>     url_resetbuf(s, s->write_flag ? AVIO_FLAG_WRITE : AVIO_FLAG_READ);
>     return 0;
> }
> @@ -1034,6 +1117,9 @@ int ffio_realloc_buf(AVIOContext *s, int buf_size)
>     uint8_t *buffer;
>     int data_size;
>
> +    if (s->metacube && s->write_flag)
> +        buf_size += sizeof(struct metacube2_block_header);
> +
>     if (!s->buffer_size)
>         return ffio_set_buf_size(s, buf_size);
>
> @@ -1052,6 +1138,11 @@ int ffio_realloc_buf(AVIOContext *s, int buf_size)
>     s->orig_buffer_size = buf_size;
>     s->buffer_size = buf_size;
>     s->buf_ptr = s->write_flag ? (s->buffer + data_size) : s->buffer;
> +
> +    // Add space for Metacube header.
> +    if (s->metacube && s->write_flag && data_size == 0)
> +        s->buf_ptr += sizeof(struct metacube2_block_header);
> +
>     if (s->write_flag)
>         s->buf_ptr_max = s->buffer + data_size;
>
> diff --git a/libavformat/http.c b/libavformat/http.c
> index 1fc95c768cd..5a0dda400c3 100644
> --- a/libavformat/http.c
> +++ b/libavformat/http.c
> @@ -500,7 +500,19 @@ static int http_write_reply(URLContext* h, int status_code)
>     default:
>         return AVERROR(EINVAL);
>     }
> -    if (body) {
> +    if (h->flags & AVIO_FLAG_METACUBE) {
> +        s->chunked_post = 0;
> +        message_len = snprintf(message, sizeof(message),
> +                 "HTTP/1.1 %03d %s\r\n"
> +                 "Content-Type: %s\r\n"
> +                 "Content-Encoding: metacube\r\n"
> +                 "%s"
> +                 "\r\n",
> +                 reply_code,
> +                 reply_text,
> +                 content_type,
> +                 s->headers ? s->headers : "");
> +    } else if (body) {
>         s->chunked_post = 0;
>         message_len = snprintf(message, sizeof(message),
>                  "HTTP/1.1 %03d %s\r\n"
> diff --git a/libavformat/metacube2.h b/libavformat/metacube2.h
> new file mode 100644
> index 00000000000..ada0031c0b8
> --- /dev/null
> +++ b/libavformat/metacube2.h
> @@ -0,0 +1,35 @@
> +#ifndef AVFORMAT_METACUBE2_H
> +#define AVFORMAT_METACUBE2_H
> +
> +/*
> + * Definitions for the Metacube2 protocol, used to communicate with Cubemap.
> + *
> + * Note: This file is meant to compile as both C and C++, for easier inclusion
> + * in other projects.
> + */
> +
> +#include <stdint.h>
> +
> +#define METACUBE2_SYNC "cube!map"  /* 8 bytes long. */
> +#define METACUBE_FLAGS_HEADER 0x1  /* NOTE: Replaces the previous header. */
> +#define METACUBE_FLAGS_NOT_SUITABLE_FOR_STREAM_START 0x2
> +
> +/*
> + * Metadata packets; should not be counted as data, but rather
> + * parsed (or ignored if you don't understand them).
> + *
> + * Metadata packets start with a uint64_t (network byte order)
> + * that describe the type; the rest is defined by the type.
> + */
> +#define METACUBE_FLAGS_METADATA 0x4
> +
> +struct metacube2_block_header {
> +	char sync[8];    /* METACUBE2_SYNC */
> +	uint32_t size;   /* Network byte order. Does not include header. */
> +	uint16_t flags;  /* Network byte order. METACUBE_FLAGS_*. */
> +	uint16_t csum;   /* Network byte order. CRC16 of size and flags.
> +                            If METACUBE_FLAGS_METADATA is set, inverted
> +                            so that older clients will ignore it as broken. */
> +};
> +
> +#endif /* AVFORMAT_METACUBE2_H */
> diff --git a/libavformat/movenc.c b/libavformat/movenc.c
> index f33792661b7..ee28f0ed7b6 100644
> --- a/libavformat/movenc.c
> +++ b/libavformat/movenc.c
> @@ -6841,8 +6841,6 @@ static int mov_write_header(AVFormatContext *s)
>         }
>     }
>
> -    avio_flush(pb);
> -
>     if (mov->flags & FF_MOV_FLAG_ISML)
>         mov_write_isml_manifest(pb, mov, s);
>
> @@ -6855,6 +6853,8 @@ static int mov_write_header(AVFormatContext *s)
>             mov->reserved_header_pos = avio_tell(pb);
>     }
>
> +    avio_flush(pb);
> +
>     return 0;
> }
>
> diff --git a/libavformat/options_table.h b/libavformat/options_table.h
> index 62c5bb40a39..9bc8edda358 100644
> --- a/libavformat/options_table.h
> +++ b/libavformat/options_table.h
> @@ -53,6 +53,7 @@ static const AVOption avformat_options[] = {
> {"bitexact", "do not write random/volatile data", 0, AV_OPT_TYPE_CONST, { .i64 = AVFMT_FLAG_BITEXACT }, 0, 0, E, "fflags" },
> {"shortest", "stop muxing with the shortest stream", 0, AV_OPT_TYPE_CONST, { .i64 = AVFMT_FLAG_SHORTEST }, 0, 0, E, "fflags" },
> {"autobsf", "add needed bsfs automatically", 0, AV_OPT_TYPE_CONST, { .i64 = AVFMT_FLAG_AUTO_BSF }, 0, 0, E, "fflags" },
> +{"metacube", "wrap output data in Metacube", 0, AV_OPT_TYPE_CONST, { .i64 = AVFMT_FLAG_METACUBE }, 0, 0, E, "fflags" },
> {"seek2any", "allow seeking to non-keyframes on demuxer level when supported", OFFSET(seek2any), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, D},
> {"analyzeduration", "specify how many microseconds are analyzed to probe the input", OFFSET(max_analyze_duration), AV_OPT_TYPE_INT64, {.i64 = 0 }, 0, INT64_MAX, D},
> {"cryptokey", "decryption key", OFFSET(key), AV_OPT_TYPE_BINARY, {.dbl = 0}, 0, 0, D},
> -- 
> 2.20.1
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>

On Mon, May 03, 2021 at 10:51:43PM +0200, Marton Balint wrote:
> It is quite ugly that you are introducing this in *avio*. Why is this not an
> option of HTTP?

Two reasons:

 - As the commit message says, it is desirable to have this on pipe output as
   well (Cubemap would like to fork out to FFmpeg to do remuxing etc. for
   it, without having to speak HTTP over the pipe).
 - The HTTP layer does not have the information about what is a header etc.,
   so it would need to be plumbed down the layers. I tried this at first, but
   it got markedly more intrusive than this, so I left it pretty early on.

/* Steinar */

On 03/05/2021 22:05, Steinar H. Gunderson wrote:
> Two reasons:
> 
>  - As the commit message says, it is desirable to have this on pipe output as
>    well (Cubemap would like to fork out to FFmpeg to do remuxing etc. for
>    it, without having to speak HTTP over the pipe).
>  - The HTTP layer does not have the information about what is a header etc.,
>    so it would need to be plumbed down the layers. I tried this at first, but
>    it got markedly more intrusive than this, so I left it pretty early on.

Can this not be accomplished outside of FFmpeg, by registering your own I/O 
callbacks? That would seem to me to be the 'proper' way to do this. More
work? Yes. But less hacky.

I am in agreement that the current placement inside AVIO itself is king of horrible.

Two other points:
  * Why are there random unexlained changes to e.g. movdec's flushing behavior bundled
    in this patch?
  * You need to provide documentation / links on, like, what 'metacube' *is*. It's
    absolutely unclear to anyone reading this patch and associated doxy what it is
    or what its usecase is. AFAICT it's your own project?

- Derek

On Tue, May 4, 2021 at 5:52 PM Derek Buitenhuis
<derek.buitenhuis@gmail.com> wrote:
>
> On 03/05/2021 22:05, Steinar H. Gunderson wrote:
> > Two reasons:
> >
> >  - As the commit message says, it is desirable to have this on pipe output as
> >    well (Cubemap would like to fork out to FFmpeg to do remuxing etc. for
> >    it, without having to speak HTTP over the pipe).
> >  - The HTTP layer does not have the information about what is a header etc.,
> >    so it would need to be plumbed down the layers. I tried this at first, but
> >    it got markedly more intrusive than this, so I left it pretty early on.
>
> Can this not be accomplished outside of FFmpeg, by registering your own I/O
> callbacks? That would seem to me to be the 'proper' way to do this. More
> work? Yes. But less hacky.
>
> I am in agreement that the current placement inside AVIO itself is king of horrible.
>

I agree with that. AVIO is supposed to be generic IO abstraction,
something like this has no place in it.

AVIO already allows for various callbacks, if needed that framework
can be extended, without putting a lot of very specific code into a
place it doesn't belong.

- Hendrik

On Tue, May 04, 2021 at 04:46:29PM +0100, Derek Buitenhuis wrote:
> Can this not be accomplished outside of FFmpeg, by registering your own I/O 
> callbacks? That would seem to me to be the 'proper' way to do this. More
> work? Yes. But less hacky.

For a libavformat-using program that does its own transports? Sure, I'm
already doing that in other projects. For anything that wants to use the
ffurl transports -- and in particular, stream from the ffmpeg(1) client?
I don't see how that would be possible, unless I'm misunderstanding what
you're suggesting.

>   * You need to provide documentation / links on, like, what 'metacube' *is*. It's
>     absolutely unclear to anyone reading this patch and associated doxy what it is
>     or what its usecase is. AFAICT it's your own project?

I believe this was already covered in the commit message...? But yes, Cubemap
is my own project, and Metacube is its framing protocol.

I'm not seeing a lot of love for this patch, so I won't be pursuing it any
further -- I wanted to upstream it due to user requests, but it is small
enough that it can live in my own tree pretty much indefinitely.

/* Steinar */

On 04/05/2021 18:22, Steinar H. Gunderson wrote:
> For a libavformat-using program that does its own transports? Sure, I'm
> already doing that in other projects. For anything that wants to use the
> ffurl transports -- and in particular, stream from the ffmpeg(1) client?
> I don't see how that would be possible, unless I'm misunderstanding what
> you're suggesting.

Yes that's what I meant. It could in theory be reworked to to be in avformat/ffurl
but not avio itself (just the generic mechanisms by which it works), though, too.


>>   * You need to provide documentation / links on, like, what 'metacube' *is*. It's
>>     absolutely unclear to anyone reading this patch and associated doxy what it is
>>     or what its usecase is. AFAICT it's your own project?
> 
> I believe this was already covered in the commit message...? But yes, Cubemap
> is my own project, and Metacube is its framing protocol.

Only sorta. It doesn't link to he project, and Google wasn't immediately helpful.
There's a bunch of information about implementation details and interactions, but
not a ton on high level things like: What is the usecase? Why do we want this feature?
What's the plan for how it'll be used? What's the end goal? It's all kind of nebulous
to me, but maybe I'm not bright enough to infer or understand these...

There's no information in the doxy at all for users of this API who may not be
familiar with what metacube is, where it's from, what what its usecase is - why
they may want to use this API feature.

> I'm not seeing a lot of love for this patch, so I won't be pursuing it any
> further -- I wanted to upstream it due to user requests, but it is small
> enough that it can live in my own tree pretty much indefinitely.

This is your perogative, so I won't argue.

I will say that it is unclear enough to me what the goals and uses are of
the patch that it came off a bit like upstreaming something purely for personal
use, even if that wasn't intended. Apologies.

Cheers,
- Derek

On Tue, May 04, 2021 at 06:38:41PM +0100, Derek Buitenhuis wrote:
> I will say that it is unclear enough to me what the goals and uses are of
> the patch that it came off a bit like upstreaming something purely for personal
> use, even if that wasn't intended. Apologies.

No offense taken. The basic goal was to make it possible to stream from
ffmpeg(1) over HTTP, to many users (thousands).[1] I believe the existing
HTTP server is unsuitable for that, and making it so would basically involve
reimplementing Cubemap.

[1] Or, if you want to see it the other way round -- to add FFmpeg input
    support to Cubemap (loosely coupled). Basically the same thing.

/* Steinar */

On Tue, 4 May 2021, Steinar H. Gunderson wrote:

> On Tue, May 04, 2021 at 04:46:29PM +0100, Derek Buitenhuis wrote:
>> Can this not be accomplished outside of FFmpeg, by registering your own I/O
>> callbacks? That would seem to me to be the 'proper' way to do this. More
>> work? Yes. But less hacky.
>
> For a libavformat-using program that does its own transports? Sure, I'm
> already doing that in other projects. For anything that wants to use the
> ffurl transports -- and in particular, stream from the ffmpeg(1) client?
> I don't see how that would be possible, unless I'm misunderstanding what
> you're suggesting.

I guess you'd need to extend the URLProtocol struct with a pointer that 
can be set to the write_data_type callback (which sounds fine to me) - and 
you could have e.g. a url in the form metacube+http://server - that would 
allow a "metacube" protocol handle it, which then can open the nested part 
of the url as a nested protocol.

>
>>   * You need to provide documentation / links on, like, what 'metacube' *is*. It's
>>     absolutely unclear to anyone reading this patch and associated doxy what it is
>>     or what its usecase is. AFAICT it's your own project?
>
> I believe this was already covered in the commit message...? But yes, Cubemap
> is my own project, and Metacube is its framing protocol.
>
> I'm not seeing a lot of love for this patch, so I won't be pursuing it any
> further -- I wanted to upstream it due to user requests, but it is small
> enough that it can live in my own tree pretty much indefinitely.

I have actually been interested in testing this and considered writing an 
implementation of this protocol at some point, but I've never got to it. 
I'd say most aversion is to the implementation - the cause itself is fine 
IMO. (While it's less of a widely known protocol than e.g. gopher, it's 
probably more useful.)

// Martin