From patchwork Mon May 3 19:59:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Steinar H. Gunderson" X-Patchwork-Id: 27578 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a10:399:0:0:0:0 with SMTP id 25csp576706pxh; Mon, 3 May 2021 12:59:57 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzKmJmPnJNl7OEIFBvfi1ucEshbNI5/VSTtze+pNy630c28NYEH2t8z4W5yDJE5NKLP5yjt X-Received: by 2002:a17:907:3f08:: with SMTP id hq8mr4134991ejc.240.1620071997518; Mon, 03 May 2021 12:59:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1620071997; cv=none; d=google.com; s=arc-20160816; b=LC830SixpGdAV5o0nYVk4EHqW2uF2r5xjtjWQnEAHaKqX0Nb+udrvHVFJqVeBGDXHZ bMjPBRQmbKg5VSM9ue/C8fc+hYCwrsmH7oHi2Sg5UERQml8IQSXDtQH4wPu24yBDScbn Uz3fk4bv5bcDGjTLdDNJOfVhoUYhTn+Ywaq++RKOxAUewYpaiGU1JswFGeVJA7/jfuqj AvmkHOsYQbtlXYtiPmF0pk5v2R3bzm23mabeRiG/rpUBLIpYimoMPRDEf4tTl2R8PgSS oLHDeJvXKIkqIHXo8ZdS0E6npI01EbBdQX1py5zTjrpRb75Ep0u6O9jQSvveEcvLzsZ+ WfgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=IC3MjF1uPTi1gGn/qlp0RwHXVxnVheiPsAsk6jPieuk=; b=gjlMrJUClR9qteyI5uRYW9MlSYeZfL5zPgsgkZNSi46VRq/gC5fQZME3rnriomX9y3 qIkxLCOewg1gautPTTm27In4P938H7+flwDapIZzS2iNFWgpFWsWvKnPsVyUKIIyHgrd 3EU5e8qm9VnCrVMyE6BCQeFU/46jZt86Uc2G+Vx3mKEtKS/77k5rKq4c00WJ7jpCOcuZ tf6IgSOFHXBDTJ0FIbhr/Xc3FIyZmkR5pPj+gN1ahJWRppqqb1EzbUOUd/q8TIU5pFS9 m+IhEh/F/k4MWO7liJ4AGct8+qyhR71FrfzbjHQm6k/fqD1kR/+9NrxJFlt1+rs9k55w TokA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a20si9925551edv.202.2021.05.03.12.59.56; Mon, 03 May 2021 12:59:57 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BA6606801ED; Mon, 3 May 2021 22:59:52 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from pannekake.samfundet.no (pannekake.samfundet.no [193.35.52.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6A6E86801ED for ; Mon, 3 May 2021 22:59:45 +0300 (EEST) Received: from sesse by pannekake.samfundet.no with local (Exim 4.92) (envelope-from ) id 1ldejI-0001O3-0q; Mon, 03 May 2021 21:59:44 +0200 From: "Steinar H. Gunderson" To: ffmpeg-devel@ffmpeg.org Date: Mon, 3 May 2021 21:59:31 +0200 Message-Id: <20210503195931.5234-1-steinar+ffmpeg@gunderson.no> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avformat/avio: Add Metacube support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "Steinar H. Gunderson" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: SDsuOVP9VzFg Support wrapping the output format in Metacube, as used by the Cubemap stream reflector (originally designed for VLC). This is nominally meant to run over HTTP, but longer-term would probably also be useful for pipe output as a subprocess of Cubemap. Integrating with Cubemap instantly gives FFmpeg access to a high-performance (40Gbit++/sec on a regular quadcore, with TLS), multi-user, proven HTTP reflector -- FFmpeg's own HTTP server still only has experimental multi-user support. However, Cubemap is deliberately dumb and thus does not understand any muxes; it is dependent on the origin (e.g. FFmpeg) to mark which bytes are headers, and at which bytes new clients can start the stream, which is why it needs its input wrapped in Metacube. Metacube output is activated by -fflags metacube. E.g., to create streamable MP4 on port 9095 that Cubemap can pick up and reflect: ffmpeg -i -f mp4 -movflags empty_moov+frag_keyframe+default_base_moof+skip_trailer \ -frag_duration 125000 -fflags metacube -listen 1 'http://[::]:9095' Tested with MP4 and Matroska. The metacube2.h header comes from the Cubemap distribution, and is nominally public domain. It can be considered to be licensed under the LGPL 2.1, like the rest of FFmpeg. Signed-off-by: Steinar H. Gunderson --- fftools/ffmpeg_opt.c | 6 ++- libavformat/avformat.h | 1 + libavformat/avio.h | 30 +++++++++++ libavformat/aviobuf.c | 103 +++++++++++++++++++++++++++++++++--- libavformat/http.c | 14 ++++- libavformat/metacube2.h | 35 ++++++++++++ libavformat/movenc.c | 4 +- libavformat/options_table.h | 1 + 8 files changed, 184 insertions(+), 10 deletions(-) create mode 100644 libavformat/metacube2.h diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index c0b9f023bd6..f6b1c6d632a 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -2588,11 +2588,15 @@ loop_end: } if (!(oc->oformat->flags & AVFMT_NOFILE)) { + int flags = AVIO_FLAG_WRITE; + if (format_flags & AVFMT_FLAG_METACUBE) + flags |= AVIO_FLAG_METACUBE; + /* test if it already exists to avoid losing precious files */ assert_file_overwrite(filename); /* open the file */ - if ((err = avio_open2(&oc->pb, filename, AVIO_FLAG_WRITE, + if ((err = avio_open2(&oc->pb, filename, flags, &oc->interrupt_callback, &of->opts)) < 0) { print_error(filename, err); diff --git a/libavformat/avformat.h b/libavformat/avformat.h index 624d2dae2c2..434ec215753 100644 --- a/libavformat/avformat.h +++ b/libavformat/avformat.h @@ -1271,6 +1271,7 @@ typedef struct AVFormatContext { #define AVFMT_FLAG_FAST_SEEK 0x80000 ///< Enable fast, but inaccurate seeks for some formats #define AVFMT_FLAG_SHORTEST 0x100000 ///< Stop muxing when the shortest stream stops. #define AVFMT_FLAG_AUTO_BSF 0x200000 ///< Add bitstream filters as requested by the muxer +#define AVFMT_FLAG_METACUBE 0x400000 ///< Wrap output bitstream in Metacube (for Cubemap) /** * Maximum size of the data read from input for determining diff --git a/libavformat/avio.h b/libavformat/avio.h index d022820a6ed..bf74f2bbbe4 100644 --- a/libavformat/avio.h +++ b/libavformat/avio.h @@ -349,6 +349,30 @@ typedef struct AVIOContext { * Try to buffer at least this amount of data before flushing it */ int min_packet_size; + + /** + * If set, all output will be wrapped in the Metacube format, + * for consumption by the Cubemap reflector. This is so that Cubemap + * can know what the header is, and where it is possible to start + * the stream (ie., from keyframes) without actually parsing and + * understanding the mux. Only relevant if write_flag is set. + * + * When wrapping in Metacube, s->buffer will have room for a 16-byte + * Metacube header while writing, which is constructed in avio_flush() + * before sending. This header is invisible to the calling code; + * e.g., it will not be counted in seeks and similar. + */ + int metacube; + + /** + * If the metacube flag is set, we need to know whether we've seen + * at least one sync point marker (AVIO_DATA_MARKER_SYNC_POINT), + * as many muxes don't send them out at all. If we haven't seen any sync + * point markers, we assume that all packets (in particular + * AVIO_DATA_MARKER_UNKNOWN) are valid sync start points. + * (This may not hold for all codecs in practice.) + */ + int seen_sync_point; } AVIOContext; /** @@ -692,6 +716,12 @@ int avio_get_str16be(AVIOContext *pb, int maxlen, char *buf, int buflen); */ #define AVIO_FLAG_NONBLOCK 8 +/** + * If set, all output will be wrapped in the Metacube format. + * See AVIOContext::metacube for more information. + */ +#define AVIO_FLAG_METACUBE 16 + /** * Use direct mode. * avio_read and avio_write should if possible be satisfied directly diff --git a/libavformat/aviobuf.c b/libavformat/aviobuf.c index ddfa4ecbf1c..dd7b58ff21f 100644 --- a/libavformat/aviobuf.c +++ b/libavformat/aviobuf.c @@ -26,10 +26,12 @@ #include "libavutil/log.h" #include "libavutil/opt.h" #include "libavutil/avassert.h" +#include "libavutil/thread.h" #include "avformat.h" #include "avio.h" #include "avio_internal.h" #include "internal.h" +#include "metacube2.h" #include "url.h" #include @@ -105,6 +107,7 @@ int ffio_init_context(AVIOContext *s, s->seekable = seek ? AVIO_SEEKABLE_NORMAL : 0; s->min_packet_size = 0; s->max_packet_size = 0; + s->metacube = 0; s->update_checksum = NULL; s->short_seek_threshold = SHORT_SEEK_THRESHOLD; @@ -174,10 +177,66 @@ static void writeout(AVIOContext *s, const uint8_t *data, int len) s->pos += len; } +static AVOnce metacube2_crc_once_control = AV_ONCE_INIT; +static AVCRC metacube2_crc_table[257]; + +static void metacube2_crc_init_table_once(void) +{ + av_assert0(av_crc_init(metacube2_crc_table, 0, 16, 0x8fdb, sizeof(metacube2_crc_table)) >= 0); +} + +static uint16_t metacube2_compute_crc(const struct metacube2_block_header *hdr) +{ + static const int data_len = sizeof(hdr->size) + sizeof(hdr->flags); + const uint8_t *data = (uint8_t *)&hdr->size; + uint16_t crc; + + ff_thread_once(&metacube2_crc_once_control, metacube2_crc_init_table_once); + + // Metacube2 specifies a CRC start of 0x1234, but its pycrc-derived CRC + // includes a finalization step that is done somewhat differently in av_crc(). + // 0x1234 alone sent through that finalization becomes 0x394a, and then we + // need a byte-swap of the CRC value (both on input and output) to account for + // differing conventions. + crc = av_crc(metacube2_crc_table, 0x4a39, data, data_len); + return av_bswap16(crc); +} + +static void finalize_metacube_block_header(AVIOContext *s) +{ + struct metacube2_block_header hdr; + int len = s->buf_ptr_max - s->buffer; + int flags = 0; + + if (s->current_type == AVIO_DATA_MARKER_SYNC_POINT) + s->seen_sync_point = 1; + else if (s->current_type == AVIO_DATA_MARKER_HEADER) + // NOTE: If there are multiple blocks marked METACUBE_FLAGS_HEADER, + // only the last one will count. This may become a problem if the + // mux flushes halfway through the stream header; if so, we would + // need to keep track of and concatenate the different parts. + flags |= METACUBE_FLAGS_HEADER; + else if (s->seen_sync_point) + flags |= METACUBE_FLAGS_NOT_SUITABLE_FOR_STREAM_START; + + memcpy(hdr.sync, METACUBE2_SYNC, sizeof(hdr.sync)); + AV_WB32(&hdr.size, len - sizeof(hdr)); + AV_WB16(&hdr.flags, flags); + AV_WB16(&hdr.csum, metacube2_compute_crc(&hdr)); + memcpy(s->buffer, &hdr, sizeof(hdr)); +} + static void flush_buffer(AVIOContext *s) { + int buffer_empty; s->buf_ptr_max = FFMAX(s->buf_ptr, s->buf_ptr_max); - if (s->write_flag && s->buf_ptr_max > s->buffer) { + if (s->metacube) + buffer_empty = s->buf_ptr_max <= s->buffer + sizeof(struct metacube2_block_header); + else + buffer_empty = s->buf_ptr_max <= s->buffer; + if (s->write_flag && !buffer_empty) { + if (s->metacube) + finalize_metacube_block_header(s); writeout(s, s->buffer, s->buf_ptr_max - s->buffer); if (s->update_checksum) { s->checksum = s->update_checksum(s->checksum, s->checksum_ptr, @@ -186,6 +245,10 @@ static void flush_buffer(AVIOContext *s) } } s->buf_ptr = s->buf_ptr_max = s->buffer; + + // Add space for Metacube header. + if (s->write_flag && s->metacube) + s->buf_ptr += sizeof(struct metacube2_block_header); if (!s->write_flag) s->buf_end = s->buffer; } @@ -214,7 +277,7 @@ void ffio_fill(AVIOContext *s, int b, int count) void avio_write(AVIOContext *s, const unsigned char *buf, int size) { - if (s->direct && !s->update_checksum) { + if (s->direct && !s->update_checksum && !s->metacube) { avio_flush(s); writeout(s, buf, size); return; @@ -264,11 +327,17 @@ int64_t avio_seek(AVIOContext *s, int64_t offset, int whence) if (whence == SEEK_CUR) { offset1 = pos + (s->buf_ptr - s->buffer); - if (offset == 0) - return offset1; + if (offset == 0) { + if (s->metacube && s->write_flag) + return offset1 - sizeof(struct metacube2_block_header); + else + return offset1; + } if (offset > INT64_MAX - offset1) return AVERROR(EINVAL); offset += offset1; + } else if (s->metacube && s->write_flag) { + offset += sizeof(struct metacube2_block_header); } if (offset < 0) return AVERROR(EINVAL); @@ -321,7 +390,10 @@ int64_t avio_seek(AVIOContext *s, int64_t offset, int whence) s->pos = offset; } s->eof_reached = 0; - return offset; + if (s->metacube && s->write_flag) + return offset - sizeof(struct metacube2_block_header); + else + return offset; } int64_t avio_skip(AVIOContext *s, int64_t offset) @@ -473,7 +545,7 @@ void avio_write_marker(AVIOContext *s, int64_t time, enum AVIODataMarkerType typ avio_flush(s); return; } - if (!s->write_data_type) + if (!s->write_data_type && !s->metacube) return; // If ignoring boundary points, just treat it as unknown if (type == AVIO_DATA_MARKER_BOUNDARY_POINT && s->ignore_boundary_point) @@ -953,6 +1025,8 @@ int ffio_fdopen(AVIOContext **s, URLContext *h) } (*s)->short_seek_get = (int (*)(void *))ffurl_get_short_seek; (*s)->av_class = &ff_avio_class; + (*s)->metacube = h->flags & AVIO_FLAG_METACUBE; + (*s)->seen_sync_point = 0; return 0; fail: av_freep(&buffer); @@ -1016,6 +1090,10 @@ int ffio_ensure_seekback(AVIOContext *s, int64_t buf_size) int ffio_set_buf_size(AVIOContext *s, int buf_size) { uint8_t *buffer; + + if (s->metacube) + buf_size += sizeof(struct metacube2_block_header); + buffer = av_malloc(buf_size); if (!buffer) return AVERROR(ENOMEM); @@ -1025,6 +1103,11 @@ int ffio_set_buf_size(AVIOContext *s, int buf_size) s->orig_buffer_size = s->buffer_size = buf_size; s->buf_ptr = s->buf_ptr_max = buffer; + + // Add space for Metacube header. + if (s->metacube) + s->buf_ptr += sizeof(struct metacube2_block_header); + url_resetbuf(s, s->write_flag ? AVIO_FLAG_WRITE : AVIO_FLAG_READ); return 0; } @@ -1034,6 +1117,9 @@ int ffio_realloc_buf(AVIOContext *s, int buf_size) uint8_t *buffer; int data_size; + if (s->metacube && s->write_flag) + buf_size += sizeof(struct metacube2_block_header); + if (!s->buffer_size) return ffio_set_buf_size(s, buf_size); @@ -1052,6 +1138,11 @@ int ffio_realloc_buf(AVIOContext *s, int buf_size) s->orig_buffer_size = buf_size; s->buffer_size = buf_size; s->buf_ptr = s->write_flag ? (s->buffer + data_size) : s->buffer; + + // Add space for Metacube header. + if (s->metacube && s->write_flag && data_size == 0) + s->buf_ptr += sizeof(struct metacube2_block_header); + if (s->write_flag) s->buf_ptr_max = s->buffer + data_size; diff --git a/libavformat/http.c b/libavformat/http.c index 1fc95c768cd..5a0dda400c3 100644 --- a/libavformat/http.c +++ b/libavformat/http.c @@ -500,7 +500,19 @@ static int http_write_reply(URLContext* h, int status_code) default: return AVERROR(EINVAL); } - if (body) { + if (h->flags & AVIO_FLAG_METACUBE) { + s->chunked_post = 0; + message_len = snprintf(message, sizeof(message), + "HTTP/1.1 %03d %s\r\n" + "Content-Type: %s\r\n" + "Content-Encoding: metacube\r\n" + "%s" + "\r\n", + reply_code, + reply_text, + content_type, + s->headers ? s->headers : ""); + } else if (body) { s->chunked_post = 0; message_len = snprintf(message, sizeof(message), "HTTP/1.1 %03d %s\r\n" diff --git a/libavformat/metacube2.h b/libavformat/metacube2.h new file mode 100644 index 00000000000..ada0031c0b8 --- /dev/null +++ b/libavformat/metacube2.h @@ -0,0 +1,35 @@ +#ifndef AVFORMAT_METACUBE2_H +#define AVFORMAT_METACUBE2_H + +/* + * Definitions for the Metacube2 protocol, used to communicate with Cubemap. + * + * Note: This file is meant to compile as both C and C++, for easier inclusion + * in other projects. + */ + +#include + +#define METACUBE2_SYNC "cube!map" /* 8 bytes long. */ +#define METACUBE_FLAGS_HEADER 0x1 /* NOTE: Replaces the previous header. */ +#define METACUBE_FLAGS_NOT_SUITABLE_FOR_STREAM_START 0x2 + +/* + * Metadata packets; should not be counted as data, but rather + * parsed (or ignored if you don't understand them). + * + * Metadata packets start with a uint64_t (network byte order) + * that describe the type; the rest is defined by the type. + */ +#define METACUBE_FLAGS_METADATA 0x4 + +struct metacube2_block_header { + char sync[8]; /* METACUBE2_SYNC */ + uint32_t size; /* Network byte order. Does not include header. */ + uint16_t flags; /* Network byte order. METACUBE_FLAGS_*. */ + uint16_t csum; /* Network byte order. CRC16 of size and flags. + If METACUBE_FLAGS_METADATA is set, inverted + so that older clients will ignore it as broken. */ +}; + +#endif /* AVFORMAT_METACUBE2_H */ diff --git a/libavformat/movenc.c b/libavformat/movenc.c index f33792661b7..ee28f0ed7b6 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -6841,8 +6841,6 @@ static int mov_write_header(AVFormatContext *s) } } - avio_flush(pb); - if (mov->flags & FF_MOV_FLAG_ISML) mov_write_isml_manifest(pb, mov, s); @@ -6855,6 +6853,8 @@ static int mov_write_header(AVFormatContext *s) mov->reserved_header_pos = avio_tell(pb); } + avio_flush(pb); + return 0; } diff --git a/libavformat/options_table.h b/libavformat/options_table.h index 62c5bb40a39..9bc8edda358 100644 --- a/libavformat/options_table.h +++ b/libavformat/options_table.h @@ -53,6 +53,7 @@ static const AVOption avformat_options[] = { {"bitexact", "do not write random/volatile data", 0, AV_OPT_TYPE_CONST, { .i64 = AVFMT_FLAG_BITEXACT }, 0, 0, E, "fflags" }, {"shortest", "stop muxing with the shortest stream", 0, AV_OPT_TYPE_CONST, { .i64 = AVFMT_FLAG_SHORTEST }, 0, 0, E, "fflags" }, {"autobsf", "add needed bsfs automatically", 0, AV_OPT_TYPE_CONST, { .i64 = AVFMT_FLAG_AUTO_BSF }, 0, 0, E, "fflags" }, +{"metacube", "wrap output data in Metacube", 0, AV_OPT_TYPE_CONST, { .i64 = AVFMT_FLAG_METACUBE }, 0, 0, E, "fflags" }, {"seek2any", "allow seeking to non-keyframes on demuxer level when supported", OFFSET(seek2any), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, D}, {"analyzeduration", "specify how many microseconds are analyzed to probe the input", OFFSET(max_analyze_duration), AV_OPT_TYPE_INT64, {.i64 = 0 }, 0, INT64_MAX, D}, {"cryptokey", "decryption key", OFFSET(key), AV_OPT_TYPE_BINARY, {.dbl = 0}, 0, 0, D},