From patchwork Tue Feb 21 22:35:51 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vittorio Giovara X-Patchwork-Id: 2631 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.65.149 with SMTP id x21csp605440vsf; Tue, 21 Feb 2017 14:42:53 -0800 (PST) X-Received: by 10.28.70.2 with SMTP id t2mr15611858wma.34.1487716973270; Tue, 21 Feb 2017 14:42:53 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 35si12379891wre.175.2017.02.21.14.42.52; Tue, 21 Feb 2017 14:42:53 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8786F68830B; Wed, 22 Feb 2017 00:42:42 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt0-f193.google.com (mail-qt0-f193.google.com [209.85.216.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 24ED16882BB for ; Wed, 22 Feb 2017 00:42:35 +0200 (EET) Received: by mail-qt0-f193.google.com with SMTP id n13so19530041qtc.0 for ; Tue, 21 Feb 2017 14:42:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=QjlwR+s0zZUGqLpoh9H+yv1B7LoRMBXtrKzeNPteJX0=; b=NT1L6FOXlvcWUH270zlxWLjXVBRUVx8PCaSop83UvkgXH0O1qMFodJmNu+gtde5xIj oOddWRXPAB2u0jELf9Ka5E7nJesswUE1JaPV8KrkxSeBeBLTjpBgQkNIL+4K3bGzbjsL W7iKRlDbvuzEE6Z2ePEUfSHAxWF7XEYjYggi1xMiimSrH9ovcGd9THMGy+KSjI3Yxsyv bvHrXLakhT+MbcNVx+6FD/31lqu0Cay0LaiH+zlbhEzpabxyOly4cGH9zkUMbkN0jMGP ZkBxeFGo/BAvmkqqS1idWKAcYWXVNNE/Vn7DoCLFJB/4MpQNFxst0lbsZ1ZYsulrNQre qjFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=QjlwR+s0zZUGqLpoh9H+yv1B7LoRMBXtrKzeNPteJX0=; b=dWezMxPnCrxJfLUrCLvcG9wwWFhmHyk53a0KBESeUKCBrxUkLJiPNkVAl5Ozf41lR2 hOUwaHN2/LUy+UD/5PFZeQsoajBfXbhUWDuv1+FS56DdcvlOYqLDRPy1yfCHjp92x9AY 540BFbClnUNBUAIRhbJnUDKyj8bkAI2rGkic0j5vefmc3Edm4lcdImqQExPemt5UZJ2s 0H/zg47PO24zNjHzi2F6uypw4GiU+udLV6FgL/8dDTKRljBI40Jj2M8TpIrVaZloZQf7 2mGJGQb/SZtGe4yGB19eF2dgaCp2pIbROF/1562pGalhUcExhl7ZJTiPeTVgbGQjpa0H BxBA== X-Gm-Message-State: AMke39mMMaXZajayHOXVsIACLr/R7ghVxXn3oU/JQNUyrofV8JQhUWpV7woOn9t0+gE6jw== X-Received: by 10.237.38.197 with SMTP id q63mr29202495qtd.14.1487716555628; Tue, 21 Feb 2017 14:35:55 -0800 (PST) Received: from vimacbookpro.vimeows.com (nyv-exweb.iac.com. [216.112.252.10]) by smtp.gmail.com with ESMTPSA id t34sm1959074qtc.3.2017.02.21.14.35.54 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 21 Feb 2017 14:35:54 -0800 (PST) From: Vittorio Giovara To: ffmpeg-devel@ffmpeg.org Date: Tue, 21 Feb 2017 17:35:51 -0500 Message-Id: <20170221223553.65520-1-vittorio.giovara@gmail.com> X-Mailer: git-send-email 2.10.0 Subject: [FFmpeg-devel] [PATCHv3 1/3] spherical: Add tiled equirectangular type and projection-specific properties X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Vittorio Giovara --- This leaves bounds unchanged, simplifying future muxing code. Add a convenience function where human-readable values are needed. Update mov and mkv in subsequent patches. Vittorio doc/APIchanges | 5 +++ ffprobe.c | 19 +++++++-- libavformat/dump.c | 15 ++++++- libavutil/spherical.c | 18 +++++++++ libavutil/spherical.h | 74 ++++++++++++++++++++++++++++++++++ libavutil/version.h | 2 +- tests/ref/fate/matroska-spherical-mono | 2 +- tests/ref/fate/mov-spherical-mono | 2 +- 8 files changed, 128 insertions(+), 9 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index d739895..569d457 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -15,6 +15,11 @@ libavutil: 2015-08-28 API changes, most recent first: +2017-02-10 - xxxxxxx - lavu 55.48.100 / 55.33.0 - spherical.h + Add AV_SPHERICAL_EQUIRECTANGULAR_TILE, av_spherical_tile_bounds(), + and projection-specific properties (bound_left, bound_top, bound_right, + bound_bottom, padding) to AVSphericalMapping. + 2017-02-13 - xxxxxxx - lavc 57.80.100 - avcodec.h Add AVCodecContext.hw_device_ctx. diff --git a/ffprobe.c b/ffprobe.c index 046f080..c85c3a1 100644 --- a/ffprobe.c +++ b/ffprobe.c @@ -1762,6 +1762,7 @@ static inline int show_tags(WriterContext *w, AVDictionary *tags, int section_id } static void print_pkt_side_data(WriterContext *w, + AVCodecParameters *par, const AVPacketSideData *side_data, int nb_side_data, SectionID id_data_list, @@ -1788,9 +1789,19 @@ static void print_pkt_side_data(WriterContext *w, const AVSphericalMapping *spherical = (AVSphericalMapping *)sd->data; if (spherical->projection == AV_SPHERICAL_EQUIRECTANGULAR) print_str("projection", "equirectangular"); - else if (spherical->projection == AV_SPHERICAL_CUBEMAP) + else if (spherical->projection == AV_SPHERICAL_CUBEMAP) { print_str("projection", "cubemap"); - else + print_int("padding", spherical->padding); + } else if (spherical->projection == AV_SPHERICAL_EQUIRECTANGULAR_TILE) { + size_t l, t, r, b; + av_spherical_tile_bounds(spherical, par->width, par->height, + &l, &t, &r, &b); + print_str("projection", "tiled equirectangular"); + print_int("bound_left", l); + print_int("bound_top", t); + print_int("bound_right", r); + print_int("bound_bottom", b); + } else print_str("projection", "unknown"); print_int("yaw", (double) spherical->yaw / (1 << 16)); @@ -1843,7 +1854,7 @@ static void show_packet(WriterContext *w, InputFile *ifile, AVPacket *pkt, int p av_dict_free(&dict); } - print_pkt_side_data(w, pkt->side_data, pkt->side_data_elems, + print_pkt_side_data(w, st->codecpar, pkt->side_data, pkt->side_data_elems, SECTION_ID_PACKET_SIDE_DATA_LIST, SECTION_ID_PACKET_SIDE_DATA); } @@ -2404,7 +2415,7 @@ static int show_stream(WriterContext *w, AVFormatContext *fmt_ctx, int stream_id ret = show_tags(w, stream->metadata, in_program ? SECTION_ID_PROGRAM_STREAM_TAGS : SECTION_ID_STREAM_TAGS); if (stream->nb_side_data) { - print_pkt_side_data(w, stream->side_data, stream->nb_side_data, + print_pkt_side_data(w, stream->codecpar, stream->side_data, stream->nb_side_data, SECTION_ID_STREAM_SIDE_DATA_LIST, SECTION_ID_STREAM_SIDE_DATA); } diff --git a/libavformat/dump.c b/libavformat/dump.c index d9aa3af..505d572 100644 --- a/libavformat/dump.c +++ b/libavformat/dump.c @@ -343,7 +343,7 @@ static void dump_mastering_display_metadata(void *ctx, AVPacketSideData* sd) { av_q2d(metadata->min_luminance), av_q2d(metadata->max_luminance)); } -static void dump_spherical(void *ctx, AVPacketSideData *sd) +static void dump_spherical(void *ctx, AVCodecParameters *par, AVPacketSideData *sd) { AVSphericalMapping *spherical = (AVSphericalMapping *)sd->data; double yaw, pitch, roll; @@ -357,6 +357,8 @@ static void dump_spherical(void *ctx, AVPacketSideData *sd) av_log(ctx, AV_LOG_INFO, "equirectangular "); else if (spherical->projection == AV_SPHERICAL_CUBEMAP) av_log(ctx, AV_LOG_INFO, "cubemap "); + else if (spherical->projection == AV_SPHERICAL_EQUIRECTANGULAR_TILE) + av_log(ctx, AV_LOG_INFO, "tiled equirectangular "); else { av_log(ctx, AV_LOG_WARNING, "unknown"); return; @@ -366,6 +368,15 @@ static void dump_spherical(void *ctx, AVPacketSideData *sd) pitch = ((double)spherical->pitch) / (1 << 16); roll = ((double)spherical->roll) / (1 << 16); av_log(ctx, AV_LOG_INFO, "(%f/%f/%f) ", yaw, pitch, roll); + + if (spherical->projection == AV_SPHERICAL_EQUIRECTANGULAR_TILE) { + size_t l, t, r, b; + av_spherical_tile_bounds(spherical, par->width, par->height, + &l, &t, &r, &b); + av_log(ctx, AV_LOG_INFO, "[%zu, %zu, %zu, %zu] ", l, t, r, b); + } else if (spherical->projection == AV_SPHERICAL_CUBEMAP) { + av_log(ctx, AV_LOG_INFO, "[pad %zu] ", spherical->padding); + } } static void dump_sidedata(void *ctx, AVStream *st, const char *indent) @@ -421,7 +432,7 @@ static void dump_sidedata(void *ctx, AVStream *st, const char *indent) break; case AV_PKT_DATA_SPHERICAL: av_log(ctx, AV_LOG_INFO, "spherical: "); - dump_spherical(ctx, &sd); + dump_spherical(ctx, st->codecpar, &sd); break; default: av_log(ctx, AV_LOG_INFO, diff --git a/libavutil/spherical.c b/libavutil/spherical.c index 816452c..f3d5cc9 100644 --- a/libavutil/spherical.c +++ b/libavutil/spherical.c @@ -32,3 +32,21 @@ AVSphericalMapping *av_spherical_alloc(size_t *size) return spherical; } + +void av_spherical_tile_bounds(AVSphericalMapping *map, + size_t width, size_t height, + size_t *left, size_t *top, + size_t *right, size_t *bottom) +{ + /* conversion from 0.32 coordinates to pixels */ + size_t orig_width = (size_t) width * UINT32_MAX / + (UINT32_MAX - map->bound_right - map->bound_left); + size_t orig_height = (size_t) height * UINT32_MAX / + (UINT32_MAX - map->bound_bottom - map->bound_top); + + /* add a (UINT32_MAX - 1) to round up integer division */ + *left = (orig_width * map->bound_left + UINT32_MAX - 1) / UINT32_MAX; + *top = (orig_height * map->bound_top + UINT32_MAX - 1) / UINT32_MAX; + *right = orig_width - width - *left; + *bottom = orig_height - height - *top; +} diff --git a/libavutil/spherical.h b/libavutil/spherical.h index eeda625..db9bdc0 100644 --- a/libavutil/spherical.h +++ b/libavutil/spherical.h @@ -63,6 +63,13 @@ enum AVSphericalProjection { * to the back. */ AV_SPHERICAL_CUBEMAP, + + /** + * Video represents a portion of a sphere mapped on a flat surface + * using equirectangular projection. The @ref bounding fields indicate + * the position of the current video in a larger surface. + */ + AV_SPHERICAL_EQUIRECTANGULAR_TILE, }; /** @@ -122,6 +129,57 @@ typedef struct AVSphericalMapping { /** * @} */ + + /** + * @name Bounding rectangle + * @anchor bounding + * @{ + * These fields indicate the location of the current tile, and where + * it should be mapped relative to the original surface. They are + * exported as 0.32 fixed point, and can be converted to classic + * pixel values with av_spherical_bounds(). + * + * @code{.unparsed} + * +----------------+----------+ + * | |bound_top | + * | +--------+ | + * | bound_left |tile | | + * +<---------->| |<--->+bound_right + * | +--------+ | + * | | | + * | bound_bottom| | + * +----------------+----------+ + * @endcode + * + * If needed, the original video surface dimensions can be derived + * by adding the current stream or frame size to the related bounds, + * like in the following example: + * + * @code{c} + * original_width = tile->width + bound_left + bound_right; + * original_height = tile->height + bound_top + bound_bottom; + * @endcode + * + * @note These values are valid only for the tiled equirectangular + * projection type (@ref AV_SPHERICAL_EQUIRECTANGULAR_TILE), + * and should be ignored in all other cases. + */ + size_t bound_left; ///< Distance from the left edge + size_t bound_top; ///< Distance from the top edge + size_t bound_right; ///< Distance from the right edge + size_t bound_bottom; ///< Distance from the bottom edge + /** + * @} + */ + + /** + * Number of pixels to pad from the edge of each cube face. + * + * @note This value is valid for only for the cubemap projection type + * (@ref AV_SPHERICAL_CUBEMAP), and should be ignored in all other + * cases. + */ + size_t padding; } AVSphericalMapping; /** @@ -133,6 +191,22 @@ typedef struct AVSphericalMapping { AVSphericalMapping *av_spherical_alloc(size_t *size); /** + * Convert the @ref bounding fields from an AVSphericalVideo + * from 0.32 fixed point to pixels. + * + * @param map The AVSphericalVideo map to read bound values from. + * @param width Width of the current frame or stream. + * @param height Height of the current frame or stream. + * @param left Pixels from the left edge. + * @param top Pixels from the top edge. + * @param right Pixels from the right edge. + * @param bottom Pixels from the bottom edge. + */ +void av_spherical_tile_bounds(AVSphericalMapping *map, + size_t width, size_t height, + size_t *left, size_t *top, + size_t *right, size_t *bottom); +/** * @} * @} */ diff --git a/libavutil/version.h b/libavutil/version.h index a8b00bf..4d5a405 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 55 -#define LIBAVUTIL_VERSION_MINOR 47 +#define LIBAVUTIL_VERSION_MINOR 48 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ diff --git a/tests/ref/fate/matroska-spherical-mono b/tests/ref/fate/matroska-spherical-mono index 9f4b4f8..8048aff 100644 --- a/tests/ref/fate/matroska-spherical-mono +++ b/tests/ref/fate/matroska-spherical-mono @@ -7,7 +7,7 @@ inverted=0 [/SIDE_DATA] [SIDE_DATA] side_data_type=Spherical Mapping -side_data_size=16 +side_data_size=56 projection=equirectangular yaw=45 pitch=30 diff --git a/tests/ref/fate/mov-spherical-mono b/tests/ref/fate/mov-spherical-mono index 9f4b4f8..8048aff 100644 --- a/tests/ref/fate/mov-spherical-mono +++ b/tests/ref/fate/mov-spherical-mono @@ -7,7 +7,7 @@ inverted=0 [/SIDE_DATA] [SIDE_DATA] side_data_type=Spherical Mapping -side_data_size=16 +side_data_size=56 projection=equirectangular yaw=45 pitch=30