From patchwork Tue Dec 6 18:05:46 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 1698 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.65.86 with SMTP id o83csp2297028vsa; Tue, 6 Dec 2016 10:06:03 -0800 (PST) X-Received: by 10.194.203.5 with SMTP id km5mr64056209wjc.230.1481047563257; Tue, 06 Dec 2016 10:06:03 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id db1si20708385wjb.153.2016.12.06.10.06.02; Tue, 06 Dec 2016 10:06:03 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 59858689FD4; Tue, 6 Dec 2016 20:05:50 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt0-f174.google.com (mail-qt0-f174.google.com [209.85.216.174]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7B810689E01 for ; Tue, 6 Dec 2016 20:05:44 +0200 (EET) Received: by mail-qt0-f174.google.com with SMTP id p16so353213404qta.0 for ; Tue, 06 Dec 2016 10:05:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:references:from:to:cc:message-id:date:user-agent :mime-version:in-reply-to; bh=QLfBQKdRsANsWCqgS4YT5NbPCg+znxOLX2T+cISoaDw=; b=KtxN/s2X1RIntqwPytnbzSmbM79r6pSbjrFCXr6k8NOrrQGUzSyhfKKsX5PHbfK6pm yKjTYeorxPa5AdCJNpvB4/ft3lwbZWe6FdPjUNfBl6bsipEr31euOXrZ85u1FHnX/P2J DnDM75d97nt0vFNKsxKhHL5nWJPltSOYxHBzdUYsRJUhcVfwa7ITGTIh29jITOO3C+4U UbwgTgslhHgfIDQgZxPoRHbSZ6SLD7kX3VTjuO9xCJ+uDxmftqyZR2tMp/qoVIljX6kp xGjpRnv/F/pfeqd0wL+BXoaqAHt92fqIPPi4pfZqdYapzFa/5a87By310AvmSdVcZ3rY nIEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:references:from:to:cc:message-id:date :user-agent:mime-version:in-reply-to; bh=QLfBQKdRsANsWCqgS4YT5NbPCg+znxOLX2T+cISoaDw=; b=PJRB4jsrwZ/buPmoebgNdJrK98jCa8hfICZsi+zwMU4FzxGq5OZiXXjoLwl9+VkvHl wkuXhykGgpW5eprTd/gKwBRmR60rjZIjs2UoyGq86x8BgVjdOa4N1dS3BqLkw2HZgqfL KpxoB1H1eBKnDiZpbPdAdkD8SlPEs9igm1PcUoJdcndmxOGBRdxHjMT9rfbDgNesOaR1 gve+dMOFHvqNFyD4CpfoODytmuoDFglvLFa58ia9V83MQU0YlbddO7q90sxqsJIx+f6a 0gLpOnH0hdV/f1N5cC5rkN16R2fIx4aFXyb06lGOiYMfTXkjLYlLMgFj0yazIIi2Avot B63g== X-Gm-Message-State: AKaTC00KOlJjme0y0hGDwhAPpcBEo+F+z4CM45Inn6fYhFqe+dwE87koWpjo5OUyEjcYWw== X-Received: by 10.237.33.240 with SMTP id m45mr55957148qtc.250.1481047553913; Tue, 06 Dec 2016 10:05:53 -0800 (PST) Received: from [192.168.1.33] ([181.22.22.219]) by smtp.googlemail.com with ESMTPSA id g13sm12659631qtg.15.2016.12.06.10.05.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Dec 2016 10:05:53 -0800 (PST) References: <20161130233642.77815-1-vittorio.giovara@gmail.com> <20161130233642.77815-3-vittorio.giovara@gmail.com> <4194b7d3-f34d-8711-a1ca-89e50825cad2@gmail.com> <5a31aff0-566a-a37c-6eaa-8f321977dab2@gmail.com> From: James Almer To: FFmpeg development discussions and patches Message-ID: <63e37be9-b967-d25f-8a80-c39be11df000@gmail.com> Date: Tue, 6 Dec 2016 15:05:46 -0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Subject: Re: [FFmpeg-devel] [PATCH 3/4] mov: Export spherical information X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vittorio Giovara Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" On 12/6/2016 2:37 PM, Vittorio Giovara wrote: > On Tue, Dec 6, 2016 at 11:06 AM, James Almer wrote: >> On 11/30/2016 9:12 PM, James Almer wrote: >>> On 11/30/2016 8:36 PM, Vittorio Giovara wrote: >>>> This implements Spherical Video V1 and V2, as described in the >>>> spatial-media collection by Google. >>>> >>>> Signed-off-by: Vittorio Giovara >>>> --- >>>> This addresses all comments from James. >>>> Vittorio >>> >>> LGTM, at least the V2 part. Seems to follow the spec right now. >>> Will trust you on the V1 part :P. > > Hopefully V1 is going to disappear soon. Thanks for the review, I've > sent a minor documentation update which reflects the latest change to > the specification, I hope that is fine too. > >>> I have written a Matroska implementation of this spec, for that >>> matter. Will send it after this patchset is committed. > > Oh cool, do you have or need a file to test (and/or add a fate)? I wrote a muxer implementation and remuxed the mov file currently in the FATE samples suite to test the demuxer implementation. You can download it here: http://0x0.st/Lpj.mkv I wont be sending the muxer implementation since i had to hack in the projection specific private data to create that file, as it's not part of AVSphericalMapping. I could use an actual file created by Google to make sure i got it right, but the spec is very clear and our Matroska de/muxers allow one to add new elements with a few lines. I'm attaching the demuxer implementation if you want to look at it and test it. From d29ead20c57011c141acd6bb522bedeed2755f1c Mon Sep 17 00:00:00 2001 From: James Almer Date: Tue, 6 Dec 2016 14:48:45 -0300 Subject: [PATCH] avformat/matroskadec: add support for Spherical Video elements Signed-off-by: James Almer --- libavformat/matroska.h | 14 +++++++++++ libavformat/matroskadec.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 76 insertions(+) diff --git a/libavformat/matroska.h b/libavformat/matroska.h index 13155e5..83c8246 100644 --- a/libavformat/matroska.h +++ b/libavformat/matroska.h @@ -153,6 +153,13 @@ #define MATROSKA_ID_VIDEOCOLOR_LUMINANCEMAX 0x55D9 #define MATROSKA_ID_VIDEOCOLOR_LUMINANCEMIN 0x55DA +#define MATROSKA_ID_VIDEOPROJECTION 0x7670 +#define MATROSKA_ID_VIDEOPROJECTIONTYPE 0x7671 +#define MATROSKA_ID_VIDEOPROJECTIONPRIVATE 0x7672 +#define MATROSKA_ID_VIDEOPROJECTIONPOSEYAW 0x7673 +#define MATROSKA_ID_VIDEOPROJECTIONPOSEPITCH 0x7674 +#define MATROSKA_ID_VIDEOPROJECTIONPOSEROLL 0x7675 + /* IDs in the trackaudio master */ #define MATROSKA_ID_AUDIOSAMPLINGFREQ 0xB5 #define MATROSKA_ID_AUDIOOUTSAMPLINGFREQ 0x78B5 @@ -331,6 +338,13 @@ typedef enum { MATROSKA_COLOUR_CHROMASITINGVERT_NB } MatroskaColourChromaSitingVert; +typedef enum { + MATROSKA_VIDEO_PROJECTION_TYPE_RECTANGULAR = 0, + MATROSKA_VIDEO_PROJECTION_TYPE_EQUIRECTANGULAR = 1, + MATROSKA_VIDEO_PROJECTION_TYPE_CUBEMAP = 2, + MATROSKA_VIDEO_PROJECTION_TYPE_MESH = 3, +} MatroskaVideoProjectionType; + /* * Matroska Codec IDs, strings */ diff --git a/libavformat/matroskadec.c b/libavformat/matroskadec.c index 017a533..f4a452e 100644 --- a/libavformat/matroskadec.c +++ b/libavformat/matroskadec.c @@ -43,6 +43,7 @@ #include "libavutil/mathematics.h" #include "libavutil/opt.h" #include "libavutil/time_internal.h" +#include "libavutil/spherical.h" #include "libavcodec/bytestream.h" #include "libavcodec/flac.h" @@ -161,6 +162,14 @@ typedef struct MatroskaTrackVideoColor { MatroskaMasteringMeta mastering_meta; } MatroskaTrackVideoColor; +typedef struct MatroskaTrackVideoProjection { + uint64_t type; + EbmlBin private; + double yaw; + double pitch; + double roll; +} MatroskaTrackVideoProjection; + typedef struct MatroskaTrackVideo { double frame_rate; uint64_t display_width; @@ -174,6 +183,7 @@ typedef struct MatroskaTrackVideo { uint64_t stereo_mode; uint64_t alpha_mode; MatroskaTrackVideoColor color; + MatroskaTrackVideoProjection projection; } MatroskaTrackVideo; typedef struct MatroskaTrackAudio { @@ -424,6 +434,15 @@ static const EbmlSyntax matroska_track_video_color[] = { { 0 } }; +static const EbmlSyntax matroska_track_video_projection[] = { + { MATROSKA_ID_VIDEOPROJECTIONTYPE, EBML_UINT, 0, offsetof(MatroskaTrackVideoProjection, type), { .u = MATROSKA_VIDEO_PROJECTION_TYPE_RECTANGULAR } }, + { MATROSKA_ID_VIDEOPROJECTIONPRIVATE, EBML_BIN, 0, offsetof(MatroskaTrackVideoProjection, private) }, + { MATROSKA_ID_VIDEOPROJECTIONPOSEYAW, EBML_FLOAT, 0, offsetof(MatroskaTrackVideoProjection, yaw), { .f=0.0 } }, + { MATROSKA_ID_VIDEOPROJECTIONPOSEPITCH, EBML_FLOAT, 0, offsetof(MatroskaTrackVideoProjection, pitch), { .f=0.0 } }, + { MATROSKA_ID_VIDEOPROJECTIONPOSEROLL, EBML_FLOAT, 0, offsetof(MatroskaTrackVideoProjection, roll), { .f=0.0 } }, + { 0 } +}; + static const EbmlSyntax matroska_track_video[] = { { MATROSKA_ID_VIDEOFRAMERATE, EBML_FLOAT, 0, offsetof(MatroskaTrackVideo, frame_rate) }, { MATROSKA_ID_VIDEODISPLAYWIDTH, EBML_UINT, 0, offsetof(MatroskaTrackVideo, display_width), { .u=-1 } }, @@ -433,6 +452,7 @@ static const EbmlSyntax matroska_track_video[] = { { MATROSKA_ID_VIDEOCOLORSPACE, EBML_BIN, 0, offsetof(MatroskaTrackVideo, color_space) }, { MATROSKA_ID_VIDEOALPHAMODE, EBML_UINT, 0, offsetof(MatroskaTrackVideo, alpha_mode) }, { MATROSKA_ID_VIDEOCOLOR, EBML_NEST, 0, offsetof(MatroskaTrackVideo, color), { .n = matroska_track_video_color } }, + { MATROSKA_ID_VIDEOPROJECTION, EBML_NEST, 0, offsetof(MatroskaTrackVideo, projection), { .n = matroska_track_video_projection } }, { MATROSKA_ID_VIDEOPIXELCROPB, EBML_NONE }, { MATROSKA_ID_VIDEOPIXELCROPT, EBML_NONE }, { MATROSKA_ID_VIDEOPIXELCROPL, EBML_NONE }, @@ -1879,6 +1899,44 @@ static int mkv_parse_video_color(AVStream *st, const MatroskaTrack *track) { return 0; } +static int mkv_parse_video_projection(AVStream *st, const MatroskaTrack *track) { + AVSphericalMapping *spherical; + enum AVSphericalProjection projection; + size_t spherical_size; + int ret; + + switch (track->video.projection.type) { + case MATROSKA_VIDEO_PROJECTION_TYPE_EQUIRECTANGULAR: + if (track->video.projection.private.size < 4) + return AVERROR_INVALIDDATA; + projection = AV_SPHERICAL_EQUIRECTANGULAR; + break; + case MATROSKA_VIDEO_PROJECTION_TYPE_CUBEMAP: + if (track->video.projection.private.size < 4) + return AVERROR_INVALIDDATA; + projection = AV_SPHERICAL_CUBEMAP; + break; + default: + return 0; + } + + spherical = av_spherical_alloc(&spherical_size); + if (!spherical) + return AVERROR(ENOMEM); + spherical->projection = projection; + + spherical->yaw = (int32_t)(track->video.projection.yaw * (1 << 16)); + spherical->pitch = (int32_t)(track->video.projection.pitch * (1 << 16)); + spherical->roll = (int32_t)(track->video.projection.roll * (1 << 16)); + + ret = av_stream_add_side_data(st, AV_PKT_DATA_SPHERICAL, (uint8_t *)spherical, + spherical_size); + if (ret < 0) + return ret; + + return 0; +} + static int get_qt_codec(MatroskaTrack *track, uint32_t *fourcc, enum AVCodecID *codec_id) { const AVCodecTag *codec_tags; @@ -2361,6 +2419,10 @@ static int matroska_parse_tracks(AVFormatContext *s) if (ret < 0) return ret; } + + ret = mkv_parse_video_projection(st, track); + if (ret < 0) + return ret; } else if (track->type == MATROSKA_TRACK_TYPE_AUDIO) { st->codecpar->codec_type = AVMEDIA_TYPE_AUDIO; st->codecpar->codec_tag = fourcc; -- 2.10.2