From patchwork Wed Nov 30 23:36:39 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vittorio Giovara X-Patchwork-Id: 1621 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.65.86 with SMTP id o83csp642423vsa; Wed, 30 Nov 2016 15:36:54 -0800 (PST) X-Received: by 10.194.108.10 with SMTP id hg10mr36598380wjb.58.1480549013971; Wed, 30 Nov 2016 15:36:53 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i128si9354666wme.137.2016.11.30.15.36.53; Wed, 30 Nov 2016 15:36:53 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 78BE8689F73; Thu, 1 Dec 2016 01:36:43 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk0-f195.google.com (mail-qk0-f195.google.com [209.85.220.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0CAB7689F73 for ; Thu, 1 Dec 2016 01:36:37 +0200 (EET) Received: by mail-qk0-f195.google.com with SMTP id x190so24072514qkb.0 for ; Wed, 30 Nov 2016 15:36:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:message-id:reply-to; bh=pyz+gGejKdKVY0sCm9mYoiAp+RlqoRBgCE3ko5vdnZ0=; b=xiBuydWpEXKqAKwLvYbhH+RY1QrtKJNyaBrbOQA+CmAyaneKnBZrk57P108iHoPVPA pBdcSHrNN/v4MXZJ7WcARBlUtsJXFqlIKs71kHaXcFshZPHLjlWLE3l5vJX5OunW+EGf T8FtPsaYeDaXsOoEQKLIm2hpI9v3KXfdkVfmI1ZZKE5kHlIMtuSAyHK1nKviN1xDGdVi pyBOyxTsjJWwD7Ph+sDUH6wvpfELiA3pt2PtLZjOfJkTcxsFvebS2LIjDEGNzD2j/aTt kFSK2BRyVDjkC213a7U/OJDx2a1ohXWNxITjxKUg1BmfXCCs6uAD9V4qqtz9wIz/ZKpF bm+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:reply-to; bh=pyz+gGejKdKVY0sCm9mYoiAp+RlqoRBgCE3ko5vdnZ0=; b=Tg2bcbYp62iKB3kQoM5Nm37SuicY6Jr67CZN/KCIDmJBWLAMyr76Yvd+dKdR3LqkJ1 1zW1ZKkS0nnYPOZbJmjeKOltQxvWx5k1+Un424Ow0z0ZtBZPz6cvkKvh+zD3o0SBIfhx btgSNu9UTUzHFPOdjDPiCswpqabNYvUO1ImSXLKqNX9RI6HfLLnL4UnhqYPTJUnoofj+ 9NlV8v9pINQk3UA75jgpzsUnqXh6aYG4/w2T4uZdgnZ2G+9cNFM7rnYTKI1Y2gKwbBau QGF4pF0fDjacUg4LLbvO8ZmPj6f1T9J6uqaF3XdBWtnzT0B38t8pExbQGvpEDUpIX3TS JeSg== X-Gm-Message-State: AKaTC025FsKsfwyQd/2oizviDjvB81YGxeYFMXs0CKGfq7OkpDn2zfGjBePXlUWEVPlXXg== X-Received: by 10.55.209.90 with SMTP id s87mr29108068qki.99.1480549003161; Wed, 30 Nov 2016 15:36:43 -0800 (PST) Received: from vimacbookpro.vimeows.com (nyv-exweb.iac.com. [216.112.252.10]) by smtp.gmail.com with ESMTPSA id p19sm34544412qte.23.2016.11.30.15.36.42 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 30 Nov 2016 15:36:42 -0800 (PST) From: Vittorio Giovara To: ffmpeg-devel@ffmpeg.org Date: Wed, 30 Nov 2016 18:36:39 -0500 Message-Id: <20161130233642.77815-1-vittorio.giovara@gmail.com> X-Mailer: git-send-email 2.10.0 Subject: [FFmpeg-devel] [PATCH 1/4 v4] lavu: Add AVSphericalMapping type and frame side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" While no decoder currently exports spherical information, this type represents a frame property that has to be passed through from container to frames. Signed-off-by: Vittorio Giovara --- Revised documentation of the new type, removed bounding rectangle values, as I misinterpreted what they actually did. Vittorio libavutil/Makefile | 2 + libavutil/avutil.h | 6 +++ libavutil/frame.h | 8 ++- libavutil/spherical.c | 34 +++++++++++++ libavutil/spherical.h | 133 ++++++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 182 insertions(+), 1 deletion(-) create mode 100644 libavutil/spherical.c create mode 100644 libavutil/spherical.h diff --git a/libavutil/Makefile b/libavutil/Makefile index c61ca9c..6474a30 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -63,6 +63,7 @@ HEADERS = adler32.h \ samplefmt.h \ sha.h \ sha512.h \ + spherical.h \ stereo3d.h \ threadmessage.h \ time.h \ @@ -140,6 +141,7 @@ OBJS = adler32.o \ samplefmt.o \ sha.o \ sha512.o \ + spherical.o \ stereo3d.o \ threadmessage.o \ time.o \ diff --git a/libavutil/avutil.h b/libavutil/avutil.h index 29dd830..e9aaa03 100644 --- a/libavutil/avutil.h +++ b/libavutil/avutil.h @@ -118,6 +118,12 @@ * * @} * + * @defgroup lavu_video Video related + * + * @{ + * + * @} + * * @defgroup lavu_audio Audio related * * @{ diff --git a/libavutil/frame.h b/libavutil/frame.h index 8e51361..b450092 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -120,7 +120,13 @@ enum AVFrameSideDataType { * The GOP timecode in 25 bit timecode format. Data format is 64-bit integer. * This is set on the first frame of a GOP that has a temporal reference of 0. */ - AV_FRAME_DATA_GOP_TIMECODE + AV_FRAME_DATA_GOP_TIMECODE, + + /** + * The data represents the AVSphericalMapping structure defined in + * libavutil/spherical.h. + */ + AV_FRAME_DATA_SPHERICAL, }; enum AVActiveFormatDescription { diff --git a/libavutil/spherical.c b/libavutil/spherical.c new file mode 100644 index 0000000..816452c --- /dev/null +++ b/libavutil/spherical.c @@ -0,0 +1,34 @@ +/* + * Copyright (c) 2016 Vittorio Giovara + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "mem.h" +#include "spherical.h" + +AVSphericalMapping *av_spherical_alloc(size_t *size) +{ + AVSphericalMapping *spherical = av_mallocz(sizeof(AVSphericalMapping)); + if (!spherical) + return NULL; + + if (size) + *size = sizeof(*spherical); + + return spherical; +} diff --git a/libavutil/spherical.h b/libavutil/spherical.h new file mode 100644 index 0000000..9a61cb3 --- /dev/null +++ b/libavutil/spherical.h @@ -0,0 +1,133 @@ +/* + * Copyright (c) 2016 Vittorio Giovara + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Spherical video + */ + +#ifndef AVUTIL_SPHERICAL_H +#define AVUTIL_SPHERICAL_H + +/** + * @addtogroup lavu_video + * @{ + * + * @defgroup lavu_video_spherical Spherical video mapping + * @{ + */ + +/** + * @addtogroup lavu_video_spherical + * A spherical video file contains surfaces that need to be mapped onto a + * sphere. Depending on how the frame was converted, a different distortion + * transformation or surface recomposition function needs to be applied before + * the video should be mapped and displayed. + */ + +/** + * Projection of the video surface(s) on a sphere. + */ +enum AVSphericalProjection { + /** + * Video represents a sphere mapped on a flat surface using + * equirectangular projection. + */ + AV_SPHERICAL_EQUIRECTANGULAR, + + /** + * Video frame is split into 6 faces of a cube, and arranged on a + * 3x2 layout. Faces are oriented upwards for the front, left, right, + * and back faces. The up face is oriented so the top of the face is + * forwards and the down face is oriented so the top of the face is + * to the back. + */ + AV_SPHERICAL_CUBEMAP, +}; + +/** + * This structure describes how to handle spherical videos, outlining + * information about projection, initial layout, and any other view modifier. + * + * @note The struct must be allocated with av_spherical_alloc() and + * its size is not a part of the public ABI. + */ +typedef struct AVSphericalMapping { + /** + * Projection type. + */ + enum AVSphericalProjection projection; + + /** + * @name Initial orientation + * @{ + * There fields describe additional rotations applied to the sphere after + * the video frame is mapped onto it. The sphere is rotated around the + * viewer, who remains stationary. + * + * A positive yaw moves the left half of the frame forward. + * A positive pitch moves the bottom half of the frame upward. + * A positive roll moves the left half of the frame downward. + * These values are exported as 16.16 fixed point. + * + * See this equirectangular projection as example: + * + * @code{.unparsed} + * Yaw + * -180 0 180 + * 90 +-------------+-------------+ 180 + * | | | up + * P | | | z| + * i | ^ | | + * t 0 +-------------X-------------+ 0 Roll 0|_____ right + * c | | | / x + * h | | | y/ + * | | | forward + * -90 +-------------+-------------+ -180 + * + * X - the default camera center + * ^ - the default up vector + * @endcode + * + * The order of transformation is always yaw, followed by pitch, and + * finally by roll. + */ + int32_t yaw; ///< Rotation around the up vector [-180, 180]. + int32_t pitch; ///< Rotation around the right vector [-90, 90]. + int32_t roll; ///< Rotation around the forward vector [-180, 180]. + /** + * @} + */ +} AVSphericalMapping; + +/** + * Allocate a AVSphericalVideo structure and initialize its fields to default + * values. + * + * @return the newly allocated struct or NULL on failure + */ +AVSphericalMapping *av_spherical_alloc(size_t *size); + +/** + * @} + * @} + */ + +#endif /* AVUTIL_SPHERICAL_H */