From patchwork Mon Jun 17 13:41:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derek Buitenhuis X-Patchwork-Id: 49977 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:9196:0:b0:460:55fa:d5ed with SMTP id s22csp1956663vqg; Mon, 17 Jun 2024 06:49:17 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXUSSZe6f/2/R+NdKhbkN0a2Art5RWEHv3XGQ9dzgh/Ku2nYIvYYXMlf76rUrh9vo3J3ejxphz0LDfbulf2FXtgDgHu4A48puY0bg== X-Google-Smtp-Source: AGHT+IF7/g+2t1k2I3Hjd469rLUiQNfH3uRoIIH8OjTagMyWDnOOdIo/5Xsx4QNQLN8zeYDxWHr4 X-Received: by 2002:a17:906:fb11:b0:a6f:25d9:e35b with SMTP id a640c23a62f3a-a6f60de61c0mr644438466b.65.1718632157266; Mon, 17 Jun 2024 06:49:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718632157; cv=none; d=google.com; s=arc-20160816; b=sOsWUOsjymg5kBCDM9y/cUtlB/xj64H4shymwiesmLcsOA7eEWtXOUTmcMKQ1RN24V vyNEHtb7ToAsAbBf7oxNnlLuk4ONqC4vajWFt8fATav28QHVQkDOwR/AbYBky6rqZ6Dp l0tFWxq9excSfJqKKp8OiDVSCan3NpU+fj1Li7UC6To9ypAaw1NVuaopGsHZLcRwlipv Lj23uAZ4G60eyBlLO3xnRrpSDmRBgnHNUoPB7PCAUdneuVE7OcCMJJbLhyazQBdla7K4 etGeXbbNvjIcAMDAGzfID+r7XASKNltkEZvkkzyxzKoxzMO+e3/KmuZOIY7e5llqpze0 1QFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=7mGEBR/2P4n9YKvsJeL5DNceoIqV5mSm9Eg1ucoFp3c=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=hECM8/+xUhPDZolR0BVCUoq4mGGDCMPtskNwiDtdkRGpKOPiSY0G3shKeiTHu/kS9w q1y+RqEAfmGRx/qxtu2KgiMFrnlBWOG5+xh3W5Nb7r5BZfCPJDaUekJOTBRjYWqZydsN +P+9DX6U/qIQcGAyRhOk1sj+3W/UpAnk92LEzMlmJCq+lyJXT9O2kiqX29QnGj5eMyfy N9ZZSQeo22nKwBACeVHNeWLFf+pniIFafBM7JRvTLLgS1ToGVWBNy3vLEniixKTH/200 t1W7Zu37L1GmcjYMreFlBvnZCR/ajlMfiktEarXADE/phEI1i4UaEPvd8wY5grmz8AJe ReOg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=Vk81Q49a; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a6f56e87c95si429970066b.1055.2024.06.17.06.49.16; Mon, 17 Jun 2024 06:49:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=Vk81Q49a; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E7B1E68D775; Mon, 17 Jun 2024 16:42:05 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4532168D5A7 for ; Mon, 17 Jun 2024 16:41:57 +0300 (EEST) Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-42278f3aea4so38300375e9.1 for ; Mon, 17 Jun 2024 06:41:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718631716; x=1719236516; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=a8LiGBSPNB1WFjlqDk8nsm9FlV+Z4iQ8LBZbQ/67Znk=; b=Vk81Q49appXs2oKq/Orpjyx/4IfOwtb7ba5yYi9ZQLHUU6hhgW8mmf6bpCYI1ggdbO cdI3Bx8eq6JZg5RElTmMbtaMR8QabDM5eesYX4XhzL6dASdJE0zgvBbVo8/luraRayK2 7fwGFd2F6AL9WUdCKMAdAAlA4hGAvsfmoBHmSaH+UnAN4lsYUEBBqv79x4r5mlQuztEg 9MbJYJQXIyObuGrqwWOtm3KrdRRoTCIQAUldUzTycUZ9vpuMVaXtuZS+fYYqdvcxI6pK NaqaPKMQag1F+fL1gwmiwYFeZjAs6rhnBBcC0uWynvqUPNxF+Z0uykt8dcJLRK6rgGkZ SymA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718631716; x=1719236516; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a8LiGBSPNB1WFjlqDk8nsm9FlV+Z4iQ8LBZbQ/67Znk=; b=nSpQm/1wv0Z+zAZNl8xk5m/il3hashla+JQzLsPmEgj1G6P+ceSqbvoVGiHp1Hu8Ug aqqTOFF2vOItfOktGNgvFgZEiCpkkP1zoEftub5L242gIWU/dyWe/ai7ehG/QQbGaKU6 eYq6mlYkE82pklofNPC+3TH0EsuJ33Q9uq5C+a7chOVDAkITxczD+hpsUH19gNw+KD42 dKvln92ojKT8vFFZ2RUu+tNQIlnHM+Ggwmd+safq5T8hj0L8SM4WBy5PTmIqSifXsQxw RVBrSq6LZRvX8eJah+xUTabROW3ByQlJ0ZGPhutMCXFkRPAT2Hg3u8Nhlcuemv99kvMS gEJQ== X-Gm-Message-State: AOJu0YyhLO/vKCHeHOW7cGegD7RBz13C8cbXY7B0COwwrD/y5K/WRPSF wYCxTGeVKPuI34SeOTuT88uCScRcaf+EklyLcKtec6AjYZsFQrxmRVu1xg== X-Received: by 2002:a05:600c:705:b0:424:6c83:a78e with SMTP id 5b1f17b1804b1-4246c83a808mr10322735e9.40.1718631715583; Mon, 17 Jun 2024 06:41:55 -0700 (PDT) Received: from localhost.localdomain (0540dce9.skybroadband.com. [5.64.220.233]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3607509c790sm11915391f8f.38.2024.06.17.06.41.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jun 2024 06:41:54 -0700 (PDT) From: Derek Buitenhuis To: ffmpeg-devel@ffmpeg.org Date: Mon, 17 Jun 2024 14:41:40 +0100 Message-ID: <20240617134141.224614-5-derek.buitenhuis@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240617134141.224614-1-derek.buitenhuis@gmail.com> References: <20240617134141.224614-1-derek.buitenhuis@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 4/5] avformat/mov: Add support for exporting Video Extension Usage info X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9E0dIK2cxk0X This box is provided by files created by the Apple Vision Pro, as well as the iPhone 15+ when capture for Vision Pro is enabled. The boxes are a mix of things documented by Apple in some PDFs, their API docs, and reverse engineering. Ideally we will have a real spec one day. Links: * https://developer.apple.com/av-foundation/Stereo-Video-ISOBMFF-Extensions.pdf * https://developer.apple.com/documentation/videotoolbox/kvtcompressionpropertykey_horizontaldisparityadjustment * https://developer.apple.com/documentation/videotoolbox/kvtcompressionpropertykey_stereocamerabaseline * https://developer.apple.com/documentation/videotoolbox/kvtcompressionpropertykey_heroeye Signed-off-by: Derek Buitenhuis --- libavformat/mov.c | 279 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 279 insertions(+) diff --git a/libavformat/mov.c b/libavformat/mov.c index 9016cd5ad0..a7ca0b5a3c 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -6477,6 +6477,284 @@ static int mov_read_sv3d(MOVContext *c, AVIOContext *pb, MOVAtom atom) return 0; } +static int mov_read_vexu_proj(MOVContext *c, AVIOContext *pb, MOVAtom atom) +{ + AVStream *st; + MOVStreamContext *sc; + int size; + uint32_t tag; + enum AVSphericalProjection projection; + + if (c->fc->nb_streams < 1) + return 0; + + st = c->fc->streams[c->fc->nb_streams - 1]; + sc = st->priv_data; + + if (atom.size != 16) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size for proj box: %"PRIu64"\n", atom.size); + return AVERROR_INVALIDDATA; + } + + size = avio_rb32(pb); + if (size != 16) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size for prji box: %d\n", size); + return AVERROR_INVALIDDATA; + } + + tag = avio_rl32(pb); + if (tag != MKTAG('p','r','j','i')) { + av_log(c->fc, AV_LOG_ERROR, "Invalid child box of proj box: 0x%08X\n", tag); + return AVERROR_INVALIDDATA; + } + + avio_skip(pb, 1); // version + avio_skip(pb, 3); // flags + + tag = avio_rl32(pb); + switch (tag) { + case MKTAG('r','e','c','t'): + projection = AV_SPHERICAL_RECTANGULAR; + break; + case MKTAG('e','q','u','i'): + projection = AV_SPHERICAL_EQUIRECTANGULAR; + break; + case MKTAG('h','e','q','u'): + projection = AV_SPHERICAL_HALF_EQUIRECTANGULAR; + break; + case MKTAG('f','i','s','h'): + projection = AV_SPHERICAL_FISHEYE; + break; + default: + av_log(c->fc, AV_LOG_ERROR, "Invalid projection type in prji box: 0x%08X\n", tag); + return AVERROR_INVALIDDATA; + } + + sc->spherical = av_spherical_alloc(&sc->spherical_size); + if (!sc->spherical) + return AVERROR(ENOMEM); + + sc->spherical->projection = projection; + + return 0; +} + +static int mov_read_eyes(MOVContext *c, AVIOContext *pb, MOVAtom atom) +{ + AVStream *st; + MOVStreamContext *sc; + int size, flags = 0; + int64_t remaining; + uint32_t tag, baseline = 0; + enum AVStereo3DView view = AV_STEREO3D_VIEW_PACKED; + enum AVStereo3DPrimaryEye primary_eye = AV_PRIMARY_EYE_NONE; + int32_t horizontal_disparity_adjustment = 0; + + if (c->fc->nb_streams < 1) + return 0; + + st = c->fc->streams[c->fc->nb_streams - 1]; + sc = st->priv_data; + + remaining = atom.size; + while (remaining > 0) { + size = avio_rb32(pb); + if (size < 8 || size > remaining ) { + av_log(c->fc, AV_LOG_ERROR, "Invalid child size in eyes box\n"); + return AVERROR_INVALIDDATA; + } + + tag = avio_rl32(pb); + switch (tag) { + case MKTAG('s','t','r','i'): { + int has_right, has_left; + uint8_t tmp; + if (size != 13) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size of stri box: %d\n", size); + return AVERROR_INVALIDDATA; + } + avio_skip(pb, 1); // version + avio_skip(pb, 3); // flags + + tmp = avio_r8(pb); + + // eye_views_reversed + if (tmp & 8) { + flags |= AV_STEREO3D_FLAG_INVERT; + } + // has_additional_views + if (tmp & 4) { + // skip... + } + + has_right = tmp & 2; // has_right_eye_view + has_left = tmp & 1; // has_left_eye_view + + if (has_left && has_right) + view = AV_STEREO3D_VIEW_PACKED; + else if (has_left) + view = AV_STEREO3D_VIEW_LEFT; + else if (has_right) + view = AV_STEREO3D_VIEW_RIGHT; + break; + } + case MKTAG('h','e','r','o'): { + int tmp; + if (size != 13) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size of hero box: %d\n", size); + return AVERROR_INVALIDDATA; + } + avio_skip(pb, 1); // version + avio_skip(pb, 3); // flags + + tmp = avio_r8(pb); + if (tmp == 0) + primary_eye = AV_PRIMARY_EYE_NONE; + else if (tmp == 1) + primary_eye = AV_PRIMARY_EYE_LEFT; + else if (tmp == 2) + primary_eye = AV_PRIMARY_EYE_RIGHT; + else + av_log(c->fc, AV_LOG_WARNING, "Unknown hero eye type: %d\n", tmp); + + break; + } + case MKTAG('c','a','m','s'): { + uint32_t subtag; + int subsize; + if (size != 24) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size of cams box: %d\n", size); + return AVERROR_INVALIDDATA; + } + + subsize = avio_rb32(pb); + if (subsize != 16) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size of blin box: %d\n", size); + return AVERROR_INVALIDDATA; + } + + subtag = avio_rl32(pb); + if (subtag != MKTAG('b','l','i','n')) { + av_log(c->fc, AV_LOG_ERROR, "Expected blin box, got 0x%08X\n", subtag); + return AVERROR_INVALIDDATA; + } + + avio_skip(pb, 1); // version + avio_skip(pb, 3); // flags + + baseline = avio_rb32(pb); + + break; + } + case MKTAG('c','m','f','y'): { + uint32_t subtag; + int subsize; + if (size != 24) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size of cmfy box: %d\n", size); + return AVERROR_INVALIDDATA; + } + + subsize = avio_rb32(pb); + if (subsize != 16) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size of dadj box: %d\n", size); + return AVERROR_INVALIDDATA; + } + + subtag = avio_rl32(pb); + if (subtag != MKTAG('d','a','d','j')) { + av_log(c->fc, AV_LOG_ERROR, "Expected dadj box, got 0x%08X\n", subtag); + return AVERROR_INVALIDDATA; + } + + avio_skip(pb, 1); // version + avio_skip(pb, 3); // flags + + horizontal_disparity_adjustment = (int32_t) avio_rb32(pb); + + break; + } + default: + av_log(c->fc, AV_LOG_WARNING, "Unknown tag in eyes: 0x%08X\n", tag); + avio_skip(pb, size - 8); + break; + } + remaining -= size; + } + + if (remaining != 0) { + av_log(c->fc, AV_LOG_ERROR, "Broken eyes box\n"); + return AVERROR_INVALIDDATA; + } + + if (!sc->stereo3d) { + sc->stereo3d = av_stereo3d_alloc(); + if (!sc->stereo3d) + return AVERROR(ENOMEM); + } + + sc->stereo3d->flags = flags; + sc->stereo3d->view = view; + sc->stereo3d->primary_eye = primary_eye; + sc->stereo3d->baseline = baseline; + sc->stereo3d->horizontal_disparity_adjustment = horizontal_disparity_adjustment; + + return 0; +} + +static int mov_read_vexu(MOVContext *c, AVIOContext *pb, MOVAtom atom) +{ + int size; + int64_t remaining; + uint32_t tag; + + if (c->fc->nb_streams < 1) + return 0; + + if (atom.size < 8) { + av_log(c->fc, AV_LOG_ERROR, "Empty video extension usage box\n"); + return AVERROR_INVALIDDATA; + } + + remaining = atom.size; + while (remaining > 0) { + size = avio_rb32(pb); + if (size < 8 || size > remaining ) { + av_log(c->fc, AV_LOG_ERROR, "Invalid child size in vexu box\n"); + return AVERROR_INVALIDDATA; + } + + tag = avio_rl32(pb); + switch (tag) { + case MKTAG('p','r','o','j'): { + MOVAtom proj = { tag, size - 8 }; + int ret = mov_read_vexu_proj(c, pb, proj); + if (ret < 0) + return ret; + break; + } + case MKTAG('e','y','e','s'): { + MOVAtom eyes = { tag, size - 8 }; + int ret = mov_read_eyes(c, pb, eyes); + if (ret < 0) + return ret; + break; + } + default: + av_log(c->fc, AV_LOG_WARNING, "Unknown tag in vexu: 0x%08X\n", tag); + avio_skip(pb, size - 8); + break; + } + remaining -= size; + } + + if (remaining != 0) { + av_log(c->fc, AV_LOG_ERROR, "Broken vexu box\n"); + return AVERROR_INVALIDDATA; + } + + return 0; +} + static int mov_parse_uuid_spherical(MOVStreamContext *sc, AVIOContext *pb, size_t len) { int ret = 0; @@ -8595,6 +8873,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('d','f','L','a'), mov_read_dfla }, { MKTAG('s','t','3','d'), mov_read_st3d }, /* stereoscopic 3D video box */ { MKTAG('s','v','3','d'), mov_read_sv3d }, /* spherical video box */ +{ MKTAG('v','e','x','u'), mov_read_vexu }, /* video extension usage */ { MKTAG('d','O','p','s'), mov_read_dops }, { MKTAG('d','m','l','p'), mov_read_dmlp }, { MKTAG('S','m','D','m'), mov_read_smdm },