From patchwork Mon Jun 10 18:44:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derek Buitenhuis X-Patchwork-Id: 35139 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:c209:0:b0:460:55fa:d5ed with SMTP id d9csp2724388vqo; Mon, 10 Jun 2024 11:44:35 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUWNuMsVhRifnlnCtnoWZghjleM1nSuZtX1rtaIr3br4u7ENeC3VJNdEIgUZ5A2+dswuacEJ7HtROi6z82m2UQkxgsznhf24LVItw== X-Google-Smtp-Source: AGHT+IGgb7lWZkTWEDt8r1hq2nZVBmYQjuD0fklWsJVAQuaVDcbIkvqJ+esrrFyud6rlvTeMcdgi X-Received: by 2002:a17:906:698:b0:a68:cc6f:cb5a with SMTP id a640c23a62f3a-a6cdbd0e35bmr616922166b.68.1718045074974; Mon, 10 Jun 2024 11:44:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718045074; cv=none; d=google.com; s=arc-20160816; b=p5bkCtmoCEpv+kNcctTPLUmL7fSkuuAl7r7AyreFZ4nlL/EaeYQ3Y+a8o/zYCeitW9 SWxOSc7X5y99TL7/OlhrVgkg2Y8iinRnKmdysG1biBUJf5uUzWhFIrEV+gMKs1c6F8ft LEgmu+1Tyac7okOupDU+DAGgDcTTCodtADl9zdz6b6XvA1Yu1iXmv2vYfEYGpCpshyYp jTRHbB2azyp3fiX8iV+wY1hpx35UP3ZuTOVBS7cQujDSgdRx8tjMc4Rdx6UlhANVGQ2j 6C9aQHJoDKtSouT1ItfgJya/w11DYoID7KRRzLEl85NvOrdKAbeAhpfIE0//KwNbZ8R0 qdkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=LNo3a9YbRFtbaGi75EqATFDyPkpdUQZum9ZAllFUv/Q=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=DjQ0xFdKCZ2LA3RyUN6tbdfaCpPPamz3OodYSEav0VtrnQWd1kz0DYggXlBRJ3pDQ3 0MAmuwyKCJusjC4mZnzwTTuTZYrjtkH4cEFIvExVEpjxSTQhD8P790Hcib+tFfOM7cfd Tw7xViNGuUzcei/g/RihHeW+dYdtXPlaqSl0W5nhLaHwo0r0AQCDWwfUsc8nyeHAwgOZ q8RGSsz5gxeSqxsYdNjLYiMi31zo+pw2dW4/dtf3qmNeFRgEsSQeLVkcedZNgQPPOp/V Ma8Cucu43HwkmzXJ9px1oOCukPBU2H3FbT8c1I/TJVybpc6SxeSVJphsp4ZvpFg3hE+b FQQQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=ATvCJG2z; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a6f10531864si218366466b.490.2024.06.10.11.44.32; Mon, 10 Jun 2024 11:44:34 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=ATvCJG2z; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1655868D772; Mon, 10 Jun 2024 21:44:29 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f44.google.com (mail-lf1-f44.google.com [209.85.167.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B860868D6D7 for ; Mon, 10 Jun 2024 21:44:21 +0300 (EEST) Received: by mail-lf1-f44.google.com with SMTP id 2adb3069b0e04-52c815e8e9eso2231363e87.0 for ; Mon, 10 Jun 2024 11:44:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718045060; x=1718649860; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=FlYj3CuajMQFA3WrI3aVsW8oBGIoiWVbmsNt4hfJMZw=; b=ATvCJG2zh5xhzUcAfTK5Q5kTyIMwouUpmZzsbGFJ3aGZQNsFyHUO5Qg3T7NiTdRkPm bUuIbvyTxa0tilEDD8KcDh5JGei2mCvyGGQB5XJnLes4xLuNSOOH3TCc0zpD5mB+i07Q AiaeDWjN7DZpHeRyiiSmvILpXzDJaclIGEv0Plwkzh2DD58eIvVBmWRfEs8LMD2Sflqp Vtm1275hDnjxUZVhyKWqQjYyCuhjfIISN0q3kAq2xHsylf6rLfvhhI/Is3uQ9gWYoZs2 VXMf/oVFPLo9U7lCFq22dtETCRYj603+gbEl9z1x5T03kwQMtZasINk56X2RCCewcZXg X54A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718045060; x=1718649860; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FlYj3CuajMQFA3WrI3aVsW8oBGIoiWVbmsNt4hfJMZw=; b=IZ/hnt9iuZzJLyj+Xbe0ZgkoBbvJl3C1hesexjyWFP9jthI2qHaK6M1ko/Sele1Ufp 62Rk+t4cCwKcJS+MZcAHA64YGXskgsAOm34X/8yRj5R4Ua9I41Du7lsVWTsEz92/PF9b MM7S95HvC4i5xFKuGRem6ilYQ+WC5jLN3eu8ldDpiI1aC4WrUUYI885Gu3H95VoggEVX N6TmMke53ns4VZZ1TdpFjbFhC5G6Icfi8r1Q9SKnFFNTeO1497XSgp4Ep8EXFjwAC5To 0MI7zKGSfq5nsB9N77O7KThHX0enJjuwKkU4Xs/TalQXnlENfMqhU1zG3hpYmlEWQDMx H5FQ== X-Gm-Message-State: AOJu0YzyFf5KBi3qkqTdauUZvs50fcORHI5enFVDuVQGvlHyWEws+KC/ TZE9aOVf/i0efEns+EI9Bh6L8Fv/wEdrYMgnVRqPyk3J+nQU+ZKIvuY24A== X-Received: by 2002:a19:8c0b:0:b0:52b:c235:30da with SMTP id 2adb3069b0e04-52bc2353258mr5186262e87.24.1718045059906; Mon, 10 Jun 2024 11:44:19 -0700 (PDT) Received: from localhost.localdomain ([51.191.0.41]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4218193b0c0sm64659165e9.31.2024.06.10.11.44.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jun 2024 11:44:19 -0700 (PDT) From: Derek Buitenhuis To: ffmpeg-devel@ffmpeg.org Date: Mon, 10 Jun 2024 19:44:02 +0100 Message-ID: <20240610184408.68171-1-derek.buitenhuis@gmail.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 0/5] Apple Spatial Metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Oi/syrN8JVKv So I am a bit hesitant to send this, because while some of the boxes are documented officially by Apple, some of what are created by the Vision Pro / iPhones are not, because the official PDF is outdated. It's a call we have to make whether or not we want to push this, or hold out faith Apple will eventually update their document. I reverse engienered the rest of the boxes based on the what the AVFoundation APIs expose for spatial files. See each commit message. Example file: https://www.dropbox.com/scl/fi/g4b0q6c0e0xb9kwxcsvaa/IMG_1722.MOV?rlkey=az1qik442xut3jwq73jvcau9n&dl=0 Example stream from Apple: https://developer.apple.com/streaming/examples/ I can provide more if needed. Anyone with an iPhone 15+ or Vision Pro can make some files, too, of course. Some other notes: * I did not implement the 'must' box, as described in [1], since its only purpose seems to be to fail input files. I can add this as information-only if people want. I have not found any samples with this box, including files made by Apple. * Right now I am checking if the side data values are zero, as this is the best I could come up with, but comments welcome. * I like the idea that the 'dadj' box sits inside a 'cmfy' box, it seems very wholesome. [1] https://developer.apple.com/av-foundation/HEVC-Stereo-Video-Profile.pdf Derek Buitenhuis (5): avutil/spherical: Add more spherical types avutil/stereo3d: Fill out stereo info provided by Vision Pro files fftools/ffprobe: Print more Stereo 3D info from side data avformat/mov: Add support for exporting Video Extension Usage info avformat/mov: Add support for reading and exporting horizontal field of view fftools/ffprobe.c | 8 ++ libavformat/mov.c | 308 ++++++++++++++++++++++++++++++++++++++++++ libavutil/spherical.c | 3 + libavutil/spherical.h | 16 +++ libavutil/stereo3d.c | 52 +++++++ libavutil/stereo3d.h | 80 +++++++++++ libavutil/version.h | 2 +- 7 files changed, 468 insertions(+), 1 deletion(-)