From patchwork Sun Aug 6 23:04:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 43145 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9329:b0:130:ccc6:6c4b with SMTP id r41csp1270263pzh; Sun, 6 Aug 2023 16:03:27 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH+9qwDz8YXzR4yyATyYH3lWAH8n7AKHFwkEpyNYGK+gf4Hg2T4dpYv8yM4ugO9Tqg/lZxS X-Received: by 2002:a17:906:51cf:b0:99b:ed8d:de4 with SMTP id v15-20020a17090651cf00b0099bed8d0de4mr7501833ejk.20.1691363007198; Sun, 06 Aug 2023 16:03:27 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id fx12-20020a170906b74c00b00988a4a7abc8si4850155ejb.75.2023.08.06.16.03.26; Sun, 06 Aug 2023 16:03:27 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=ogP5mtDr; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3E00468C569; Mon, 7 Aug 2023 02:03:22 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05olkn2063.outbound.protection.outlook.com [40.92.90.63]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DBCD168C507 for ; Mon, 7 Aug 2023 02:03:15 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RRUtswjhGD/QLl3plczS5WCBKy15aMMdvgPH3P0+VzX6o/LuTKJ7LTJjtjqlGdTnDQ6PvLzXQIUL9fuTp9GjPy846KpU8G1JpxOOC4ED9PdE6Ts8jtg2PMuVeW0kr7eaX8DFBGgcWMdqrcrS2MDzcezqZIUB2Ycc1Xc8BC3FL03uv49BK2eRgSuELymSPqGuOgB5rTvM1Jky/iujNKGyNpeb/xBjAbdCySu6Y7l3x0u8guq64FAyT6vxFuVZhxtjbP7coxkzXvaE5rk6lvLq3QvO2nB9TsdoGD0m6FwVn8yVOaI3UfUhIL13tZyMj2iiAB+Yd6P1yXhujCEcDHD13g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=oqqCE5ZwY3qjRTNticbFyZKMTW1RGQy52vMLJyCZGGk=; b=F2SeHm3IQkNsK2TNeq8QFAssQvUOsp/oEgJXjLFN6zGu1s4bB6LzL+BQuFu0MzOGcXoRZ2/MUhUEjlsjyDIlVT+N0+P6HD5BqdqcXFVVPWgA/zrIYkvK5e1zE+SICXUQKDbPHPNKMgaO1QMazzs6owVTxpRPyRGhj8svwl+Yu9Fx6uUizAZOaYPG4c2JT1Vzaydj/eqlR39s873PzXp1K6Q7gmsvtfrA8bSU/RDE5v/7Mgp8wLFYoi8uHCGECMZ6wFMoS7//F/wtPFSQgYS3O6m1pTyEgDjpSJ4vibwC52poF9VszY8MDFQH8cREcRJw2rzPShUlL6KqOky5BsNJrQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oqqCE5ZwY3qjRTNticbFyZKMTW1RGQy52vMLJyCZGGk=; b=ogP5mtDrFUw3LEUwOa3tyczYMte4t08HbjDdlnFTNHeWwPIhUGcvNs8m5+HSs41svZ6oSfkVoR6Yr1sjPRwhg9GgJT5j9XTPeeYGZhGLbENYWDUbNJuN73WFdPJdbkLbqxXgsl1n8iHyanse5lGoFrPLiXIuZn/WkP2YaHAiMYQiUrjcC1rOm7HC+mHmJk8m6ozy+aMXYuRwQkxrzFwGFNYQUzcokJ/MXxfHPjfaS53Fyda0Xk6dw+8rR9Fx59+4T5z8nZQgNIN7pxmRhcyohYcZszw/t6NsbA4XbL6nQxMUHW9k8RcosVR2+MyRz8RXYDV5P64L3+bUl1uA87+3xw== Received: from AS8P250MB0744.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:541::14) by AM8P250MB0011.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:3da::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47; Sun, 6 Aug 2023 23:03:13 +0000 Received: from AS8P250MB0744.EURP250.PROD.OUTLOOK.COM ([fe80::5e01:aea5:d3a8:cafa]) by AS8P250MB0744.EURP250.PROD.OUTLOOK.COM ([fe80::5e01:aea5:d3a8:cafa%3]) with mapi id 15.20.6652.021; Sun, 6 Aug 2023 23:03:13 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Mon, 7 Aug 2023 01:04:16 +0200 Message-ID: X-Mailer: git-send-email 2.34.1 X-TMN: [EiGgRDwk5IvjctuyU32RhvcnMhx0au0C7hqw8KkZ9vU=] X-ClientProxiedBy: ZR0P278CA0172.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:45::6) To AS8P250MB0744.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:541::14) X-Microsoft-Original-Message-ID: <20230806230420.3186110-1-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AS8P250MB0744:EE_|AM8P250MB0011:EE_ X-MS-Office365-Filtering-Correlation-Id: bd21eebb-4751-4671-7713-08db96d14fca X-MS-Exchange-SLBlob-MailProps: AZnQBsB9Xmre2cOvxteVHck60FqChmDU7zLmutdaFrca5ZTR7XeOBzFxjYYl6f0iTgLkvJgMFx/t3NS0NdqrVoiUY2PNBTFqDhVggiKna3Kn3uNao8fwplQ7HpgF50AKrOqlVnkFkjxTAi3TY5tqsuJX2TFsb/ALUZDEfJPaRgWhECwjuFxQPcMWgE0KlyyAyVJCbETjzg+77FEid8FYBUSnelCgMQA7Od2tTWevbt9zvjJdoPQ3Ly9b+Yg3cr0qC57TbOgGDpuJSpS2QNln2OGebiDcEaXE9DnNeK+cksHHfOC7z296rFKMQxYZjYZd8G/jctHcMePBPxilqwI3BlmFZq4Pam2UuUJQGic11JZMrTp+ygVsq6mWN4vlMy548IXTjLe/E5z6oOKJWtuCpH5vsx4tK/DuU7v9+u4UFoXKsRJj7ik71mnIrwP8v/S5mI+LNHaOrqLcH3BcgPnA/84s8EWyQdFS4mn3NLkNjbP6ocIKAQFjo/+SmARt8RmUfD+hFWRHo3su30ZJ9RLyWwBJKbSut1Exb4btLwqog60ILEpkvid2fYx1y21HkplRA0718bC9WAJdmEjPnjKCEuTUT6L6eOh4FvowFshWbiwR2dUszKAD6Rrvcq41EnCRJUwNQVH0P8eZr3HXqnSYbgPjvwsFK7nSC2uoq5z8PhWlOL9IUBpszfM9X5PrcuKi9reinwISHT90gSZm9YDiWZDJ/eDYTBMkyvv2pAX85vrNF+uXeaxpkCvFJqAiF/VTtGL1NBI4axU= X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6v/0H+wUVRKFpyTphaz5f+CAFqzPOyMUTj1q4pkJcH0fxi3EUhajISIRdFmayFk3pBhXap/3kqhf6eYByDG+N2a0J+G7oTwI+UPTE0224yOrGEMaKi1MFmY0s2aIo0OJ5wSM/6c5e2e9XAkHiO0pOx24TOKTe1g6aqxZMqyx+C705P3x+gC92DH7lxRsjeKYssgYBesNLLZvSDwBRXojJn26HIN2uLZKnu/woBTFlxCVBYGrYNifbkhQ/FsmOZB9/ALF4Ghn3DwjpL+1bygy9lep3IFM3v90auGO+MwQa/ek65JrNQ0BBoWocH7prTYg+FtnJ7dKw6fNYth2ukXOeajWblTom+BMbx2XQsZsrzZJ5IOv3b6sbLaGHKqor3FwNVbRUvIi/T6csfYgOztY04YvBmBIEIfvaH0ZAnnn8NT3gBDmjP3Ql1Wks9hoQumENB+7CZASF4JVqc22ZPDu9ZqAc9hbUW58FjcmWNZWTfcisdaLLoWaxLX+SokzQy0A41VDMzNuc27iJ8cVu/b1wjeDsg3vGokk6MTv/JIqvXfAVTFU/eX8ZksRyxQ6xICMbEEb5RI/0vaDCnXfho/174Bn3eniUstqWixo4LoSIJVJXF/G7KjOM9snJOhH+MvH X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: UJEEFMzH/eIWWKRpqfjIKAq2RrEktsRh+E2haF6sSOGbjzDCHjjzykBuPkT6KybqoorDmK9vGhFbbS2XRykPrAix7Y4ccbWBrL/yTgHO4RO9l4Zh8eIomyRsVUEDBj7j0Igho1rFS3FDPOiyu1Wf4TZE2gsI8Oz8eVq/8jQahu7mPPljvVaLU0sqYjgKmO1ZjHNzdYY/OGmOlQjoFlkF1cffubYGGMHMf08PF7IQczpnB6gBED+aVJIxHwqMqNPbjJRFTg1KiBdF7ca6WbJmwlOop+AjpGJLEsT495gQi79SlDGJQ+gdbvVs9vc2msoALNKKB0u9hcghYOsXHojGhmsKOVwBNxvBapYsqRcAv05w8bVsw4J9CStNunNGdPKqrgb1xJYeYV5oaZPrIJz7P/jo6jv2fDbEHYqoOcujxcB7jeMWKEyJdMZTfF2AU8sQOiIZTpe7inCDZFShXV/4IOPcpXxgt90xdBabYtpiNk21YcH9w0Ig8vzzcR4yZscrudGOA/F3ZpJDaKWOSp3J4oDPp5XH6Wojw8Q2Py5GnfYjXZC7XNiCgq8hHOCcKXz3563XYMSBY51epNVTWfMcQREUqL1yJ550pWT2Na9FnaCTkp7YFDpaQZBWUf4o88P1ueul2Bb/w0ONDf9QL1lggNvRcOvpgbJKGllkp8ySzM72actDdRq0B0/hNKmigXfE0W3MtVsx6E5IUCdk4WJx/JyM6R1v2q9EyQ3a7oOFS/0vAVw6HbmXDXB4f7c4gQo7uKX5zW6RN1KoLv1881qPihDYpPfPdpcKG7CwB5lqdgFBVqZ075zbaXPP3pWlXh77bwrfcJEOYf+lWRz13fhi9Cr5tGfZRptrMxRLsSdmFxOn+qbbZNnfMrpXbhCKeMzpEfuygXp/heHpzzpmg1nJS9r7kKAsQGf6haIsSZFeLD+1tH+b7AvMMa6q3+1HilyUccZhDs8HBX4Zt8iy5P/L3B5MIfqcR7rKzcF9cMG5Tf3AoEwZekYUBof0OlVHu15FgAfMc3thk76T+ufC6qzJEGX28VDiZGWnOcxioigrKKJsZ9eaNH8pEK5N3hYqiaOxabTqSKy5at3GE0dRHL9OKMB5gu/3iycThfG1gtRl+ytcc8DJ1+W9SnZGVmSptnFRCiBvxr2NqemqHirBTYPXE8jFR8l/jegU98RvmGzO5jCnRDIIaJWsnf/2+RQy+oI3oivMwGQUvennr/ev30Nbpy4sXmAJTIT+9N2pn6n2yijbF3f6zgWwMvVYpWOrKAxw X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: bd21eebb-4751-4671-7713-08db96d14fca X-MS-Exchange-CrossTenant-AuthSource: AS8P250MB0744.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Aug 2023 23:03:13.6912 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8P250MB0011 Subject: [FFmpeg-devel] [PATCH 1/5] avformat/matroskaenc: Support rotations X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: oU2oWN9OUkiB Matroska supports orthogonal transformations (both pure rotations as well as reflections) via its 3D-projection elements, namely ProjectionPoseYaw (for a horizontal reflection) as well as ProjectionPoseRoll (for rotations). This commit adds support for this. Support for this in the demuxer has been added in 937bb6bbc1e8654633737e69e403e95a37113058 and the sample used in the matroska-dovi-write-config8 FATE-test includes a displaymatrix indicating a rotation which is now properly written and read, thereby providing coverage for the relevant code in the muxer as well as the demuxer. Signed-off-by: Andreas Rheinhardt --- Honestly, I am not really sure how to handle the floating-point inaccuracies here (in atan2). libavformat/matroskaenc.c | 100 +++++++++++++++++---- tests/ref/fate/matroska-dovi-write-config8 | 13 ++- 2 files changed, 94 insertions(+), 19 deletions(-) diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index 41e13b273d..c1f40b26e6 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -1403,25 +1403,75 @@ static void mkv_write_video_color(EbmlWriter *writer, const AVStream *st, } #define MAX_VIDEO_PROJECTION_ELEMS 6 -static void mkv_write_video_projection(AVFormatContext *s, EbmlWriter *writer, - const AVStream *st, uint8_t private[]) +static void mkv_handle_rotation(void *logctx, const AVStream *st, + double *yaw, double *roll) +{ + const int32_t *matrix = + (const int32_t*)av_stream_get_side_data(st, AV_PKT_DATA_DISPLAYMATRIX, NULL); + + if (!matrix) + return; + + /* Check whether this is an affine transformation */ + if (matrix[2] || matrix[5]) + goto ignore; + + /* This together with the checks below test whether + * the upper-left 2x2 matrix is nonsingular. */ + if (!matrix[0] && !matrix[1]) + goto ignore; + + /* We ignore the translation part of the matrix (matrix[6] and matrix[7]) + * as well as any scaling, i.e. we only look at the upper left 2x2 matrix. + * We only accept matrices that are an exact multiple of an orthogonal one. + * Apart from the multiple, every such matrix can be obtained by + * potentially flipping in the x-direction (corresponding to yaw = 180) + * followed by a rotation of (say) an angle phi in the counterclockwise + * direction. The upper-left 2x2 matrix then looks like this: + * | (+/-)cos(phi) (-/+)sin(phi) | + * scale * | | + * | sin(phi) cos(phi) | + * The first set of signs in the first row apply in case of no flipping, + * the second set applies in case of flipping. */ + + /* The casts to int64_t are needed because -INT32_MIN doesn't fit + * in an int32_t. */ + if (matrix[0] == matrix[4] && -(int64_t)matrix[1] == matrix[3]) { + /* No flipping case */ + *yaw = 0; + } else if (-(int64_t)matrix[0] == matrix[4] && matrix[1] == matrix[3]) { + /* Horizontal flip */ + *yaw = 180; + } else { +ignore: + av_log(logctx, AV_LOG_INFO, "Ignoring display matrix indicating " + "non-orthogonal transformation.\n"); + return; + } + *roll = 180 / M_PI * atan2(matrix[3], matrix[4]); + + /* We do not write a ProjectionType element indicating "rectangular", + * because this is the default value. */ +} + +static int mkv_handle_spherical(void *logctx, EbmlWriter *writer, + const AVStream *st, uint8_t private[], + double *yaw, double *pitch, double *roll) { const AVSphericalMapping *spherical = (const AVSphericalMapping *)av_stream_get_side_data(st, AV_PKT_DATA_SPHERICAL, NULL); if (!spherical) - return; + return 0; if (spherical->projection != AV_SPHERICAL_EQUIRECTANGULAR && spherical->projection != AV_SPHERICAL_EQUIRECTANGULAR_TILE && spherical->projection != AV_SPHERICAL_CUBEMAP) { - av_log(s, AV_LOG_WARNING, "Unknown projection type\n"); - return; + av_log(logctx, AV_LOG_WARNING, "Unknown projection type\n"); + return 0; } - ebml_writer_open_master(writer, MATROSKA_ID_VIDEOPROJECTION); - switch (spherical->projection) { case AV_SPHERICAL_EQUIRECTANGULAR: case AV_SPHERICAL_EQUIRECTANGULAR_TILE: @@ -1455,17 +1505,33 @@ static void mkv_write_video_projection(AVFormatContext *s, EbmlWriter *writer, av_assert0(0); } - if (spherical->yaw) - ebml_writer_add_float(writer, MATROSKA_ID_VIDEOPROJECTIONPOSEYAW, - (double) spherical->yaw / (1 << 16)); - if (spherical->pitch) - ebml_writer_add_float(writer, MATROSKA_ID_VIDEOPROJECTIONPOSEPITCH, - (double) spherical->pitch / (1 << 16)); - if (spherical->roll) - ebml_writer_add_float(writer, MATROSKA_ID_VIDEOPROJECTIONPOSEROLL, - (double) spherical->roll / (1 << 16)); + *yaw = (double) spherical->yaw / (1 << 16); + *pitch = (double) spherical->pitch / (1 << 16); + *roll = (double) spherical->roll / (1 << 16); - ebml_writer_close_master(writer); + return 1; /* Projection included */ +} + +static void mkv_write_video_projection(void *logctx, EbmlWriter *wr, + const AVStream *st, uint8_t private[]) +{ + double yaw = 0, pitch = 0, roll = 0; + int ret; + + ebml_writer_open_master(wr, MATROSKA_ID_VIDEOPROJECTION); + + ret = mkv_handle_spherical(logctx, wr, st, private, &yaw, &pitch, &roll); + if (!ret) + mkv_handle_rotation(logctx, st, &yaw, &roll); + + if (yaw) + ebml_writer_add_float(wr, MATROSKA_ID_VIDEOPROJECTIONPOSEYAW, yaw); + if (pitch) + ebml_writer_add_float(wr, MATROSKA_ID_VIDEOPROJECTIONPOSEPITCH, pitch); + if (roll) + ebml_writer_add_float(wr, MATROSKA_ID_VIDEOPROJECTIONPOSEROLL, roll); + + ebml_writer_close_or_discard_master(wr); } #define MAX_FIELD_ORDER_ELEMS 2 diff --git a/tests/ref/fate/matroska-dovi-write-config8 b/tests/ref/fate/matroska-dovi-write-config8 index bb22563eee..58eb454865 100644 --- a/tests/ref/fate/matroska-dovi-write-config8 +++ b/tests/ref/fate/matroska-dovi-write-config8 @@ -1,5 +1,5 @@ -09ff3c0a038eec0cdf4773929b24f41a *tests/data/fate/matroska-dovi-write-config8.matroska -3600606 tests/data/fate/matroska-dovi-write-config8.matroska +80d2b23a6f27ab28b02a907b37b9649c *tests/data/fate/matroska-dovi-write-config8.matroska +3600620 tests/data/fate/matroska-dovi-write-config8.matroska #extradata 0: 551, 0xa18acf66 #extradata 1: 2, 0x00340022 #tb 0: 1/1000 @@ -46,6 +46,15 @@ 1, 395, 395, 23, 439, 0x7d85e4c9 [STREAM] [SIDE_DATA] +side_data_type=Display Matrix +displaymatrix= +00000000: 0 65536 0 +00000001: -65536 0 0 +00000002: 0 0 1073741824 + +rotation=-90 +[/SIDE_DATA] +[SIDE_DATA] side_data_type=DOVI configuration record dv_version_major=1 dv_version_minor=0