From patchwork Fri Jul 7 00:36:39 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Louis O'Bryan X-Patchwork-Id: 4249 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.1.76 with SMTP id 73csp2857685vsb; Thu, 6 Jul 2017 17:43:25 -0700 (PDT) X-Received: by 10.223.163.12 with SMTP id c12mr42266438wrb.85.1499388205026; Thu, 06 Jul 2017 17:43:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1499388204; cv=none; d=google.com; s=arc-20160816; b=jDi5HY4oaM0KPeVen+vKiJ5Pe5uCkmmS4PGvDcR5ISQn2D8gd9saLu1mbG1zbb9TS1 UIzfRxYRVON7jviV3Eq2anYwZWH9kmHDXlxOrzXZUpMvAQddW1nZS4uxrlQyNcLlQufv grdViE3AWVDj8XSwIjhucRLFx0fS+I+eCvFtbMpAo7RwtbO8StNQrUApE/ecBhveh+Pd bcGNk/72+zRZXQCJQZ925ODilbRmO1qESbtrgdlcN1kgsm4BKcOKYq47+urQ9KhJWsIe 28x6YXKE8QdnG16X/o01FfxP1XY5RaTf5LhvGpsY2An7AGtvzDeSLipP3SGGyu9hX8bT 0AIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=cmVKQzV+tQzRZNhVOvm9YpM6dij39DuGBDtumVhcbs8=; b=vZjaiTejI0LHcQxjlojozkStk6D5AsE1sM21OW3jEXT1er8MtnCUT0VWkEJ9QJLEmp JDpagL4bhLA6fVnjfkLL/gcGrY8j7EL/nm3O1Ukd+bhvOzZrX6hPf5OVI9qbkWm3gTC9 t/PcVL6JFSjhkkD0DPqBDIA838b3WhYz/vtbZ9tyGmj8SQ9r10WwmbP76vK/tzpNQABG cZYJjPPfaZCW9swSJOMJ68KbarmggRujQJYPch2R8SJUBb9kQZf8nKxOSLncIy4ODRG+ 44soGBKLE7gmQXzv4329rM6rdS8eRnxrCxU0T1lIyS5YUEbdw5ak6Gl4SYcXWdUh9Ob+ Hzjg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.b=s9RYP0cv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g204si1532099wma.58.2017.07.06.17.43.24; Thu, 06 Jul 2017 17:43:24 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.b=s9RYP0cv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7E378689888; Fri, 7 Jul 2017 03:43:19 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf0-f196.google.com (mail-pf0-f196.google.com [209.85.192.196]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DA45D6883D3 for ; Fri, 7 Jul 2017 03:43:12 +0300 (EEST) Received: by mail-pf0-f196.google.com with SMTP id q85so2313245pfq.2 for ; Thu, 06 Jul 2017 17:43:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=F7akUVRMq1hyKg3Itw/KpL59hm79lA1kVlUKKQC4wmY=; b=s9RYP0cveKWH5pWWuygKVAa/Jm8aY33+PL3zD0HAiaLCVV56jonRnu9o/U5N/NjHDV 1wXsSLVmqmohfF1wwj5JVg4b8vFKSqVMFbuaYNCOhLf3XtWh+ffgaehejfbdNFfmCAwZ wxdXPHDsUJAWabbpfQZTzdE1gGFaZLXsWA0UiKpw8D6NBP7JrWFnBvNHL49VQKmSdNe7 YJ/fYooU3PPP1tr9JU8KpznRHNbr7OmCivFAnwm2wSNPx1KodYaYP5K/yAbR705ZX8DT Kof1ajCXqY9LomYzvII28q5J4FC9kW3SzP1l9/EBfVX9o0N1zrDIqQAIh3qRvsNlx5W8 2wRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=F7akUVRMq1hyKg3Itw/KpL59hm79lA1kVlUKKQC4wmY=; b=HeWyJ/xFnULyiQytrIWG0ge40g62NFt60GkYu6gwXeQwX7BHEkdQPBn1y/xlfWQ7oO 1Lw8i2PyK2BKxlspQP0Rfph/UZPh+2ZDB+CYqfTx9vOEduoxKj8Yz54naGxa6CVC85qE 4EQc3LOItd0ddfyWz44YwJK8fzAK9UirtG8b92WavYZ+AXBcsxMCqXJpR9YFBxb71aIf LvH+GSUlUqLJ4sZlT4qDQIZwe5f3gbSeTDRApTaiQ3RFflue7BJ1gNnQwMoOJfChEa4F I7OqfbIaBsVtrRvfD7V+wMM7pM2Y5ze2hVMLU/P6O4Yx+aM4U2FjF6IHOVLt4QICOAHo zgIA== X-Gm-Message-State: AIVw110DHu4U7uE+/f4cJjtUJqDXs3amx64iwPxGskPabqKNCqQI7N8a 4PM1Ka7Iynin2kCv7gdq9g== X-Received: by 10.99.97.4 with SMTP id v4mr29278853pgb.254.1499387820336; Thu, 06 Jul 2017 17:37:00 -0700 (PDT) Received: from louiso0.mtv.corp.google.com ([100.98.46.81]) by smtp.googlemail.com with ESMTPSA id y192sm2049191pgd.38.2017.07.06.17.36.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 06 Jul 2017 17:36:59 -0700 (PDT) From: "Louis O'Bryan" X-Google-Original-From: Louis O'Bryan To: ffmpeg-devel@ffmpeg.org Date: Thu, 6 Jul 2017 17:36:39 -0700 Message-Id: <20170707003639.27739-2-louiso@louiso0.mtv.corp.google.com> X-Mailer: git-send-email 2.13.2.725.g09c95d1e9-goog In-Reply-To: <20170707003639.27739-1-louiso@louiso0.mtv.corp.google.com> References: <20170707003639.27739-1-louiso@louiso0.mtv.corp.google.com> Subject: [FFmpeg-devel] [PATCH 1/1] This change adds an encoder for Camera metadata motion. This is a type of sensor data associated with video, such as GPS, acceleration, gyro, and camera orientation. It does not encode video itself, but rather, this metadata. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Louis O'Bryan MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Louis O'Bryan Signed-off-by: Louis O'Bryan --- Changelog | 1 + doc/general.texi | 2 + libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 3 + libavcodec/avcodec.h | 1 + libavcodec/cammenc.c | 299 ++++++++++++++++++++++++++++++++++++++++++++++++ libavcodec/codec_desc.c | 6 + libavcodec/version.h | 4 +- libavformat/isom.c | 6 + libavformat/movenc.c | 200 +++++++++++++++++--------------- 10 files changed, 431 insertions(+), 92 deletions(-) create mode 100644 libavcodec/cammenc.c diff --git a/Changelog b/Changelog index a8726c6736..5f98385b53 100644 --- a/Changelog +++ b/Changelog @@ -2,6 +2,7 @@ Entries are sorted chronologically from oldest to youngest within each release, releases are sorted from youngest to oldest. version : +- Camera metadata motion encoder - deflicker video filter - doubleweave video filter - lumakey video filter diff --git a/doc/general.texi b/doc/general.texi index 8f582d586f..06996c81e8 100644 --- a/doc/general.texi +++ b/doc/general.texi @@ -912,6 +912,8 @@ following image formats are supported: @tab part of LCL, encoder experimental @item Zip Motion Blocks Video @tab X @tab X @tab Encoder works only in PAL8. +@item Camera metadata motion @tab X @tab + @tab Encoder for camera sensor data. @end multitable @code{X} means that encoding (resp. decoding) is supported. diff --git a/libavcodec/Makefile b/libavcodec/Makefile index b440a00746..306cc793ee 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -680,6 +680,7 @@ OBJS-$(CONFIG_ZLIB_DECODER) += lcldec.o OBJS-$(CONFIG_ZLIB_ENCODER) += lclenc.o OBJS-$(CONFIG_ZMBV_DECODER) += zmbv.o OBJS-$(CONFIG_ZMBV_ENCODER) += zmbvenc.o +OBJS-$(CONFIG_CAMERA_MOTION_METADATA_ENCODER) += cammenc.o # (AD)PCM decoders/encoders OBJS-$(CONFIG_PCM_ALAW_DECODER) += pcm.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 0243f47358..eff51f4042 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -651,6 +651,9 @@ static void register_all(void) REGISTER_DECODER(XBIN, xbin); REGISTER_DECODER(IDF, idf); + /* data */ + REGISTER_ENCODER(CAMERA_MOTION_METADATA, camera_motion_metadata); + /* external libraries, that shouldn't be used by default if one of the * above is available */ REGISTER_ENCDEC (LIBOPENH264, libopenh264); diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index b697afa0ae..622383f453 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -681,6 +681,7 @@ enum AVCodecID { AV_CODEC_ID_DVD_NAV, AV_CODEC_ID_TIMED_ID3, AV_CODEC_ID_BIN_DATA, + AV_CODEC_ID_CAMERA_MOTION_METADATA, AV_CODEC_ID_PROBE = 0x19000, ///< codec_id is not known (like AV_CODEC_ID_NONE) but lavf should attempt to identify it diff --git a/libavcodec/cammenc.c b/libavcodec/cammenc.c new file mode 100644 index 0000000000..d7592ab69d --- /dev/null +++ b/libavcodec/cammenc.c @@ -0,0 +1,299 @@ +/* + * Reference implementation for the CAMM Metadata encoder. + * Encodes sensor data for 360-degree cameras such as + * GPS, gyro, and acceleration. This is stored in a track separate from video + * and audio. + * + * Copyright (c) 2017 Louis O'Bryan + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * CAMM Metadata encoder + * @author Louis O'Bryan + */ + +#include +#include +#include "avcodec.h" +#include "internal.h" +#include "bytestream.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" + +#define DUMMY_ENCODER_SIZE 1 + +typedef struct CammContext { + AVCodecContext *avctx; +} CammContext; + +// Sizes of each type of metadata. +static int metadata_type_sizes[] = { + 3 * sizeof(float), + 2 * sizeof(uint64_t), + 3 * sizeof(float), + 3 * sizeof(float), + 3 * sizeof(float), + 3 * sizeof(float), + 3 * sizeof(double) + sizeof(uint32_t) + 7 * sizeof(float), + 3 * sizeof(float) +}; + +static int min_packet_size = sizeof(uint16_t) * 2; + +/** + * Validates that the latitude has a valid value. Returns 1 if ok, else 0. + */ +static int validate_latitude(AVCodecContext *avctx, double latitude) { + if (latitude < -M_PI / 4 || latitude > M_PI / 4) { + av_log(avctx, AV_LOG_ERROR, + "Latitude %f is not in bounds (%f, %f)\n", + latitude, -M_PI / 4, M_PI / 4); + return 0; + } + return 1; +} + +/** + * Validates that the longitude has a valid value. Returns 1 if ok, else 0. + */ +static int validate_longitude(AVCodecContext *avctx, double longitude) { + if (longitude < -M_PI / 2 || longitude > M_PI / 2) { + av_log(avctx, AV_LOG_ERROR, + "longitude %f is not in bounds (%f, %f)\n", + longitude, -M_PI / 2, M_PI / 2); + return 0; + } + return 1; +} + +static void log_float_values(AVCodecContext *avctx, const char *name, float *data) { + int i; + float value; + for (i = 0; i < 3; ++i) { + value = data[i]; + av_log(avctx, AV_LOG_DEBUG, "%s[%d] = %f\n", name, i, value); + } +} + +/** + * Validates that the incoming data is correctly formatted and returns + * 1 if it is, 0 otherwise. + */ +static int validate_data(AVCodecContext *avctx, const AVFrame *data) { + uint16_t packet_type; + int expected_packet_size; + uint64_t pixel_exposure_time; + uint64_t rolling_shutter_skew_time; + uint32_t *camm_data; + double time_gps_epoch; + uint32_t gps_fix_type; + double latitude; + double longitude; + float altitude; + + if (data->pkt_size < min_packet_size) { + av_log(avctx, AV_LOG_ERROR, + "CAMM input data with size %d is too small\n", data->pkt_size); + return 0; + } + + packet_type = ((uint16_t*)(*data->extended_data))[1]; + if (packet_type > sizeof(metadata_type_sizes) / sizeof(int)) { + av_log(avctx, AV_LOG_ERROR, + "Packet type %d is not recognized\n", packet_type); + return 0; + } + + expected_packet_size = metadata_type_sizes[packet_type]; + if (expected_packet_size != data->pkt_size) { + av_log(avctx, AV_LOG_ERROR, + "Packet size %d does not match expected size %d for type %d\n", + data->pkt_size, expected_packet_size, packet_type); + return 0; + } + + // The actual sensor data starts after the reserved slot and packet type. + camm_data = ((uint32_t*)(*data->extended_data)) + 1; + switch (packet_type) { + // float angle_axis[3] + case 0: + log_float_values(avctx, "angle_axis", (float*)camm_data); + break; + // int32 pixel_exposure_time + // int32 rolling_shutter_skew_time + case 1: + pixel_exposure_time = ((uint32_t*)camm_data)[0]; + av_log(avctx, AV_LOG_DEBUG, + "pixel_exposure_time = %lu\n", pixel_exposure_time); + rolling_shutter_skew_time = ((uint32_t*)camm_data)[1]; + av_log(avctx, AV_LOG_DEBUG, "rolling_shutter_skew_time = %lu\n", + rolling_shutter_skew_time); + break; + // float gyro[3] + case 2: + log_float_values(avctx, "gyro", (float*)camm_data); + break; + case 3: + log_float_values(avctx, "acceleration", (float*)camm_data); + break; + // float position[3] + case 4: + log_float_values(avctx, "position", (float*)camm_data); + break; + // float latitude + // float longitude + // float altitude + case 5: + if (!validate_latitude(avctx, ((float*)camm_data)[0])) return 0; + if (!validate_longitude(avctx, ((float*)camm_data)[1])) return 0; + + av_log(avctx, AV_LOG_DEBUG, "latitude = %.6f\n", + ((float*)camm_data)[0]); + av_log(avctx, AV_LOG_DEBUG, "longitude = %.6f\n", + ((float*)camm_data)[1]); + av_log(avctx, AV_LOG_DEBUG, "altitude = %.6f\n", + ((float*)camm_data)[2]); + break; + case 6: + time_gps_epoch = ((double*)camm_data)[0]; + if (time_gps_epoch < 0) { + av_log(avctx, AV_LOG_ERROR, + "Time gps epoch %.6f is less than 0.\n", + time_gps_epoch); + return 0; + } + av_log(avctx, AV_LOG_DEBUG, "time_gps_epoch = %.6f\n", + time_gps_epoch); + camm_data = (uint32_t*) (((double*)camm_data) + 1); + + gps_fix_type = ((uint32_t*)camm_data)[0]; + if (gps_fix_type != 0 && gps_fix_type != 1 && gps_fix_type != 2) { + av_log(avctx, AV_LOG_ERROR, + "GPS fix type %d is not valid. Should be 0, 1, or 2\n", + gps_fix_type); + return 0; + } + av_log(avctx, AV_LOG_DEBUG, "gps_fix_type = %d\n", gps_fix_type); + camm_data = (uint32_t*) (((int32_t*)camm_data) + 1); + + latitude = ((double*)camm_data)[0]; + if (!validate_latitude(avctx, latitude)) return 0; + av_log(avctx, AV_LOG_DEBUG, "latitude = %.6f\n", latitude); + camm_data = (uint32_t*) (((double*)camm_data) + 1); + + longitude = ((double*)camm_data)[0]; + if (!validate_longitude(avctx, longitude)) return 0; + av_log(avctx, AV_LOG_DEBUG, "longitude = %.6f\n", latitude); + camm_data = (uint32_t*) (((double*)camm_data) + 1); + + altitude = ((float*)camm_data)[0]; + av_log(avctx, AV_LOG_DEBUG, "altitude = %.6f\n", altitude); + camm_data = (uint32_t*) (((float*)camm_data) + 1); + + av_log(avctx, AV_LOG_DEBUG, "horizontal accuracy = %.6f\n", + ((float*)camm_data)[0]); + camm_data = (uint32_t*) (((float*)camm_data) + 1); + + av_log(avctx, AV_LOG_DEBUG, "vertical accuracy %.6f\n", + ((float*)camm_data)[0]); + camm_data = (uint32_t*) (((float*)camm_data) + 1); + + av_log(avctx, AV_LOG_DEBUG, "vertical east %.6f\n", + ((float*)camm_data)[0]); + camm_data = (uint32_t*) (((float*)camm_data) + 1); + + av_log(avctx, AV_LOG_DEBUG, "vertical north %.6f\n", + ((float*)camm_data)[0]); + camm_data = (uint32_t*) (((float*)camm_data) + 1); + + av_log(avctx, AV_LOG_DEBUG, "vertical up %.6f\n", + ((float*)camm_data)[0]); + camm_data = (uint32_t*) (((float*)camm_data) + 1); + + av_log(avctx, AV_LOG_DEBUG, "speed accuracy %.6f\n", + ((float*)camm_data)[0]); + camm_data = (uint32_t*) (((float*)camm_data) + 1); + break; + // float magnetic_field[3] + case 7: + log_float_values(avctx, "magnetic_field", (float*)camm_data); + break; + } + + return 1; +} + +/** + * Encode camera motion metadata. Essentially, this encoder validates the + * input data and then just copies the data to an output AVPacket. + * @param avctx The AVCodecContext context. + * @param packet The output packet to write the encoded data. + * @param data The input camera sensor data. + * @param got_packet 1 if a non-empty packet was returned, 0 otherwise. + */ +static int encode_data(AVCodecContext *avctx, AVPacket *packet, + const AVFrame *data, int *got_packet) +{ + int ret; + + if (!validate_data(avctx, data)) { + return AVERROR(EINVAL); + } + + // Make sure there is enough size allocated in packet->data. + if ((ret = ff_alloc_packet2(avctx, packet, data->pkt_size, 0)) < 0) { + av_log(avctx, AV_LOG_ERROR, "Can't allocate %d bytes in packet\n", + data->pkt_size); + return ret; + } + + // Copy necessary fields to the AVPacket. + packet->pts = data->pts; + packet->duration = data->pkt_duration; + memcpy(packet->data, *(data->extended_data), data->pkt_size); + + // Indicate that a packet was created. + *got_packet = 1; + return 0; +} + +/** + * Initializes the codec. + * + * @param avctx The AVCodecContext context. + */ +static av_cold int encode_init(AVCodecContext *avctx) { + // Use dummy values for the height and width. + avctx->width = DUMMY_ENCODER_SIZE; + avctx->height = DUMMY_ENCODER_SIZE; + avctx->max_pixels = DUMMY_ENCODER_SIZE; + + return 0; +} + +// Define the encoder that ffmpeg will use for CAMM data. +AVCodec ff_camera_motion_metadata_encoder = { + .name = "camm", + .long_name = NULL_IF_CONFIG_SMALL("camera motion metadata"), + .type = AVMEDIA_TYPE_DATA, + .id = AV_CODEC_ID_CAMERA_MOTION_METADATA, + .priv_data_size = sizeof(CammContext), + .encode2 = encode_data, + .init = encode_init, +}; diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c index cf1246e431..014c1cd6b3 100644 --- a/libavcodec/codec_desc.c +++ b/libavcodec/codec_desc.c @@ -3100,6 +3100,12 @@ static const AVCodecDescriptor codec_descriptors[] = { .name = "scte_35", .long_name = NULL_IF_CONFIG_SMALL("SCTE 35 Message Queue"), }, + { + .id = AV_CODEC_ID_CAMERA_MOTION_METADATA, + .type = AVMEDIA_TYPE_DATA, + .name = "camm", + .long_name = NULL_IF_CONFIG_SMALL("camera motion metadata"), + }, /* deprecated codec ids */ }; diff --git a/libavcodec/version.h b/libavcodec/version.h index 3c5fea9327..5b99785a72 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,8 +28,8 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 57 -#define LIBAVCODEC_VERSION_MINOR 100 -#define LIBAVCODEC_VERSION_MICRO 103 +#define LIBAVCODEC_VERSION_MINOR 101 +#define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \ diff --git a/libavformat/isom.c b/libavformat/isom.c index 3a9b3baf96..ca5107bf7f 100644 --- a/libavformat/isom.c +++ b/libavformat/isom.c @@ -67,6 +67,7 @@ const AVCodecTag ff_mp4_obj_type[] = { { AV_CODEC_ID_VORBIS , 0xDD }, /* nonstandard, gpac uses it */ { AV_CODEC_ID_DVD_SUBTITLE, 0xE0 }, /* nonstandard, see unsupported-embedded-subs-2.mp4 */ { AV_CODEC_ID_QCELP , 0xE1 }, + { AV_CODEC_ID_CAMERA_MOTION_METADATA, 0xE2 }, { AV_CODEC_ID_MPEG4SYSTEMS, 0x01 }, { AV_CODEC_ID_MPEG4SYSTEMS, 0x02 }, { AV_CODEC_ID_NONE , 0 }, @@ -368,6 +369,11 @@ const AVCodecTag ff_codec_movsubtitle_tags[] = { { AV_CODEC_ID_NONE, 0 }, }; +const AVCodecTag ff_codec_movdata_tags[] = { + { AV_CODEC_ID_CAMERA_MOTION_METADATA, MKTAG('c','a','m','m') }, +}; + + /* map numeric codes from mdhd atom to ISO 639 */ /* cf. QTFileFormat.pdf p253, qtff.pdf p205 */ /* http://developer.apple.com/documentation/mac/Text/Text-368.html */ diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 88f2f2c819..49528f8635 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -35,8 +35,6 @@ #include "libavcodec/dnxhddata.h" #include "libavcodec/flac.h" #include "libavcodec/get_bits.h" - -#include "libavcodec/internal.h" #include "libavcodec/put_bits.h" #include "libavcodec/vc1_common.h" #include "libavcodec/raw.h" @@ -1126,10 +1124,7 @@ static int mov_write_hvcc_tag(AVIOContext *pb, MOVTrack *track) avio_wb32(pb, 0); ffio_wfourcc(pb, "hvcC"); - if (track->tag == MKTAG('h','v','c','1')) - ff_isom_write_hvcc(pb, track->vos_data, track->vos_len, 1); - else - ff_isom_write_hvcc(pb, track->vos_data, track->vos_len, 0); + ff_isom_write_hvcc(pb, track->vos_data, track->vos_len, 0); return update_size(pb, pos); } @@ -1230,6 +1225,60 @@ static int mov_write_dpxe_tag(AVIOContext *pb, MOVTrack *track) return 0; } +static int mp4_get_codec_tag(AVFormatContext *s, MOVTrack *track) +{ + int tag = track->par->codec_tag; + + if (!ff_codec_get_tag(ff_mp4_obj_type, track->par->codec_id)) + return 0; + + if (track->par->codec_id == AV_CODEC_ID_H264) tag = MKTAG('a','v','c','1'); + else if (track->par->codec_id == AV_CODEC_ID_HEVC) tag = MKTAG('h','e','v','1'); + else if (track->par->codec_id == AV_CODEC_ID_VP9) tag = MKTAG('v','p','0','9'); + else if (track->par->codec_id == AV_CODEC_ID_AC3) tag = MKTAG('a','c','-','3'); + else if (track->par->codec_id == AV_CODEC_ID_EAC3) tag = MKTAG('e','c','-','3'); + else if (track->par->codec_id == AV_CODEC_ID_DIRAC) tag = MKTAG('d','r','a','c'); + else if (track->par->codec_id == AV_CODEC_ID_MOV_TEXT) tag = MKTAG('t','x','3','g'); + else if (track->par->codec_id == AV_CODEC_ID_VC1) tag = MKTAG('v','c','-','1'); + else if (track->par->codec_id == AV_CODEC_ID_FLAC) tag = MKTAG('f','L','a','C'); + else if (track->par->codec_id == AV_CODEC_ID_OPUS) tag = MKTAG('O','p','u','s'); + else if (track->par->codec_type == AVMEDIA_TYPE_VIDEO) tag = MKTAG('m','p','4','v'); + else if (track->par->codec_type == AVMEDIA_TYPE_AUDIO) tag = MKTAG('m','p','4','a'); + else if (track->par->codec_id == AV_CODEC_ID_DVD_SUBTITLE) tag = MKTAG('m','p','4','s'); + + return tag; +} + +static const AVCodecTag codec_ipod_tags[] = { + { AV_CODEC_ID_H264, MKTAG('a','v','c','1') }, + { AV_CODEC_ID_MPEG4, MKTAG('m','p','4','v') }, + { AV_CODEC_ID_AAC, MKTAG('m','p','4','a') }, + { AV_CODEC_ID_ALAC, MKTAG('a','l','a','c') }, + { AV_CODEC_ID_AC3, MKTAG('a','c','-','3') }, + { AV_CODEC_ID_MOV_TEXT, MKTAG('t','x','3','g') }, + { AV_CODEC_ID_MOV_TEXT, MKTAG('t','e','x','t') }, + { AV_CODEC_ID_NONE, 0 }, +}; + +static int ipod_get_codec_tag(AVFormatContext *s, MOVTrack *track) +{ + int tag = track->par->codec_tag; + + // keep original tag for subs, ipod supports both formats + if (!(track->par->codec_type == AVMEDIA_TYPE_SUBTITLE && + (tag == MKTAG('t', 'x', '3', 'g') || + tag == MKTAG('t', 'e', 'x', 't')))) + tag = ff_codec_get_tag(codec_ipod_tags, track->par->codec_id); + + if (!av_match_ext(s->filename, "m4a") && + !av_match_ext(s->filename, "m4b") && + !av_match_ext(s->filename, "m4v")) + av_log(s, AV_LOG_WARNING, "Warning, extension is not .m4a, .m4v nor .m4b " + "Quicktime/Ipod might not play the file\n"); + + return tag; +} + static int mov_get_dv_codec_tag(AVFormatContext *s, MOVTrack *track) { int tag; @@ -1506,25 +1555,42 @@ static int mov_get_codec_tag(AVFormatContext *s, MOVTrack *track) return tag; } +static const AVCodecTag codec_3gp_tags[] = { + { AV_CODEC_ID_H263, MKTAG('s','2','6','3') }, + { AV_CODEC_ID_H264, MKTAG('a','v','c','1') }, + { AV_CODEC_ID_MPEG4, MKTAG('m','p','4','v') }, + { AV_CODEC_ID_AAC, MKTAG('m','p','4','a') }, + { AV_CODEC_ID_AMR_NB, MKTAG('s','a','m','r') }, + { AV_CODEC_ID_AMR_WB, MKTAG('s','a','w','b') }, + { AV_CODEC_ID_MOV_TEXT, MKTAG('t','x','3','g') }, + { AV_CODEC_ID_NONE, 0 }, +}; + +static const AVCodecTag codec_f4v_tags[] = { // XXX: add GIF/PNG/JPEG? + { AV_CODEC_ID_MP3, MKTAG('.','m','p','3') }, + { AV_CODEC_ID_AAC, MKTAG('m','p','4','a') }, + { AV_CODEC_ID_H264, MKTAG('a','v','c','1') }, + { AV_CODEC_ID_VP6A, MKTAG('V','P','6','A') }, + { AV_CODEC_ID_VP6F, MKTAG('V','P','6','F') }, + { AV_CODEC_ID_NONE, 0 }, +}; + static int mov_find_codec_tag(AVFormatContext *s, MOVTrack *track) { int tag; if (track->mode == MODE_MP4 || track->mode == MODE_PSP) - tag = track->par->codec_tag; - else if (track->mode == MODE_ISM) - tag = track->par->codec_tag; - else if (track->mode == MODE_IPOD) { - if (!av_match_ext(s->filename, "m4a") && - !av_match_ext(s->filename, "m4v") && - !av_match_ext(s->filename, "m4b")) - av_log(s, AV_LOG_WARNING, "Warning, extension is not .m4a nor .m4v " - "Quicktime/Ipod might not play the file\n"); - tag = track->par->codec_tag; - } else if (track->mode & MODE_3GP) - tag = track->par->codec_tag; + tag = mp4_get_codec_tag(s, track); + else if (track->mode == MODE_ISM) { + tag = mp4_get_codec_tag(s, track); + if (!tag && track->par->codec_id == AV_CODEC_ID_WMAPRO) + tag = MKTAG('w', 'm', 'a', ' '); + } else if (track->mode == MODE_IPOD) + tag = ipod_get_codec_tag(s, track); + else if (track->mode & MODE_3GP) + tag = ff_codec_get_tag(codec_3gp_tags, track->par->codec_id); else if (track->mode == MODE_F4V) - tag = track->par->codec_tag; + tag = ff_codec_get_tag(codec_f4v_tags, track->par->codec_id); else tag = mov_get_codec_tag(s, track); @@ -2060,6 +2126,19 @@ static int mov_write_tmcd_tag(AVIOContext *pb, MOVTrack *track) return update_size(pb, pos); } +static int mov_write_camm_tag(AVIOContext *pb) { + int64_t size_update; + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "camm"); + avio_wb32(pb, 0); /* Reserved */ + avio_wb16(pb, 0); /* Reserved */ + avio_wb16(pb, 1); /* Data-reference index */ + + size_update = update_size(pb, pos); + return size_update; +} + static int mov_write_stsd_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext *mov, MOVTrack *track) { int64_t pos = avio_tell(pb); @@ -2077,6 +2156,10 @@ static int mov_write_stsd_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext mov_write_rtp_tag(pb, track); else if (track->par->codec_tag == MKTAG('t','m','c','d')) mov_write_tmcd_tag(pb, track); + else if (track->par->codec_tag + == ff_codec_get_tag(ff_mp4_obj_type, + AV_CODEC_ID_CAMERA_MOTION_METADATA)) + mov_write_camm_tag(pb); return update_size(pb, pos); } @@ -2443,6 +2526,11 @@ static int mov_write_hdlr_tag(AVFormatContext *s, AVIOContext *pb, MOVTrack *tra } else if (track->par->codec_tag == MKTAG('t','m','c','d')) { hdlr_type = "tmcd"; descr = "TimeCodeHandler"; + } else if (track->par->codec_tag + == ff_codec_get_tag(ff_mp4_obj_type, + AV_CODEC_ID_CAMERA_MOTION_METADATA)) { + hdlr_type = "camm"; + descr = "CameraMetadataMotionHandler"; } else { av_log(s, AV_LOG_WARNING, "Unknown hldr_type for %s, writing dummy values\n", @@ -6422,73 +6510,6 @@ static int mov_check_bitstream(struct AVFormatContext *s, const AVPacket *pkt) return ret; } -static const AVCodecTag codec_3gp_tags[] = { - { AV_CODEC_ID_H263, MKTAG('s','2','6','3') }, - { AV_CODEC_ID_H264, MKTAG('a','v','c','1') }, - { AV_CODEC_ID_MPEG4, MKTAG('m','p','4','v') }, - { AV_CODEC_ID_AAC, MKTAG('m','p','4','a') }, - { AV_CODEC_ID_AMR_NB, MKTAG('s','a','m','r') }, - { AV_CODEC_ID_AMR_WB, MKTAG('s','a','w','b') }, - { AV_CODEC_ID_MOV_TEXT, MKTAG('t','x','3','g') }, - { AV_CODEC_ID_NONE, 0 }, -}; - -const AVCodecTag codec_mp4_tags[] = { - { AV_CODEC_ID_MPEG4 , MKTAG('m', 'p', '4', 'v') }, - { AV_CODEC_ID_H264 , MKTAG('a', 'v', 'c', '1') }, - { AV_CODEC_ID_HEVC , MKTAG('h', 'e', 'v', '1') }, - { AV_CODEC_ID_HEVC , MKTAG('h', 'v', 'c', '1') }, - { AV_CODEC_ID_MPEG2VIDEO , MKTAG('m', 'p', '4', 'v') }, - { AV_CODEC_ID_MPEG1VIDEO , MKTAG('m', 'p', '4', 'v') }, - { AV_CODEC_ID_MJPEG , MKTAG('m', 'p', '4', 'v') }, - { AV_CODEC_ID_PNG , MKTAG('m', 'p', '4', 'v') }, - { AV_CODEC_ID_JPEG2000 , MKTAG('m', 'p', '4', 'v') }, - { AV_CODEC_ID_VC1 , MKTAG('v', 'c', '-', '1') }, - { AV_CODEC_ID_DIRAC , MKTAG('d', 'r', 'a', 'c') }, - { AV_CODEC_ID_TSCC2 , MKTAG('m', 'p', '4', 'v') }, - { AV_CODEC_ID_VP9 , MKTAG('v', 'p', '0', '9') }, - { AV_CODEC_ID_AAC , MKTAG('m', 'p', '4', 'a') }, - { AV_CODEC_ID_MP4ALS , MKTAG('m', 'p', '4', 'a') }, - { AV_CODEC_ID_MP3 , MKTAG('m', 'p', '4', 'a') }, - { AV_CODEC_ID_MP2 , MKTAG('m', 'p', '4', 'a') }, - { AV_CODEC_ID_AC3 , MKTAG('a', 'c', '-', '3') }, - { AV_CODEC_ID_EAC3 , MKTAG('e', 'c', '-', '3') }, - { AV_CODEC_ID_DTS , MKTAG('m', 'p', '4', 'a') }, - { AV_CODEC_ID_FLAC , MKTAG('f', 'L', 'a', 'C') }, - { AV_CODEC_ID_OPUS , MKTAG('O', 'p', 'u', 's') }, - { AV_CODEC_ID_VORBIS , MKTAG('m', 'p', '4', 'a') }, - { AV_CODEC_ID_QCELP , MKTAG('m', 'p', '4', 'a') }, - { AV_CODEC_ID_EVRC , MKTAG('m', 'p', '4', 'a') }, - { AV_CODEC_ID_DVD_SUBTITLE, MKTAG('m', 'p', '4', 's') }, - { AV_CODEC_ID_MOV_TEXT , MKTAG('t', 'x', '3', 'g') }, - { AV_CODEC_ID_NONE , 0 }, -}; - -const AVCodecTag codec_ism_tags[] = { - { AV_CODEC_ID_WMAPRO , MKTAG('w', 'm', 'a', ' ') }, - { AV_CODEC_ID_NONE , 0 }, -}; - -static const AVCodecTag codec_ipod_tags[] = { - { AV_CODEC_ID_H264, MKTAG('a','v','c','1') }, - { AV_CODEC_ID_MPEG4, MKTAG('m','p','4','v') }, - { AV_CODEC_ID_AAC, MKTAG('m','p','4','a') }, - { AV_CODEC_ID_ALAC, MKTAG('a','l','a','c') }, - { AV_CODEC_ID_AC3, MKTAG('a','c','-','3') }, - { AV_CODEC_ID_MOV_TEXT, MKTAG('t','x','3','g') }, - { AV_CODEC_ID_MOV_TEXT, MKTAG('t','e','x','t') }, - { AV_CODEC_ID_NONE, 0 }, -}; - -static const AVCodecTag codec_f4v_tags[] = { - { AV_CODEC_ID_MP3, MKTAG('.','m','p','3') }, - { AV_CODEC_ID_AAC, MKTAG('m','p','4','a') }, - { AV_CODEC_ID_H264, MKTAG('a','v','c','1') }, - { AV_CODEC_ID_VP6A, MKTAG('V','P','6','A') }, - { AV_CODEC_ID_VP6F, MKTAG('V','P','6','F') }, - { AV_CODEC_ID_NONE, 0 }, -}; - #if CONFIG_MOV_MUXER MOV_CLASS(mov) AVOutputFormat ff_mov_muxer = { @@ -6549,7 +6570,7 @@ AVOutputFormat ff_mp4_muxer = { .write_trailer = mov_write_trailer, .deinit = mov_free, .flags = AVFMT_GLOBALHEADER | AVFMT_ALLOW_FLUSH | AVFMT_TS_NEGATIVE, - .codec_tag = (const AVCodecTag* const []){ codec_mp4_tags, 0 }, + .codec_tag = (const AVCodecTag* const []){ ff_mp4_obj_type, 0 }, .check_bitstream = mov_check_bitstream, .priv_class = &mp4_muxer_class, }; @@ -6570,7 +6591,7 @@ AVOutputFormat ff_psp_muxer = { .write_trailer = mov_write_trailer, .deinit = mov_free, .flags = AVFMT_GLOBALHEADER | AVFMT_ALLOW_FLUSH | AVFMT_TS_NEGATIVE, - .codec_tag = (const AVCodecTag* const []){ codec_mp4_tags, 0 }, + .codec_tag = (const AVCodecTag* const []){ ff_mp4_obj_type, 0 }, .check_bitstream = mov_check_bitstream, .priv_class = &psp_muxer_class, }; @@ -6632,8 +6653,7 @@ AVOutputFormat ff_ismv_muxer = { .write_trailer = mov_write_trailer, .deinit = mov_free, .flags = AVFMT_GLOBALHEADER | AVFMT_ALLOW_FLUSH | AVFMT_TS_NEGATIVE, - .codec_tag = (const AVCodecTag* const []){ - codec_mp4_tags, codec_ism_tags, 0 }, + .codec_tag = (const AVCodecTag* const []){ ff_mp4_obj_type, 0 }, .check_bitstream = mov_check_bitstream, .priv_class = &ismv_muxer_class, };