From patchwork Thu Nov 3 22:54:53 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vittorio Giovara X-Patchwork-Id: 1278 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.90.1 with SMTP id o1csp1040985vsb; Thu, 3 Nov 2016 16:11:09 -0700 (PDT) X-Received: by 10.28.22.21 with SMTP id 21mr318926wmw.31.1478214668948; Thu, 03 Nov 2016 16:11:08 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 188si1650606wmu.22.2016.11.03.16.11.07; Thu, 03 Nov 2016 16:11:08 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 35E69689E1E; Fri, 4 Nov 2016 01:11:02 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk0-f196.google.com (mail-qk0-f196.google.com [209.85.220.196]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A6523689208 for ; Fri, 4 Nov 2016 01:10:55 +0200 (EET) Received: by mail-qk0-f196.google.com with SMTP id n204so4438864qke.2 for ; Thu, 03 Nov 2016 16:10:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:message-id; bh=2maOdvTulkGpdo3Cn5U59AaLAm6HHuLHoH9PiwxqxZ8=; b=Dqb5QqkUAbTZmOdeK+0fvh1mxtHZVUQ1aNH/ln3RR4DUCQj1n65rRR5B5chEcxIVTO IWz/dTdj6OOLZDyag5HtAmGj2XZYHVTR4Wdl3XJcfEBvgGes2z+Z5VdzlGXlPahLwzKc KCdC1Pn9smhIV+iuo99Xearka4ZN8eJgV7WI6qm8PlRP5muoSCnpat8skmbcdiA49xDC 6smlnKHxSbQBp/QswfwNSxtGvbUYQQAEa3KC8u400Uusr7L8fF/XtHwdJDmq1AdHawCa dpB4C4nl5550CxUqeoSEjL3bHW8nb0CMZBxDnSHNRLY/DxY1luaaskY6I97FcVLiONiP R5sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id; bh=2maOdvTulkGpdo3Cn5U59AaLAm6HHuLHoH9PiwxqxZ8=; b=LsTdAI8cEgl0IPEExFkjLnxjOKwgNauXNRuqZNLRrBGOnzjE1ERmU6l21bApNtftXT N5SifpzNcVJRtXOA7JQg0WjYc4GddR4XtG9wOp882E7jqXdxRFY+bvpd3EF/lQmRWee8 GoCigliOcloMQoId/R3RSKv+LmTe+hiJ5fNTSN5Nn0Mgl2HO47WWzF7C6PPx5/KQxi5Q JeFKAUyjbH2mMS5BAW+XqWzc2hf0t//YttLAjijxczIAgp9kW7DvMFaCVmLUz1AIMbSG R8socBq2aR6CUTgpsDUuo5dN3OfzkX3ZrfMRjHk88jikrA/HGgsh5xI4DIZhpr32+88P jUTA== X-Gm-Message-State: ABUngvc/J/SucwYUMVeXuoih/zAHX1DOYidAACbKqPg9Rm6He8fyaZsC8d4VyGdOG5Xm6w== X-Received: by 10.55.5.134 with SMTP id 128mr11344008qkf.261.1478213694370; Thu, 03 Nov 2016 15:54:54 -0700 (PDT) Received: from vimacbookpro.vimeo.iac.corp ([216.112.253.9]) by smtp.gmail.com with ESMTPSA id n128sm5793273qka.49.2016.11.03.15.54.53 for (version=TLS1 cipher=AES128-SHA bits=128/128); Thu, 03 Nov 2016 15:54:53 -0700 (PDT) From: Vittorio Giovara To: ffmpeg-devel@ffmpeg.org Date: Thu, 3 Nov 2016 18:54:53 -0400 Message-Id: <20161103225453.17528-1-vittorio.giovara@gmail.com> X-Mailer: git-send-email 2.10.0 Subject: [FFmpeg-devel] [PATCHv4] mov: Evaluate the movie display matrix X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This matrix needs to be applied after all others have (currently only display matrix from trak), but cannot be handled in movie box, since streams are not allocated yet. So store it in main context, and apply it when appropriate, that is after parsing the tkhd one. Signed-off-by: Vittorio Giovara --- Updated according everybody's review. Changed the loop to always perform multiplication, so that it's simpler to apply and to test. Changed computation to fixed point instead of conversion to double. Added a single test to cover this usecase, https://www.dropbox.com/s/qfio4bjhkpz3p4o/displaymatrix.mov?dl=0 Please CC. Vittorio libavformat/isom.h | 1 + libavformat/mov.c | 48 +++++++++++++++++++++++++++++----------- tests/fate/mov.mak | 5 ++++- tests/ref/fate/mov-displaymatrix | 10 +++++++++ 4 files changed, 50 insertions(+), 14 deletions(-) create mode 100644 tests/ref/fate/mov-displaymatrix diff --git a/libavformat/isom.h b/libavformat/isom.h index d684502..02bfedd 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -240,6 +240,7 @@ typedef struct MOVContext { uint8_t *decryption_key; int decryption_key_len; int enable_drefs; + int32_t movie_display_matrix[3][3]; ///< display matrix from mvhd } MOVContext; int ff_mp4_read_descr_len(AVIOContext *pb); diff --git a/libavformat/mov.c b/libavformat/mov.c index 35782fe..beedaef 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -1231,6 +1231,7 @@ static int mov_read_mdhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) static int mov_read_mvhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) { + int i; int64_t creation_time; int version = avio_r8(pb); /* version */ avio_rb24(pb); /* flags */ @@ -1258,7 +1259,12 @@ static int mov_read_mvhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) avio_skip(pb, 10); /* reserved */ - avio_skip(pb, 36); /* display matrix */ + /* movie display matrix, store it in main context and use it later on */ + for (i = 0; i < 3; i++) { + c->movie_display_matrix[i][0] = avio_rb32(pb); // 16.16 fixed point + c->movie_display_matrix[i][1] = avio_rb32(pb); // 16.16 fixed point + c->movie_display_matrix[i][2] = avio_rb32(pb); // 2.30 fixed point + } avio_rb32(pb); /* preview time */ avio_rb32(pb); /* preview duration */ @@ -3815,12 +3821,22 @@ static int mov_read_meta(MOVContext *c, AVIOContext *pb, MOVAtom atom) return 0; } +// return 1 when matrix is identity, 0 otherwise +#define IS_MATRIX_IDENT(matrix) \ + ( (matrix)[0][0] == (1 << 16) && \ + (matrix)[1][1] == (1 << 16) && \ + (matrix)[2][2] == (1 << 30) && \ + !(matrix)[0][1] && !(matrix)[0][2] || \ + !(matrix)[1][0] && !(matrix)[1][2] || \ + !(matrix)[2][0] && !(matrix)[2][1]) + static int mov_read_tkhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) { - int i; + int i, j, e; int width; int height; int display_matrix[3][3]; + int res_display_matrix[3][3] = { { 0 } }; AVStream *st; MOVStreamContext *sc; int version; @@ -3870,15 +3886,20 @@ static int mov_read_tkhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) sc->width = width >> 16; sc->height = height >> 16; - // save the matrix and add rotate metadata when it is not the default - // identity - if (display_matrix[0][0] != (1 << 16) || - display_matrix[1][1] != (1 << 16) || - display_matrix[2][2] != (1 << 30) || - display_matrix[0][1] || display_matrix[0][2] || - display_matrix[1][0] || display_matrix[1][2] || - display_matrix[2][0] || display_matrix[2][1]) { - int i, j; + // apply the moov display matrix (after the tkhd one) + for (i = 0; i < 3; i++) { + const int sh[3] = { 16, 16, 30 }; + for (j = 0; j < 3; j++) { + for (e = 0; e < 3; e++) { + res_display_matrix[i][j] += + ((int64_t) display_matrix[i][e] * + c->movie_display_matrix[e][j]) >> sh[e]; + } + } + } + + // save the matrix when it is not the default identity + if (!IS_MATRIX_IDENT(res_display_matrix)) { double rotate; av_freep(&sc->display_matrix); @@ -3888,7 +3909,7 @@ static int mov_read_tkhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) for (i = 0; i < 3; i++) for (j = 0; j < 3; j++) - sc->display_matrix[i * 3 + j] = display_matrix[i][j]; + sc->display_matrix[i * 3 + j] = res_display_matrix[i][j]; rotate = av_display_rotation_get(sc->display_matrix); if (!isnan(rotate)) { @@ -3907,7 +3928,8 @@ static int mov_read_tkhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) double disp_transform[2]; for (i = 0; i < 2; i++) - disp_transform[i] = hypot(display_matrix[i][0], display_matrix[i][1]); + disp_transform[i] = hypot(sc->display_matrix[0 + i], + sc->display_matrix[3 + i]); if (disp_transform[0] > 0 && disp_transform[1] > 0 && disp_transform[0] < (1<<24) && disp_transform[1] < (1<<24) && diff --git a/tests/fate/mov.mak b/tests/fate/mov.mak index 6b79832..2c73d73 100644 --- a/tests/fate/mov.mak +++ b/tests/fate/mov.mak @@ -5,7 +5,8 @@ FATE_MOV = fate-mov-3elist \ fate-mov-elist-starts-ctts-2ndsample \ fate-mov-1elist-ends-last-bframe \ fate-mov-2elist-elist1-ends-bframe \ - fate-mov-aac-2048-priming + fate-mov-aac-2048-priming \ + fate-mov-displaymatrix FATE_SAMPLES_AVCONV += $(FATE_MOV) @@ -30,3 +31,5 @@ fate-mov-2elist-elist1-ends-bframe: CMD = framemd5 -i $(TARGET_SAMPLES)/mov/mov- fate-mov-aac-2048-priming: ffprobe$(PROGSSUF)$(EXESUF) fate-mov-aac-2048-priming: CMD = run ffprobe$(PROGSSUF)$(EXESUF) -show_packets -print_format compact $(TARGET_SAMPLES)/mov/aac-2048-priming.mov + +fate-mov-displaymatrix: CMD = framemd5 -i $(TARGET_SAMPLES)/mov/displaymatrix.mov -frames 1 diff --git a/tests/ref/fate/mov-displaymatrix b/tests/ref/fate/mov-displaymatrix new file mode 100644 index 0000000..52528c1 --- /dev/null +++ b/tests/ref/fate/mov-displaymatrix @@ -0,0 +1,10 @@ +#format: frame checksums +#version: 2 +#hash: MD5 +#tb 0: 1001/30000 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 240x160 +#sar 0: 2/1 +#stream#, dts, pts, duration, size, hash +0, 0, 0, 1, 57600, be949aa661551010f461069804f68e76