From patchwork Thu Oct 13 22:50:19 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vittorio Giovara X-Patchwork-Id: 997 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.140.66 with SMTP id o63csp11652vsd; Thu, 13 Oct 2016 16:08:09 -0700 (PDT) X-Received: by 10.28.64.133 with SMTP id n127mr3620379wma.31.1476400089742; Thu, 13 Oct 2016 16:08:09 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h3si20738568wji.157.2016.10.13.16.08.09; Thu, 13 Oct 2016 16:08:09 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 61EAB68993A; Fri, 14 Oct 2016 02:08:06 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk0-f195.google.com (mail-qk0-f195.google.com [209.85.220.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7FBA86898BC for ; Fri, 14 Oct 2016 02:07:59 +0300 (EEST) Received: by mail-qk0-f195.google.com with SMTP id n189so7314486qke.1 for ; Thu, 13 Oct 2016 16:08:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:message-id:in-reply-to:references; bh=AI/EughuUerfaXB/mMeJJ79mK75vRWlqJRzIpc+YDWI=; b=n4K79E6pKLIGJFbE+R9Xsmx5a6lWk/9vnY48/ufc3K/FBjy4HEmvfgoNG+XDtnoDKP h7Av5MYG/FSqeHc8ndYryICoJNMCkRBUNoKkn2FXcF/bycYWqaaHhTXoR8IFO87dM1/f YY9STjOerj/2r77UOcMeQdxw0vzweSOzoirlI0GdxZctl5T1kKS7dcNvRS4owgDS98jb QKP0gxbt2wnpkP2N25OeMu7lXlXMKtrQo+0HWr3XEmLdDd028fJimABtrWi/sPTrJiDX /L8/C4zsE4rmWh2bDeA9aod/kW+XczhvRYTAom+wc6ZYskiVLlsOx1YWY44nbo1w41VF O23A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=AI/EughuUerfaXB/mMeJJ79mK75vRWlqJRzIpc+YDWI=; b=RFyi8x0aocGX1T8meRdnBRBQGWetKo3t/giRipMu3CoewPwINEJqQH+Mr2xx7T2VE6 wk0+u5cWSuDYfOBxRdIIUggB2xg00/e2QLO+6Kxw/Zuyivdl42yF1YXUuAXUOKI4MAc+ cT7QoUl104hSDsJAU0aI2dBKS029dDCA+6ZHdTROyEPOqr7Hv3SVulqiugYPf8/Yd3kf ai4XezjSwtsJw23sbOjnbD/vlPM9KDvNbZbW5WnUHD5w3gWJxhG21Y2Pu9gm19l9Riki fZLMkhLE9Lx1XuIZj6UeMJ4BOn718oJRlGUGj464x68kRAef/5vB9H8F7LvgeTYln6+f AklA== X-Gm-Message-State: AA6/9Rniam39pBcZDZcFrMnLtbiPZduZs5gPoZ1sTiikl9qy/RCVe4eqdOAv3ZvSBKxBeg== X-Received: by 10.55.73.23 with SMTP id w23mr8631245qka.13.1476399021301; Thu, 13 Oct 2016 15:50:21 -0700 (PDT) Received: from vimacbookpro.vimeows.com (nyv-exweb.iac.com. [216.112.252.10]) by smtp.gmail.com with ESMTPSA id y33sm5988631qtc.34.2016.10.13.15.50.20 for (version=TLS1 cipher=AES128-SHA bits=128/128); Thu, 13 Oct 2016 15:50:20 -0700 (PDT) From: Vittorio Giovara To: ffmpeg-devel@ffmpeg.org Date: Thu, 13 Oct 2016 18:50:19 -0400 Message-Id: <20161013225019.71643-1-vittorio.giovara@gmail.com> X-Mailer: git-send-email 2.10.0 In-Reply-To: References: Subject: [FFmpeg-devel] [PATCHv3] mov: Evaluate the movie display matrix X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This matrix needs to be applied after all others have (currently only display matrix from trak), but cannot be handled in movie box, since streams are not allocated yet. So store it in main context, and apply it when appropriate, that is after parsing the tkhd one. Signed-off-by: Vittorio Giovara --- Reworked second matrix handling so that it is always applied, which makes the code easier to read and test. Updated according reviews, rolled back a couple of points for the reasons explained in the thread. Needs a new sample to be uploaded to fate, https://www.dropbox.com/s/qfio4bjhkpz3p4o/displaymatrix.mov?dl=0, the previous one can be deleted. Cheers, Vittorio libavformat/isom.h | 2 ++ libavformat/mov.c | 53 +++++++++++++++++++++++++++++++-------- tests/fate/mov.mak | 6 ++++- tests/ref/fate/mov-display-matrix | 10 ++++++++ 4 files changed, 59 insertions(+), 12 deletions(-) create mode 100644 tests/ref/fate/mov-display-matrix diff --git a/libavformat/isom.h b/libavformat/isom.h index 2246fed..2aeb8fa 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -238,6 +238,8 @@ typedef struct MOVContext { uint8_t *decryption_key; int decryption_key_len; int enable_drefs; + + int32_t movie_display_matrix[3][3]; ///< display matrix from mvhd } MOVContext; int ff_mp4_read_descr_len(AVIOContext *pb); diff --git a/libavformat/mov.c b/libavformat/mov.c index a15c8d1..e8da77f 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -1211,6 +1211,7 @@ static int mov_read_mdhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) static int mov_read_mvhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) { + int i; int64_t creation_time; int version = avio_r8(pb); /* version */ avio_rb24(pb); /* flags */ @@ -1238,7 +1239,12 @@ static int mov_read_mvhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) avio_skip(pb, 10); /* reserved */ - avio_skip(pb, 36); /* display matrix */ + /* movie display matrix, store it in main context and use it later on */ + for (i = 0; i < 3; i++) { + c->movie_display_matrix[i][0] = avio_rb32(pb); // 16.16 fixed point + c->movie_display_matrix[i][1] = avio_rb32(pb); // 16.16 fixed point + c->movie_display_matrix[i][2] = avio_rb32(pb); // 2.30 fixed point + } avio_rb32(pb); /* preview time */ avio_rb32(pb); /* preview duration */ @@ -3798,16 +3804,33 @@ static int mov_read_meta(MOVContext *c, AVIOContext *pb, MOVAtom atom) return 0; } +// return 0 when matrix is identity, 1 otherwise +#define IS_MATRIX_FULL(matrix) \ + (matrix[0][0] != (1 << 16) || \ + matrix[1][1] != (1 << 16) || \ + matrix[2][2] != (1 << 30) || \ + matrix[0][1] || matrix[0][2] || \ + matrix[1][0] || matrix[1][2] || \ + matrix[2][0] || matrix[2][1]) + +// fixed point to double +#define CONV_FP(x, sh) ((double) (x)) / (1 << (sh)) + +// double to fixed point +#define CONV_DB(x, sh) ((int32_t) ((x) * (1 << (sh)))) + static int mov_read_tkhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) { - int i; + int i, j, e; int width; int height; int display_matrix[3][3]; + int res_display_matrix[3][3]; AVStream *st; MOVStreamContext *sc; int version; int flags; + double val = 0; if (c->fc->nb_streams < 1) return 0; @@ -3853,15 +3876,22 @@ static int mov_read_tkhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) sc->width = width >> 16; sc->height = height >> 16; + // apply the moov display matrix + for (i = 0; i < 3; i++) { + for (j = 0; j < 3; j++) { + int sh = j == 2 ? 30 : 16; + for (e = 0; e < 3; e++) { + val += CONV_FP(display_matrix[i][e], sh) * + CONV_FP(c->movie_display_matrix[e][j], sh); + } + res_display_matrix[i][j] = CONV_DB(val, sh); + val = 0; + } + } + // save the matrix and add rotate metadata when it is not the default // identity - if (display_matrix[0][0] != (1 << 16) || - display_matrix[1][1] != (1 << 16) || - display_matrix[2][2] != (1 << 30) || - display_matrix[0][1] || display_matrix[0][2] || - display_matrix[1][0] || display_matrix[1][2] || - display_matrix[2][0] || display_matrix[2][1]) { - int i, j; + if (IS_MATRIX_FULL(res_display_matrix)) { double rotate; av_freep(&sc->display_matrix); @@ -3871,7 +3901,7 @@ static int mov_read_tkhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) for (i = 0; i < 3; i++) for (j = 0; j < 3; j++) - sc->display_matrix[i * 3 + j] = display_matrix[i][j]; + sc->display_matrix[i * 3 + j] = res_display_matrix[i][j]; rotate = av_display_rotation_get(sc->display_matrix); if (!isnan(rotate)) { @@ -3890,7 +3920,8 @@ static int mov_read_tkhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) double disp_transform[2]; for (i = 0; i < 2; i++) - disp_transform[i] = hypot(display_matrix[i][0], display_matrix[i][1]); + disp_transform[i] = hypot(sc->display_matrix[i * 3], + sc->display_matrix[i * 3 + 1]); if (disp_transform[0] > 0 && disp_transform[1] > 0 && disp_transform[0] < (1<<24) && disp_transform[1] < (1<<24) && diff --git a/tests/fate/mov.mak b/tests/fate/mov.mak index 4b5885b..42a4296 100644 --- a/tests/fate/mov.mak +++ b/tests/fate/mov.mak @@ -4,7 +4,8 @@ FATE_MOV = fate-mov-3elist \ fate-mov-1elist-noctts \ fate-mov-elist-starts-ctts-2ndsample \ fate-mov-1elist-ends-last-bframe \ - fate-mov-2elist-elist1-ends-bframe + fate-mov-2elist-elist1-ends-bframe \ + fate-mov-display-matrix \ FATE_SAMPLES_AVCONV += $(FATE_MOV) @@ -26,3 +27,6 @@ fate-mov-1elist-ends-last-bframe: CMD = framemd5 -i $(TARGET_SAMPLES)/mov/mov-1e # Makes sure that we handle timestamps of packets in case of multiple edit lists with one of them ending on a B-frame correctly. fate-mov-2elist-elist1-ends-bframe: CMD = framemd5 -i $(TARGET_SAMPLES)/mov/mov-2elist-elist1-ends-bframe.mov + +# File contains both a movie and a track display matrix +fate-mov-display-matrix: CMD = framemd5 -i $(TARGET_SAMPLES)/mov/displaymatrix.mov -frames 1 diff --git a/tests/ref/fate/mov-display-matrix b/tests/ref/fate/mov-display-matrix new file mode 100644 index 0000000..762e421 --- /dev/null +++ b/tests/ref/fate/mov-display-matrix @@ -0,0 +1,10 @@ +#format: frame checksums +#version: 2 +#hash: MD5 +#tb 0: 1001/30000 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 240x160 +#sar 0: 1/2 +#stream#, dts, pts, duration, size, hash +0, 0, 0, 1, 57600, be949aa661551010f461069804f68e76