From patchwork Mon Jun 19 23:42:37 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hein-Pieter van Braam X-Patchwork-Id: 4052 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.22.4 with SMTP id 4csp1139958vsw; Mon, 19 Jun 2017 16:43:32 -0700 (PDT) X-Received: by 10.223.176.253 with SMTP id j58mr17085840wra.65.1497915812286; Mon, 19 Jun 2017 16:43:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1497915812; cv=none; d=google.com; s=arc-20160816; b=UZbyQBm9NKrRi/Odc64rUGkXUrFO98O1O2f+oh+lQzziqacRvLhTUZ1XKJR9TkoB35 M5PDTjFO3TXszcKseIt+XuLotroBPUPN9X/pz3BJRtgFD7mj91AS7toHOfZQZG2CwRvG a1Y/3nKXgDm1oB7rV/ufIhNRstAAXVLHSc+6DY3lPLocF4LoigiV8ZEzZhDF1yN2GmU+ +vR5YDMlhOaZrLPjLLfK3S6ESJ+01uXgngfHM0oL2CbpgCCky53s5+F7yomcBH7atiE1 Nk3yh17SJYGP06C2iY4rsXTkcs5faeTdvGHS43fIqkUJBAOUHC5JwfRQb7KWODT96RxD cTyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:dkim-filter:delivered-to :arc-authentication-results; bh=G6Z4KE/IyP1IhVR7HRJiTRq+6Gg4CNdqg8qJySaLmSE=; b=w873mHgXv9awcmha1PH5rFhy4BJ8qNdCRBzgrLBSBSyqJ6wKp4hNc3G7glyBis4UMr wK3zIQI1E0iY6+n9bWfG8zfasdPXIHmWAynKSdOjJWAOdNldVwwYIJH/SL9qBW0eILyG eMNzcPzE/cm+LKUj7xY4uxx3aPvScA8kicucoVkfiTpPd4JuQWQoNMhpsij6LoZidWr0 MyBHaWhtiq+o6uV5LT0qr677+xZ0k62XUF0J4K754WPyp0I9/Jx94HiNXhouJ3xAAMj4 llNWwRT8QtRT2DcABlofHZWKFYKys2msk23jZE8NGKeYey6Pi325ssz8bPK/gVW8a31c IAdQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm.cx header.b=OLjFbmey; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=REJECT dis=NONE) header.from=tmm.cx Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n68si7602918wmn.63.2017.06.19.16.43.31; Mon, 19 Jun 2017 16:43:32 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm.cx header.b=OLjFbmey; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=REJECT dis=NONE) header.from=tmm.cx Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4D73E68A636; Tue, 20 Jun 2017 02:42:50 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from eva.tmm.cx (eva.tmm.cx [5.9.73.21]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0C99268A639 for ; Tue, 20 Jun 2017 02:42:43 +0300 (EEST) Received: from lola.ttg.global (unknown [92.110.172.3]) by eva.tmm.cx (Postfix) with ESMTPSA id 50D1217DD535; Mon, 19 Jun 2017 23:42:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 eva.tmm.cx 50D1217DD535 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tmm.cx; s=mail; t=1497915768; bh=vvu11wJ0+s70fP+Ww/YoSI521O+bmWJgnIlq3BLwdvg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OLjFbmeyegs4OcZRSmLIQDOuWRZgQghcEHyJ3A+soyWaoHj8AgUmabxG2uDTb9E3n kqfw41fpirvZVSeqfvGNWaNr/PaivfCSRIGcSEiEAl8HicKmouHcW2CJuZ0l+DcHF+ ZCWrzFI6AhR/O9QmqmfQWN4gW85dmQu4ZoOd76gE= From: Hein-Pieter van Braam To: FFmpeg development discussions and patches Date: Tue, 20 Jun 2017 01:42:37 +0200 Message-Id: <20170619234238.1364-5-hp@tmm.cx> X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170619234238.1364-1-hp@tmm.cx> References: <20170619234238.1364-1-hp@tmm.cx> X-Spam-Status: No, score=0.0 required=2.4 tests=UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on eva.tmm.cx Subject: [FFmpeg-devel] [PATCH v2 4/5] Interplay MVE: Implement frame format 0x10 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Hein-Pieter van Braam MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This implements the 0x10 frame format for Interplay MVE movies. The format is a variation on the 0x06 format with some changes. In addition to the decoding map there's also a skip map. This skip map is used to determine what 8x8 blocks can change in a particular frame. This format expects to be able to copy an 8x8 block from before the last time it was changed. This can be an arbitrary time in the past. In order to implement this this decoder allocates two additional AVFrames where actual decoding happens. At the end of a frame decoding changed blocks are copied to a finished frame based on the skip map. The skip map's encoding is a little convulted, I'll refer to the code for details. Values in the decoding map are the same as in format 0x06. Signed-off-by: Hein-Pieter van Braam --- libavcodec/interplayvideo.c | 182 ++++++++++++++++++++++++++++++++++++++++++-- libavformat/ipmovie.c | 68 +++++++++++++---- 2 files changed, 227 insertions(+), 23 deletions(-) diff --git a/libavcodec/interplayvideo.c b/libavcodec/interplayvideo.c index 431eeb1..421de26 100644 --- a/libavcodec/interplayvideo.c +++ b/libavcodec/interplayvideo.c @@ -55,8 +55,15 @@ typedef struct IpvideoContext { HpelDSPContext hdsp; AVFrame *second_last_frame; AVFrame *last_frame; + + /* For format 0x10 */ + AVFrame *cur_decode_frame; + AVFrame *prev_decode_frame; + const unsigned char *decoding_map; int decoding_map_size; + const unsigned char *skip_map; + int skip_map_size; int is_16bpp; GetByteContext stream_ptr, mv_ptr; @@ -977,6 +984,114 @@ static void ipvideo_decode_format_06_opcodes(IpvideoContext *s, AVFrame *frame) } } +static void ipvideo_format_10_firstpass(IpvideoContext *s, AVFrame *frame, short opcode) +{ + int line; + + if (!opcode) { + for (line = 0; line < 8; ++line) { + bytestream2_get_buffer(&s->stream_ptr, s->pixel_ptr, 8); + s->pixel_ptr += s->stride; + } + } +} + +static void ipvideo_format_10_secondpass(IpvideoContext *s, AVFrame *frame, short opcode) +{ + int off_x, off_y; + + if (opcode < 0) { + off_x = ((unsigned short)opcode - 0xC000) % s->cur_decode_frame->linesize[0]; + off_y = ((unsigned short)opcode - 0xC000) / s->cur_decode_frame->linesize[0]; + copy_from(s, s->prev_decode_frame, s->cur_decode_frame, off_x, off_y); + } else if (opcode > 0) { + off_x = ((unsigned short)opcode - 0x4000) % s->cur_decode_frame->linesize[0]; + off_y = ((unsigned short)opcode - 0x4000) / s->cur_decode_frame->linesize[0]; + copy_from(s, s->cur_decode_frame, s->cur_decode_frame, off_x, off_y); + } +} + +static void (* const ipvideo_format_10_passes[])(IpvideoContext *s, AVFrame *frame, short op) = { + ipvideo_format_10_firstpass, ipvideo_format_10_secondpass, +}; + +static void ipvideo_decode_format_10_opcodes(IpvideoContext *s, AVFrame *frame) +{ + int pass, x, y, changed_block; + short opcode, skip; + GetByteContext decoding_map_ptr; + GetByteContext skip_map_ptr; + + bytestream2_skip(&s->stream_ptr, 14); /* data starts 14 bytes in */ + + /* this is PAL8, so make the palette available */ + memcpy(frame->data[1], s->pal, AVPALETTE_SIZE); + s->stride = frame->linesize[0]; + + s->line_inc = s->stride - 8; + s->upper_motion_limit_offset = (s->avctx->height - 8) * frame->linesize[0] + + (s->avctx->width - 8) * (1 + s->is_16bpp); + + bytestream2_init(&decoding_map_ptr, s->decoding_map, s->decoding_map_size); + bytestream2_init(&skip_map_ptr, s->skip_map, s->skip_map_size); + + for (pass = 0; pass < 2; ++pass) { + bytestream2_seek(&decoding_map_ptr, 0, SEEK_SET); + bytestream2_seek(&skip_map_ptr, 0, SEEK_SET); + skip = bytestream2_get_le16(&skip_map_ptr); + + for (y = 0; y < s->avctx->height; y += 8) { + for (x = 0; x < s->avctx->width; x += 8) { + s->pixel_ptr = s->cur_decode_frame->data[0] + x + y * s->cur_decode_frame->linesize[0]; + + while (skip <= 0) { + if (skip != -0x8000 && skip) { + opcode = bytestream2_get_le16(&decoding_map_ptr); + ipvideo_format_10_passes[pass](s, frame, opcode); + break; + } + skip = bytestream2_get_le16(&skip_map_ptr); + } + skip *= 2; + } + } + } + + bytestream2_seek(&skip_map_ptr, 0, SEEK_SET); + skip = bytestream2_get_le16(&skip_map_ptr); + for (y = 0; y < s->avctx->height; y += 8) { + for (x = 0; x < s->avctx->width; x += 8) { + changed_block = 0; + s->pixel_ptr = frame->data[0] + x + y*frame->linesize[0]; + + while (skip <= 0) { + if (skip != -0x8000 && skip) { + changed_block = 1; + break; + } + skip = bytestream2_get_le16(&skip_map_ptr); + } + + if (changed_block) { + copy_from(s, s->cur_decode_frame, frame, 0, 0); + } else { + /* Don't try to copy last_frame data on the first frame */ + if (s->avctx->frame_number) + copy_from(s, s->last_frame, frame, 0, 0); + } + skip *= 2; + } + } + + FFSWAP(AVFrame*, s->prev_decode_frame, s->cur_decode_frame); + + if (bytestream2_get_bytes_left(&s->stream_ptr) > 1) { + av_log(s->avctx, AV_LOG_DEBUG, + "decode finished with %d bytes left over\n", + bytestream2_get_bytes_left(&s->stream_ptr)); + } +} + static void ipvideo_decode_format_11_opcodes(IpvideoContext *s, AVFrame *frame) { int x, y; @@ -1046,12 +1161,27 @@ static av_cold int ipvideo_decode_init(AVCodecContext *avctx) s->last_frame = av_frame_alloc(); s->second_last_frame = av_frame_alloc(); - if (!s->last_frame || !s->second_last_frame) { + s->cur_decode_frame = av_frame_alloc(); + s->prev_decode_frame = av_frame_alloc(); + if (!s->last_frame || !s->second_last_frame || + !s->cur_decode_frame || !s->prev_decode_frame) { av_frame_free(&s->last_frame); av_frame_free(&s->second_last_frame); + av_frame_free(&s->cur_decode_frame); + av_frame_free(&s->prev_decode_frame); return AVERROR(ENOMEM); } + s->cur_decode_frame->width = avctx->width; + s->prev_decode_frame->width = avctx->width; + s->cur_decode_frame->height = avctx->height; + s->prev_decode_frame->height = avctx->height; + s->cur_decode_frame->format = avctx->pix_fmt; + s->prev_decode_frame->format = avctx->pix_fmt; + + ff_get_buffer(avctx, s->cur_decode_frame, 0); + ff_get_buffer(avctx, s->prev_decode_frame, 0); + return 0; } @@ -1073,13 +1203,14 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, av_frame_unref(s->second_last_frame); } - if (buf_size < 6) + if (buf_size < 8) return AVERROR_INVALIDDATA; frame_format = AV_RL8(buf); send_buffer = AV_RL8(buf + 1); video_data_size = AV_RL16(buf + 2); s->decoding_map_size = AV_RL16(buf + 4); + s->skip_map_size = AV_RL16(buf + 6); switch(frame_format) { case 0x06: @@ -1088,6 +1219,11 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, return AVERROR_INVALIDDATA; } + if (s->skip_map_size) { + av_log(avctx, AV_LOG_ERROR, "Skip map for format 0x06\n"); + return AVERROR_INVALIDDATA; + } + if (s->is_16bpp) { av_log(avctx, AV_LOG_ERROR, "Video format 0x06 does not support 16bpp movies\n"); return AVERROR_INVALIDDATA; @@ -1095,9 +1231,31 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, /* Decoding map for 0x06 frame format is at the top of pixeldata */ s->decoding_map_size = ((s->avctx->width / 8) * (s->avctx->height / 8)) * 2; - s->decoding_map = buf + 6 + 14; /* 14 bits of op data */ + s->decoding_map = buf + 8 + 14; /* 14 bits of op data */ video_data_size -= s->decoding_map_size + 14; - bytestream2_init(&s->stream_ptr, buf + 6 + s->decoding_map_size + 14, video_data_size); + bytestream2_init(&s->stream_ptr, buf + 8 + s->decoding_map_size + 14, video_data_size); + + break; + + case 0x10: + if (! s->decoding_map_size) { + av_log(avctx, AV_LOG_ERROR, "Empty decoding map for format 0x10\n"); + return AVERROR_INVALIDDATA; + } + + if (! s->skip_map_size) { + av_log(avctx, AV_LOG_ERROR, "Empty skip map for format 0x10\n"); + return AVERROR_INVALIDDATA; + } + + if (s->is_16bpp) { + av_log(avctx, AV_LOG_ERROR, "Video format 0x10 does not support 16bpp movies\n"); + return AVERROR_INVALIDDATA; + } + + bytestream2_init(&s->stream_ptr, buf + 8, video_data_size); + s->decoding_map = buf + 8 + video_data_size; + s->skip_map = buf + 8 + video_data_size + s->decoding_map_size; break; @@ -1107,8 +1265,13 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, return AVERROR_INVALIDDATA; } - bytestream2_init(&s->stream_ptr, buf + 6, video_data_size); - s->decoding_map = buf + 6 + video_data_size; + if (s->skip_map_size) { + av_log(avctx, AV_LOG_ERROR, "Skip map for format 0x11\n"); + return AVERROR_INVALIDDATA; + } + + bytestream2_init(&s->stream_ptr, buf + 8, video_data_size); + s->decoding_map = buf + 8 + video_data_size; break; @@ -1117,7 +1280,7 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, } /* ensure we can't overread the packet */ - if (buf_size < 6 + s->decoding_map_size + video_data_size) { + if (buf_size < 8 + s->decoding_map_size + video_data_size + s->skip_map_size) { av_log(avctx, AV_LOG_ERROR, "Invalid IP packet size\n"); return AVERROR_INVALIDDATA; } @@ -1140,6 +1303,9 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, case 0x06: ipvideo_decode_format_06_opcodes(s, frame); break; + case 0x10: + ipvideo_decode_format_10_opcodes(s, frame); + break; case 0x11: ipvideo_decode_format_11_opcodes(s, frame); break; @@ -1163,6 +1329,8 @@ static av_cold int ipvideo_decode_end(AVCodecContext *avctx) av_frame_free(&s->last_frame); av_frame_free(&s->second_last_frame); + av_frame_free(&s->cur_decode_frame); + av_frame_free(&s->prev_decode_frame); return 0; } diff --git a/libavformat/ipmovie.c b/libavformat/ipmovie.c index 7f2454b..7f5a8c6 100644 --- a/libavformat/ipmovie.c +++ b/libavformat/ipmovie.c @@ -66,9 +66,9 @@ #define OPCODE_CREATE_GRADIENT 0x0B #define OPCODE_SET_PALETTE 0x0C #define OPCODE_SET_PALETTE_COMPRESSED 0x0D -#define OPCODE_UNKNOWN_0E 0x0E +#define OPCODE_SET_SKIP_MAP 0x0E #define OPCODE_SET_DECODING_MAP 0x0F -#define OPCODE_UNKNOWN_10 0x10 +#define OPCODE_VIDEO_DATA_10 0x10 #define OPCODE_VIDEO_DATA_11 0x11 #define OPCODE_UNKNOWN_12 0x12 #define OPCODE_UNKNOWN_13 0x13 @@ -107,6 +107,8 @@ typedef struct IPMVEContext { int audio_chunk_size; int64_t video_chunk_offset; int video_chunk_size; + int64_t skip_map_chunk_offset; + int skip_map_chunk_size; int64_t decode_map_chunk_offset; int decode_map_chunk_size; @@ -156,9 +158,9 @@ static int load_ipmovie_packet(IPMVEContext *s, AVIOContext *pb, } else if (s->frame_format) { - /* send the frame format, decode map, the video data, and the send_buffer flag together */ + /* send the frame format, decode map, the video data, skip map, and the send_buffer flag together */ - if (av_new_packet(pkt, 6 + s->decode_map_chunk_size + s->video_chunk_size)) + if (av_new_packet(pkt, 8 + s->decode_map_chunk_size + s->video_chunk_size + s->skip_map_chunk_size)) return CHUNK_NOMEM; if (s->has_palette) { @@ -181,6 +183,7 @@ static int load_ipmovie_packet(IPMVEContext *s, AVIOContext *pb, AV_WL8(pkt->data + 1, s->send_buffer); AV_WL16(pkt->data + 2, s->video_chunk_size); AV_WL16(pkt->data + 4, s->decode_map_chunk_size); + AV_WL16(pkt->data + 6, s->skip_map_chunk_size); s->frame_format = 0; s->send_buffer = 0; @@ -189,24 +192,39 @@ static int load_ipmovie_packet(IPMVEContext *s, AVIOContext *pb, avio_seek(pb, s->video_chunk_offset, SEEK_SET); s->video_chunk_offset = 0; - if (avio_read(pb, pkt->data + 6, s->video_chunk_size) != + if (avio_read(pb, pkt->data + 8, s->video_chunk_size) != s->video_chunk_size) { av_packet_unref(pkt); return CHUNK_EOF; } - pkt->pos = s->decode_map_chunk_offset; - avio_seek(pb, s->decode_map_chunk_offset, SEEK_SET); - s->decode_map_chunk_offset = 0; + if (s->decode_map_chunk_size) { + pkt->pos = s->decode_map_chunk_offset; + avio_seek(pb, s->decode_map_chunk_offset, SEEK_SET); + s->decode_map_chunk_offset = 0; - if (avio_read(pb, pkt->data + 6 + s->video_chunk_size, - s->decode_map_chunk_size) != s->decode_map_chunk_size) { - av_packet_unref(pkt); - return CHUNK_EOF; + if (avio_read(pb, pkt->data + 8 + s->video_chunk_size, + s->decode_map_chunk_size) != s->decode_map_chunk_size) { + av_packet_unref(pkt); + return CHUNK_EOF; + } + } + + if (s->skip_map_chunk_size) { + pkt->pos = s->skip_map_chunk_offset; + avio_seek(pb, s->skip_map_chunk_offset, SEEK_SET); + s->skip_map_chunk_offset = 0; + + if (avio_read(pb, pkt->data + 8 + s->video_chunk_size + s->decode_map_chunk_size, + s->skip_map_chunk_size) != s->skip_map_chunk_size) { + av_packet_unref(pkt); + return CHUNK_EOF; + } } s->video_chunk_size = 0; s->decode_map_chunk_size = 0; + s->skip_map_chunk_size = 0; pkt->stream_index = s->video_stream_index; pkt->pts = s->video_pts; @@ -444,8 +462,6 @@ static int process_ipmovie_chunk(IPMVEContext *s, AVIOContext *pb, s->video_width, s->video_height); break; - case OPCODE_UNKNOWN_0E: - case OPCODE_UNKNOWN_10: case OPCODE_UNKNOWN_12: case OPCODE_UNKNOWN_13: case OPCODE_UNKNOWN_14: @@ -527,6 +543,15 @@ static int process_ipmovie_chunk(IPMVEContext *s, AVIOContext *pb, avio_skip(pb, opcode_size); break; + case OPCODE_SET_SKIP_MAP: + av_log(s->avf, AV_LOG_TRACE, "set skip map\n"); + + /* log position and move on for now */ + s->skip_map_chunk_offset = avio_tell(pb); + s->skip_map_chunk_size = opcode_size; + avio_skip(pb, opcode_size); + break; + case OPCODE_SET_DECODING_MAP: av_log(s->avf, AV_LOG_TRACE, "set decoding map\n"); @@ -546,6 +571,16 @@ static int process_ipmovie_chunk(IPMVEContext *s, AVIOContext *pb, avio_skip(pb, opcode_size); break; + case OPCODE_VIDEO_DATA_10: + av_log(s->avf, AV_LOG_TRACE, "set video data format 0x10\n"); + s->frame_format = 0x10; + + /* log position and move on for now */ + s->video_chunk_offset = avio_tell(pb); + s->video_chunk_size = opcode_size; + avio_skip(pb, opcode_size); + break; + case OPCODE_VIDEO_DATA_11: av_log(s->avf, AV_LOG_TRACE, "set video data format 0x11\n"); s->frame_format = 0x11; @@ -614,8 +649,9 @@ static int ipmovie_read_header(AVFormatContext *s) /* initialize private context members */ ipmovie->video_pts = ipmovie->audio_frame_count = 0; ipmovie->audio_chunk_offset = ipmovie->video_chunk_offset = - ipmovie->decode_map_chunk_offset = 0; - ipmovie->decode_map_chunk_size = ipmovie->video_chunk_size = 0; + ipmovie->decode_map_chunk_offset = ipmovie->skip_map_chunk_offset = 0; + ipmovie->decode_map_chunk_size = ipmovie->video_chunk_size = + ipmovie->skip_map_chunk_size = 0; ipmovie->send_buffer = ipmovie->frame_format = 0; /* on the first read, this will position the stream at the first chunk */