From patchwork Sun Jun 18 01:25:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hein-Pieter van Braam X-Patchwork-Id: 4016 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.22.4 with SMTP id 4csp298827vsw; Sat, 17 Jun 2017 18:26:46 -0700 (PDT) X-Received: by 10.223.175.18 with SMTP id z18mr7506046wrc.22.1497749206742; Sat, 17 Jun 2017 18:26:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1497749206; cv=none; d=google.com; s=arc-20160816; b=Y5ZnNquQApL6YgvQ/9eSYJeWG2gKU/lam/eq6OD24bQIfAcNVSnVFX+LIUJNXOVyS1 qB9rHKKZ8Hb1Si3FdrWYnKnFZhOFIYzku61HnvLdCDmRyGCLtdeh0rIPQyQ/hUKe5pBj fIb+LSbCGd7esFTQ/WJHw6pUj7SNpPDZXLHZiYjJLVAav+g+ZrGI+HlUV8+GgPhL8zZB fvWwVL44rKKWagUJerS9dpGohf7eAqGsvDOXz8LQcAZGW7QfibasB2995t1NTui22akv uTlvKyGQj9H8XHitTm0fB1t1HslJYhjGo8XEUyQRyuWn8xOmb5qVWaB3M+M/W8wJOZOU kqtg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:dkim-filter:delivered-to :arc-authentication-results; bh=7V7V3/RswQ5pLBne0/sDZqM6j3NPZEFVihAcZ8m+nnk=; b=ulSJfeIevDlpknYFsgi0hSfxrIZqnU1BYEKiIe4BpjNVPVw0WF/6dTzFjj7fVvCPqO MxsEtAvkKtQA+S8BXxc7T2O64f5Eal+cpgwVjhyX8SaLUbVL0Ljj2uRLFWZP/AVE1I5y z6S0H2p+Ox3dCBKISpD+1b9xfha5vbnSQ5rM79cEbWGC+CopJxIZT02eRaK9khNh0O1f VTI8Yxw4Wly/XFoTRtX2ESeu1tULgTKVxYkgw6km7XE74EpD3qCJDB7xcn6NGjfO8gp5 MepWrg4R0q+mWTmN9QyrhxXDj9KRjjvL9jWWs/i+Pyo/RrcB+hP/xLrIdBpx6AsjUDDC gw8A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm.cx header.b=qkDdBH+V; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=REJECT dis=NONE) header.from=tmm.cx Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r15si6771993wrb.259.2017.06.17.18.26.46; Sat, 17 Jun 2017 18:26:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm.cx header.b=qkDdBH+V; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=REJECT dis=NONE) header.from=tmm.cx Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4C053680D19; Sun, 18 Jun 2017 04:25:57 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from eva.tmm.cx (eva.tmm.cx [5.9.73.21]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 14D1A689F53 for ; Sun, 18 Jun 2017 04:25:48 +0300 (EEST) Received: from lola.ttg.global (unknown [92.110.172.3]) by eva.tmm.cx (Postfix) with ESMTPSA id 67C8117CE638; Sun, 18 Jun 2017 01:25:51 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 eva.tmm.cx 67C8117CE638 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tmm.cx; s=mail; t=1497749152; bh=ywMxeFE2BRlOjoTuTTmVI/LWkGds8pA5npIZ9rXm5rw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qkDdBH+VVAYMbv31YgFKeZADdgdXmaz4HGuknFfVzdAK9WXnyJ4ArFAZfi4FT4ppM Di7ETwxcd5oYVmkDh//xbDK1cn+DpCboIrTUs6+UcD9jCBXzT58fE6QyKsAVkX7xYP ZSsnZ3uV4PRHBPvuYToz/eHCUgIUcMOyIRpd9tO4= From: Hein-Pieter van Braam To: FFmpeg development discussions and patches Date: Sun, 18 Jun 2017 03:25:46 +0200 Message-Id: <20170618012547.13614-5-hp@tmm.cx> X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170618012547.13614-1-hp@tmm.cx> References: <20170618012547.13614-1-hp@tmm.cx> X-Spam-Status: No, score=0.0 required=2.4 tests=UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on eva.tmm.cx Subject: [FFmpeg-devel] [PATCH 4/5] Interplay MVE: Implement frame format 0x10 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Hein-Pieter van Braam MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This implements the 0x10 frame format for Interplay MVE movies. The format is a variation on the 0x06 format with some changes. In addition to the decoding map there's also a skip map. This skip map is used to determine what 8x8 blocks can change in a particular frame. This format expects to be able to copy an 8x8 block from before the last time it was changed. This can be an arbitrary time in the past. In order to implement this this decoder allocates two additional AVFrames where actual decoding happens. At the end of a frame decoding changed blocks are copied to a finished frame based on the skip map. The skip map's encoding is a little convulted, I'll refer to the code for details. Values in the decoding map are the same as in format 0x06. Signed-off-by: Hein-Pieter van Braam --- libavcodec/interplayvideo.c | 179 ++++++++++++++++++++++++++++++++++++++++++-- libavformat/ipmovie.c | 68 +++++++++++++---- 2 files changed, 226 insertions(+), 21 deletions(-) diff --git a/libavcodec/interplayvideo.c b/libavcodec/interplayvideo.c index e227533..69c3a78 100644 --- a/libavcodec/interplayvideo.c +++ b/libavcodec/interplayvideo.c @@ -55,8 +55,15 @@ typedef struct IpvideoContext { HpelDSPContext hdsp; AVFrame *second_last_frame; AVFrame *last_frame; + + /* For format 0x10 */ + AVFrame *cur_decode_frame; + AVFrame *prev_decode_frame; + const unsigned char *decoding_map; int decoding_map_size; + const unsigned char *skip_map; + int skip_map_size; int is_16bpp; GetByteContext stream_ptr, mv_ptr; @@ -978,6 +985,115 @@ static void ipvideo_decode_format_06_opcodes(IpvideoContext *s, AVFrame *frame) } } +static void ipvideo_format_10_firstpass(IpvideoContext *s, AVFrame *frame, short opcode) +{ + int line; + + if (! opcode) { + for (line = 0; line < 8; ++line) { + bytestream2_get_buffer(&s->stream_ptr, s->pixel_ptr, 8); + s->pixel_ptr += s->stride; + } + } +} + +static void ipvideo_format_10_secondpass(IpvideoContext *s, AVFrame *frame, short opcode) +{ + int off_x, off_y; + + if (opcode < 0) { + off_x = ((unsigned short)opcode - 0xC000) % s->cur_decode_frame->linesize[0]; + off_y = ((unsigned short)opcode - 0xC000) / s->cur_decode_frame->linesize[0]; + copy_from(s, s->prev_decode_frame, s->cur_decode_frame, off_x, off_y); + } + if (opcode > 0) { + off_x = ((unsigned short)opcode - 0x4000) % s->cur_decode_frame->linesize[0]; + off_y = ((unsigned short)opcode - 0x4000) / s->cur_decode_frame->linesize[0]; + copy_from(s, s->cur_decode_frame, s->cur_decode_frame, off_x, off_y); + } +} + +static void (* const ipvideo_format_10_passes[])(IpvideoContext *s, AVFrame *frame, short op) = { + ipvideo_format_10_firstpass, ipvideo_format_10_secondpass, +}; + +static void ipvideo_decode_format_10_opcodes(IpvideoContext *s, AVFrame *frame) +{ + int pass, x, y, changed_block; + short opcode, skip; + GetByteContext decoding_map_ptr; + GetByteContext skip_map_ptr; + + bytestream2_skip(&s->stream_ptr, 14); /* data starts 14 bytes in */ + + /* this is PAL8, so make the palette available */ + memcpy(frame->data[1], s->pal, AVPALETTE_SIZE); + s->stride = frame->linesize[0]; + + s->line_inc = s->stride - 8; + s->upper_motion_limit_offset = (s->avctx->height - 8) * frame->linesize[0] + + (s->avctx->width - 8) * (1 + s->is_16bpp); + + bytestream2_init(&decoding_map_ptr, s->decoding_map, s->decoding_map_size); + bytestream2_init(&skip_map_ptr, s->skip_map, s->skip_map_size); + + for (pass = 0; pass < 2; ++pass) { + bytestream2_seek(&decoding_map_ptr, 0, SEEK_SET); + bytestream2_seek(&skip_map_ptr, 0, SEEK_SET); + skip = bytestream2_get_le16(&skip_map_ptr); + + for (y = 0; y < s->avctx->height; y += 8) { + for (x = 0; x < s->avctx->width; x += 8) { + s->pixel_ptr = s->cur_decode_frame->data[0] + x + y*s->cur_decode_frame->linesize[0]; + + while (skip <= 0) { + if (skip != -0x8000 && skip) { + opcode = bytestream2_get_le16(&decoding_map_ptr); + ipvideo_format_10_passes[pass](s, frame, opcode); + break; + } + skip = bytestream2_get_le16(&skip_map_ptr); + } + skip *= 2; + } + } + } + + bytestream2_seek(&skip_map_ptr, 0, SEEK_SET); + skip = bytestream2_get_le16(&skip_map_ptr); + for (y = 0; y < s->avctx->height; y += 8) { + for (x = 0; x < s->avctx->width; x += 8) { + changed_block = 0; + s->pixel_ptr = frame->data[0] + x + y*frame->linesize[0]; + + while (skip <= 0) { + if (skip != -0x8000 && skip) { + changed_block = 1; + break; + } + skip = bytestream2_get_le16(&skip_map_ptr); + } + + if (changed_block) { + copy_from(s, s->cur_decode_frame, frame, 0, 0); + } else { + /* Don't try to copy last_frame data on the first frame */ + if (s->avctx->frame_number) + copy_from(s, s->last_frame, frame, 0, 0); + } + skip *= 2; + } + } + + FFSWAP(AVFrame*, s->prev_decode_frame, s->cur_decode_frame); + + if (bytestream2_get_bytes_left(&s->stream_ptr) > 1) { + av_log(s->avctx, AV_LOG_DEBUG, + "decode finished with %d bytes left over\n", + bytestream2_get_bytes_left(&s->stream_ptr)); + } +} + static void ipvideo_decode_format_11_opcodes(IpvideoContext *s, AVFrame *frame) { int x, y; @@ -1047,12 +1163,27 @@ static av_cold int ipvideo_decode_init(AVCodecContext *avctx) s->last_frame = av_frame_alloc(); s->second_last_frame = av_frame_alloc(); - if (!s->last_frame || !s->second_last_frame) { + s->cur_decode_frame = av_frame_alloc(); + s->prev_decode_frame = av_frame_alloc(); + if (!s->last_frame || !s->second_last_frame || + !s->cur_decode_frame || !s->prev_decode_frame) { av_frame_free(&s->last_frame); av_frame_free(&s->second_last_frame); + av_frame_free(&s->cur_decode_frame); + av_frame_free(&s->prev_decode_frame); return AVERROR(ENOMEM); } + s->cur_decode_frame->width = avctx->width; + s->prev_decode_frame->width = avctx->width; + s->cur_decode_frame->height = avctx->height; + s->prev_decode_frame->height = avctx->height; + s->cur_decode_frame->format = avctx->pix_fmt; + s->prev_decode_frame->format = avctx->pix_fmt; + + ff_get_buffer(avctx, s->cur_decode_frame, 0); + ff_get_buffer(avctx, s->prev_decode_frame, 0); + return 0; } @@ -1081,6 +1212,7 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, send_buffer = AV_RL8(buf + 1); video_data_size = AV_RL16(buf + 2); s->decoding_map_size = AV_RL16(buf + 4); + s->skip_map_size = AV_RL16(buf + 6); switch(frame_format) { case 0x06: @@ -1089,6 +1221,11 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, return AVERROR_INVALIDDATA; } + if (s->skip_map_size) { + av_log(avctx, AV_LOG_ERROR, "Skip map for format 0x06\n"); + return AVERROR_INVALIDDATA; + } + if (s->is_16bpp) { av_log(avctx, AV_LOG_ERROR, "Video format 0x06 does not support 16bpp movies\n"); return AVERROR_INVALIDDATA; @@ -1096,8 +1233,30 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, /* Decoding map for 0x06 frame format is at the top of pixeldata */ s->decoding_map_size = ((s->avctx->width / 8) * (s->avctx->height / 8)) * 2; - s->decoding_map = buf + 6 + 14; /* 14 bits of op data */ - bytestream2_init(&s->stream_ptr, buf + 6 + s->decoding_map_size + 14, video_data_size); + s->decoding_map = buf + 8 + 14; /* 14 bits of op data */ + bytestream2_init(&s->stream_ptr, buf + 8 + s->decoding_map_size + 14, video_data_size); + + break; + + case 0x10: + if (! s->decoding_map_size) { + av_log(avctx, AV_LOG_ERROR, "Empty decoding map for format 0x10\n"); + return AVERROR_INVALIDDATA; + } + + if (! s->skip_map_size) { + av_log(avctx, AV_LOG_ERROR, "Empty skip map for format 0x10\n"); + return AVERROR_INVALIDDATA; + } + + if (s->is_16bpp) { + av_log(avctx, AV_LOG_ERROR, "Video format 0x10 does not support 16bpp movies\n"); + return AVERROR_INVALIDDATA; + } + + bytestream2_init(&s->stream_ptr, buf + 8, video_data_size); + s->decoding_map = buf + 8 + video_data_size; + s->skip_map = buf + 8 + video_data_size + s->decoding_map_size; break; @@ -1107,8 +1266,13 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, return AVERROR_INVALIDDATA; } - bytestream2_init(&s->stream_ptr, buf + 6, video_data_size); - s->decoding_map = buf + 6 + video_data_size; + if (s->skip_map_size) { + av_log(avctx, AV_LOG_ERROR, "Skip map for format 0x11\n"); + return AVERROR_INVALIDDATA; + } + + bytestream2_init(&s->stream_ptr, buf + 8, video_data_size); + s->decoding_map = buf + 8 + video_data_size; break; @@ -1139,6 +1303,9 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, case 0x06: ipvideo_decode_format_06_opcodes(s, frame); break; + case 0x10: + ipvideo_decode_format_10_opcodes(s, frame); + break; case 0x11: ipvideo_decode_format_11_opcodes(s, frame); break; @@ -1162,6 +1329,8 @@ static av_cold int ipvideo_decode_end(AVCodecContext *avctx) av_frame_free(&s->last_frame); av_frame_free(&s->second_last_frame); + av_frame_free(&s->cur_decode_frame); + av_frame_free(&s->prev_decode_frame); return 0; } diff --git a/libavformat/ipmovie.c b/libavformat/ipmovie.c index 6f4d90c..df3e059 100644 --- a/libavformat/ipmovie.c +++ b/libavformat/ipmovie.c @@ -66,9 +66,9 @@ #define OPCODE_CREATE_GRADIENT 0x0B #define OPCODE_SET_PALETTE 0x0C #define OPCODE_SET_PALETTE_COMPRESSED 0x0D -#define OPCODE_UNKNOWN_0E 0x0E +#define OPCODE_SET_SKIP_MAP 0x0E #define OPCODE_SET_DECODING_MAP 0x0F -#define OPCODE_UNKNOWN_10 0x10 +#define OPCODE_VIDEO_DATA_10 0x10 #define OPCODE_VIDEO_DATA_11 0x11 #define OPCODE_UNKNOWN_12 0x12 #define OPCODE_UNKNOWN_13 0x13 @@ -107,6 +107,8 @@ typedef struct IPMVEContext { int audio_chunk_size; int64_t video_chunk_offset; int video_chunk_size; + int64_t skip_map_chunk_offset; + int skip_map_chunk_size; int64_t decode_map_chunk_offset; int decode_map_chunk_size; @@ -156,9 +158,9 @@ static int load_ipmovie_packet(IPMVEContext *s, AVIOContext *pb, } else if (s->frame_format) { - /* send the frame format, decode map, the video data, and the send_buffer flag together */ + /* send the frame format, decode map, the video data, skip map, and the send_buffer flag together */ - if (av_new_packet(pkt, 6 + s->decode_map_chunk_size + s->video_chunk_size)) + if (av_new_packet(pkt, 8 + s->decode_map_chunk_size + s->video_chunk_size + s->skip_map_chunk_size)) return CHUNK_NOMEM; if (s->has_palette) { @@ -181,6 +183,7 @@ static int load_ipmovie_packet(IPMVEContext *s, AVIOContext *pb, AV_WL8(pkt->data + 1, s->send_buffer); AV_WL16(pkt->data + 2, s->video_chunk_size); AV_WL16(pkt->data + 4, s->decode_map_chunk_size); + AV_WL16(pkt->data + 6, s->skip_map_chunk_size); s->frame_format = 0; s->send_buffer = 0; @@ -189,24 +192,39 @@ static int load_ipmovie_packet(IPMVEContext *s, AVIOContext *pb, avio_seek(pb, s->video_chunk_offset, SEEK_SET); s->video_chunk_offset = 0; - if (avio_read(pb, pkt->data + 6, s->video_chunk_size) != + if (avio_read(pb, pkt->data + 8, s->video_chunk_size) != s->video_chunk_size) { av_packet_unref(pkt); return CHUNK_EOF; } - pkt->pos = s->decode_map_chunk_offset; - avio_seek(pb, s->decode_map_chunk_offset, SEEK_SET); - s->decode_map_chunk_offset = 0; + if (s->decode_map_chunk_size) { + pkt->pos = s->decode_map_chunk_offset; + avio_seek(pb, s->decode_map_chunk_offset, SEEK_SET); + s->decode_map_chunk_offset = 0; - if (avio_read(pb, pkt->data + 6 + s->video_chunk_size, - s->decode_map_chunk_size) != s->decode_map_chunk_size) { - av_packet_unref(pkt); - return CHUNK_EOF; + if (avio_read(pb, pkt->data + 8 + s->video_chunk_size, + s->decode_map_chunk_size) != s->decode_map_chunk_size) { + av_packet_unref(pkt); + return CHUNK_EOF; + } + } + + if (s->skip_map_chunk_size) { + pkt->pos = s->skip_map_chunk_offset; + avio_seek(pb, s->skip_map_chunk_offset, SEEK_SET); + s->skip_map_chunk_offset = 0; + + if (avio_read(pb, pkt->data + 8 + s->video_chunk_size + s->decode_map_chunk_size, + s->skip_map_chunk_size) != s->skip_map_chunk_size) { + av_packet_unref(pkt); + return CHUNK_EOF; + } } s->video_chunk_size = 0; s->decode_map_chunk_size = 0; + s->skip_map_chunk_size = 0; pkt->stream_index = s->video_stream_index; pkt->pts = s->video_pts; @@ -444,8 +462,6 @@ static int process_ipmovie_chunk(IPMVEContext *s, AVIOContext *pb, s->video_width, s->video_height); break; - case OPCODE_UNKNOWN_0E: - case OPCODE_UNKNOWN_10: case OPCODE_UNKNOWN_12: case OPCODE_UNKNOWN_13: case OPCODE_UNKNOWN_14: @@ -527,6 +543,15 @@ static int process_ipmovie_chunk(IPMVEContext *s, AVIOContext *pb, avio_skip(pb, opcode_size); break; + case OPCODE_SET_SKIP_MAP: + av_log(s->avf, AV_LOG_TRACE, "set skip map\n"); + + /* log position and move on for now */ + s->skip_map_chunk_offset = avio_tell(pb); + s->skip_map_chunk_size = opcode_size; + avio_skip(pb, opcode_size); + break; + case OPCODE_SET_DECODING_MAP: av_log(s->avf, AV_LOG_TRACE, "set decoding map\n"); @@ -546,6 +571,16 @@ static int process_ipmovie_chunk(IPMVEContext *s, AVIOContext *pb, avio_skip(pb, opcode_size); break; + case OPCODE_VIDEO_DATA_10: + av_log(s->avf, AV_LOG_TRACE, "set video data format 0x10\n"); + s->frame_format = 0x10; + + /* log position and move on for now */ + s->video_chunk_offset = avio_tell(pb); + s->video_chunk_size = opcode_size; + avio_skip(pb, opcode_size); + break; + case OPCODE_VIDEO_DATA_11: av_log(s->avf, AV_LOG_TRACE, "set video data format 0x11\n"); s->frame_format = 0x11; @@ -614,8 +649,9 @@ static int ipmovie_read_header(AVFormatContext *s) /* initialize private context members */ ipmovie->video_pts = ipmovie->audio_frame_count = 0; ipmovie->audio_chunk_offset = ipmovie->video_chunk_offset = - ipmovie->decode_map_chunk_offset = 0; - ipmovie->decode_map_chunk_size = ipmovie->video_chunk_size = 0; + ipmovie->decode_map_chunk_offset = ipmovie->skip_map_chunk_offset = 0; + ipmovie->decode_map_chunk_size = ipmovie->video_chunk_size = + ipmovie->skip_map_chunk_size = 0; ipmovie->send_buffer = ipmovie->frame_format = 0; /* on the first read, this will position the stream at the first chunk */