From patchwork Sun Jun 25 19:59:03 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hein-Pieter van Braam X-Patchwork-Id: 4113 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.1.76 with SMTP id 73csp611036vsb; Sun, 25 Jun 2017 12:59:53 -0700 (PDT) X-Received: by 10.28.21.80 with SMTP id 77mr11103790wmv.79.1498420793658; Sun, 25 Jun 2017 12:59:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1498420793; cv=none; d=google.com; s=arc-20160816; b=DUcpbQo5C3b0Y7oQcNnwW0/gw3kBcEjuix2ORacjiQpAFSmvA1zhi89YAXzf0qTtgn mQAJz9emIaZKAd85w8bYYTVg1mVFftKUoMtnWs8HnzSyXUEA5y2ZTs5OQkru3VSUiGE4 QuJJsFjtDpL2u4jW5KZ2EYi8pI9l+gvlOI2D9OpKQpPfYfojCINrF889HmMrvzbfhDKY JkYmrbhR5PPiRgINH3kimP4aovtSF/yqz7ZqyIj8pk869HITcc2sGDLLU3cHtC8sSt0S sWvI99PVhAe+ESwOjfBWjlRQB4R43jQSncV7Use/hiHa3GyOgpkvKQa9Lw90zhIFZskx /DTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:dkim-filter:delivered-to :arc-authentication-results; bh=G6Z4KE/IyP1IhVR7HRJiTRq+6Gg4CNdqg8qJySaLmSE=; b=tetjqsgPfky2iREhy3Vh2aSiZTrahQ3klmVrVgepMp8GkuAB0yB9KXTdYcDQeYvgch 9g8gl+zah0Zhwml3Q3vptWVoP4cOFXug2bcLssVogb6fWWKSPrp7CZEt6Plit0oVdPtf 7kiQr94/lhGPNDHcDLHzqTaRnhEij4ZFl5JtAvxkOGxxG9HJ210qu9vBLM8vZNw2Btsz 5snm4wP2VzfH0VOEyJ2mrFvf4sl6Ui0S+l/suh47l8EKeqdyrlsazSIOsEJOkBJs4mkK Z2A7ggzLzHteYVGnTkmg59mxkSYarjyq1pwoTr6urV/GDi2CdX7vHWEkeVZwloFUoUye /JwA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm.cx header.b=fbEgAypl; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=REJECT dis=NONE) header.from=tmm.cx Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u10si3796400wmg.100.2017.06.25.12.59.53; Sun, 25 Jun 2017 12:59:53 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm.cx header.b=fbEgAypl; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=REJECT dis=NONE) header.from=tmm.cx Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 097DA68A1F4; Sun, 25 Jun 2017 22:59:18 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from eva.tmm.cx (eva.tmm.cx [5.9.73.21]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3CE6D689DF6 for ; Sun, 25 Jun 2017 22:59:10 +0300 (EEST) Received: from lola.ttg.global (unknown [92.110.172.3]) by eva.tmm.cx (Postfix) with ESMTPSA id 46CAE1810549; Sun, 25 Jun 2017 19:59:10 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 eva.tmm.cx 46CAE1810549 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tmm.cx; s=mail; t=1498420751; bh=vvu11wJ0+s70fP+Ww/YoSI521O+bmWJgnIlq3BLwdvg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fbEgAypldIzXupBC8JMdM5S3OW6g7bViWTdp4hqWX9qfclR6uSl8jcJhDT1iBLBMk 6F/6Z8/1QYB9ZiqpYsH27NtIveSC0kQktyc75W7EST1l+oqf+Qpd7Sd2HzD/QJH1SR ak50BmyAZwUeECh3C2uiNT/eXlI28j8crYetc+xA= From: Hein-Pieter van Braam To: FFmpeg development discussions and patches Date: Sun, 25 Jun 2017 21:59:03 +0200 Message-Id: <20170625195904.1716-5-hp@tmm.cx> X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170625195904.1716-1-hp@tmm.cx> References: <20170625195904.1716-1-hp@tmm.cx> X-Spam-Status: No, score=0.0 required=2.4 tests=UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on eva.tmm.cx Subject: [FFmpeg-devel] [PATCH v3 4/5] Interplay MVE: Implement frame format 0x10 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Hein-Pieter van Braam MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This implements the 0x10 frame format for Interplay MVE movies. The format is a variation on the 0x06 format with some changes. In addition to the decoding map there's also a skip map. This skip map is used to determine what 8x8 blocks can change in a particular frame. This format expects to be able to copy an 8x8 block from before the last time it was changed. This can be an arbitrary time in the past. In order to implement this this decoder allocates two additional AVFrames where actual decoding happens. At the end of a frame decoding changed blocks are copied to a finished frame based on the skip map. The skip map's encoding is a little convulted, I'll refer to the code for details. Values in the decoding map are the same as in format 0x06. Signed-off-by: Hein-Pieter van Braam --- libavcodec/interplayvideo.c | 182 ++++++++++++++++++++++++++++++++++++++++++-- libavformat/ipmovie.c | 68 +++++++++++++---- 2 files changed, 227 insertions(+), 23 deletions(-) diff --git a/libavcodec/interplayvideo.c b/libavcodec/interplayvideo.c index 431eeb1..421de26 100644 --- a/libavcodec/interplayvideo.c +++ b/libavcodec/interplayvideo.c @@ -55,8 +55,15 @@ typedef struct IpvideoContext { HpelDSPContext hdsp; AVFrame *second_last_frame; AVFrame *last_frame; + + /* For format 0x10 */ + AVFrame *cur_decode_frame; + AVFrame *prev_decode_frame; + const unsigned char *decoding_map; int decoding_map_size; + const unsigned char *skip_map; + int skip_map_size; int is_16bpp; GetByteContext stream_ptr, mv_ptr; @@ -977,6 +984,114 @@ static void ipvideo_decode_format_06_opcodes(IpvideoContext *s, AVFrame *frame) } } +static void ipvideo_format_10_firstpass(IpvideoContext *s, AVFrame *frame, short opcode) +{ + int line; + + if (!opcode) { + for (line = 0; line < 8; ++line) { + bytestream2_get_buffer(&s->stream_ptr, s->pixel_ptr, 8); + s->pixel_ptr += s->stride; + } + } +} + +static void ipvideo_format_10_secondpass(IpvideoContext *s, AVFrame *frame, short opcode) +{ + int off_x, off_y; + + if (opcode < 0) { + off_x = ((unsigned short)opcode - 0xC000) % s->cur_decode_frame->linesize[0]; + off_y = ((unsigned short)opcode - 0xC000) / s->cur_decode_frame->linesize[0]; + copy_from(s, s->prev_decode_frame, s->cur_decode_frame, off_x, off_y); + } else if (opcode > 0) { + off_x = ((unsigned short)opcode - 0x4000) % s->cur_decode_frame->linesize[0]; + off_y = ((unsigned short)opcode - 0x4000) / s->cur_decode_frame->linesize[0]; + copy_from(s, s->cur_decode_frame, s->cur_decode_frame, off_x, off_y); + } +} + +static void (* const ipvideo_format_10_passes[])(IpvideoContext *s, AVFrame *frame, short op) = { + ipvideo_format_10_firstpass, ipvideo_format_10_secondpass, +}; + +static void ipvideo_decode_format_10_opcodes(IpvideoContext *s, AVFrame *frame) +{ + int pass, x, y, changed_block; + short opcode, skip; + GetByteContext decoding_map_ptr; + GetByteContext skip_map_ptr; + + bytestream2_skip(&s->stream_ptr, 14); /* data starts 14 bytes in */ + + /* this is PAL8, so make the palette available */ + memcpy(frame->data[1], s->pal, AVPALETTE_SIZE); + s->stride = frame->linesize[0]; + + s->line_inc = s->stride - 8; + s->upper_motion_limit_offset = (s->avctx->height - 8) * frame->linesize[0] + + (s->avctx->width - 8) * (1 + s->is_16bpp); + + bytestream2_init(&decoding_map_ptr, s->decoding_map, s->decoding_map_size); + bytestream2_init(&skip_map_ptr, s->skip_map, s->skip_map_size); + + for (pass = 0; pass < 2; ++pass) { + bytestream2_seek(&decoding_map_ptr, 0, SEEK_SET); + bytestream2_seek(&skip_map_ptr, 0, SEEK_SET); + skip = bytestream2_get_le16(&skip_map_ptr); + + for (y = 0; y < s->avctx->height; y += 8) { + for (x = 0; x < s->avctx->width; x += 8) { + s->pixel_ptr = s->cur_decode_frame->data[0] + x + y * s->cur_decode_frame->linesize[0]; + + while (skip <= 0) { + if (skip != -0x8000 && skip) { + opcode = bytestream2_get_le16(&decoding_map_ptr); + ipvideo_format_10_passes[pass](s, frame, opcode); + break; + } + skip = bytestream2_get_le16(&skip_map_ptr); + } + skip *= 2; + } + } + } + + bytestream2_seek(&skip_map_ptr, 0, SEEK_SET); + skip = bytestream2_get_le16(&skip_map_ptr); + for (y = 0; y < s->avctx->height; y += 8) { + for (x = 0; x < s->avctx->width; x += 8) { + changed_block = 0; + s->pixel_ptr = frame->data[0] + x + y*frame->linesize[0]; + + while (skip <= 0) { + if (skip != -0x8000 && skip) { + changed_block = 1; + break; + } + skip = bytestream2_get_le16(&skip_map_ptr); + } + + if (changed_block) { + copy_from(s, s->cur_decode_frame, frame, 0, 0); + } else { + /* Don't try to copy last_frame data on the first frame */ + if (s->avctx->frame_number) + copy_from(s, s->last_frame, frame, 0, 0); + } + skip *= 2; + } + } + + FFSWAP(AVFrame*, s->prev_decode_frame, s->cur_decode_frame); + + if (bytestream2_get_bytes_left(&s->stream_ptr) > 1) { + av_log(s->avctx, AV_LOG_DEBUG, + "decode finished with %d bytes left over\n", + bytestream2_get_bytes_left(&s->stream_ptr)); + } +} + static void ipvideo_decode_format_11_opcodes(IpvideoContext *s, AVFrame *frame) { int x, y; @@ -1046,12 +1161,27 @@ static av_cold int ipvideo_decode_init(AVCodecContext *avctx) s->last_frame = av_frame_alloc(); s->second_last_frame = av_frame_alloc(); - if (!s->last_frame || !s->second_last_frame) { + s->cur_decode_frame = av_frame_alloc(); + s->prev_decode_frame = av_frame_alloc(); + if (!s->last_frame || !s->second_last_frame || + !s->cur_decode_frame || !s->prev_decode_frame) { av_frame_free(&s->last_frame); av_frame_free(&s->second_last_frame); + av_frame_free(&s->cur_decode_frame); + av_frame_free(&s->prev_decode_frame); return AVERROR(ENOMEM); } + s->cur_decode_frame->width = avctx->width; + s->prev_decode_frame->width = avctx->width; + s->cur_decode_frame->height = avctx->height; + s->prev_decode_frame->height = avctx->height; + s->cur_decode_frame->format = avctx->pix_fmt; + s->prev_decode_frame->format = avctx->pix_fmt; + + ff_get_buffer(avctx, s->cur_decode_frame, 0); + ff_get_buffer(avctx, s->prev_decode_frame, 0); + return 0; } @@ -1073,13 +1203,14 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, av_frame_unref(s->second_last_frame); } - if (buf_size < 6) + if (buf_size < 8) return AVERROR_INVALIDDATA; frame_format = AV_RL8(buf); send_buffer = AV_RL8(buf + 1); video_data_size = AV_RL16(buf + 2); s->decoding_map_size = AV_RL16(buf + 4); + s->skip_map_size = AV_RL16(buf + 6); switch(frame_format) { case 0x06: @@ -1088,6 +1219,11 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, return AVERROR_INVALIDDATA; } + if (s->skip_map_size) { + av_log(avctx, AV_LOG_ERROR, "Skip map for format 0x06\n"); + return AVERROR_INVALIDDATA; + } + if (s->is_16bpp) { av_log(avctx, AV_LOG_ERROR, "Video format 0x06 does not support 16bpp movies\n"); return AVERROR_INVALIDDATA; @@ -1095,9 +1231,31 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, /* Decoding map for 0x06 frame format is at the top of pixeldata */ s->decoding_map_size = ((s->avctx->width / 8) * (s->avctx->height / 8)) * 2; - s->decoding_map = buf + 6 + 14; /* 14 bits of op data */ + s->decoding_map = buf + 8 + 14; /* 14 bits of op data */ video_data_size -= s->decoding_map_size + 14; - bytestream2_init(&s->stream_ptr, buf + 6 + s->decoding_map_size + 14, video_data_size); + bytestream2_init(&s->stream_ptr, buf + 8 + s->decoding_map_size + 14, video_data_size); + + break; + + case 0x10: + if (! s->decoding_map_size) { + av_log(avctx, AV_LOG_ERROR, "Empty decoding map for format 0x10\n"); + return AVERROR_INVALIDDATA; + } + + if (! s->skip_map_size) { + av_log(avctx, AV_LOG_ERROR, "Empty skip map for format 0x10\n"); + return AVERROR_INVALIDDATA; + } + + if (s->is_16bpp) { + av_log(avctx, AV_LOG_ERROR, "Video format 0x10 does not support 16bpp movies\n"); + return AVERROR_INVALIDDATA; + } + + bytestream2_init(&s->stream_ptr, buf + 8, video_data_size); + s->decoding_map = buf + 8 + video_data_size; + s->skip_map = buf + 8 + video_data_size + s->decoding_map_size; break; @@ -1107,8 +1265,13 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, return AVERROR_INVALIDDATA; } - bytestream2_init(&s->stream_ptr, buf + 6, video_data_size); - s->decoding_map = buf + 6 + video_data_size; + if (s->skip_map_size) { + av_log(avctx, AV_LOG_ERROR, "Skip map for format 0x11\n"); + return AVERROR_INVALIDDATA; + } + + bytestream2_init(&s->stream_ptr, buf + 8, video_data_size); + s->decoding_map = buf + 8 + video_data_size; break; @@ -1117,7 +1280,7 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, } /* ensure we can't overread the packet */ - if (buf_size < 6 + s->decoding_map_size + video_data_size) { + if (buf_size < 8 + s->decoding_map_size + video_data_size + s->skip_map_size) { av_log(avctx, AV_LOG_ERROR, "Invalid IP packet size\n"); return AVERROR_INVALIDDATA; } @@ -1140,6 +1303,9 @@ static int ipvideo_decode_frame(AVCodecContext *avctx, case 0x06: ipvideo_decode_format_06_opcodes(s, frame); break; + case 0x10: + ipvideo_decode_format_10_opcodes(s, frame); + break; case 0x11: ipvideo_decode_format_11_opcodes(s, frame); break; @@ -1163,6 +1329,8 @@ static av_cold int ipvideo_decode_end(AVCodecContext *avctx) av_frame_free(&s->last_frame); av_frame_free(&s->second_last_frame); + av_frame_free(&s->cur_decode_frame); + av_frame_free(&s->prev_decode_frame); return 0; } diff --git a/libavformat/ipmovie.c b/libavformat/ipmovie.c index 7f2454b..7f5a8c6 100644 --- a/libavformat/ipmovie.c +++ b/libavformat/ipmovie.c @@ -66,9 +66,9 @@ #define OPCODE_CREATE_GRADIENT 0x0B #define OPCODE_SET_PALETTE 0x0C #define OPCODE_SET_PALETTE_COMPRESSED 0x0D -#define OPCODE_UNKNOWN_0E 0x0E +#define OPCODE_SET_SKIP_MAP 0x0E #define OPCODE_SET_DECODING_MAP 0x0F -#define OPCODE_UNKNOWN_10 0x10 +#define OPCODE_VIDEO_DATA_10 0x10 #define OPCODE_VIDEO_DATA_11 0x11 #define OPCODE_UNKNOWN_12 0x12 #define OPCODE_UNKNOWN_13 0x13 @@ -107,6 +107,8 @@ typedef struct IPMVEContext { int audio_chunk_size; int64_t video_chunk_offset; int video_chunk_size; + int64_t skip_map_chunk_offset; + int skip_map_chunk_size; int64_t decode_map_chunk_offset; int decode_map_chunk_size; @@ -156,9 +158,9 @@ static int load_ipmovie_packet(IPMVEContext *s, AVIOContext *pb, } else if (s->frame_format) { - /* send the frame format, decode map, the video data, and the send_buffer flag together */ + /* send the frame format, decode map, the video data, skip map, and the send_buffer flag together */ - if (av_new_packet(pkt, 6 + s->decode_map_chunk_size + s->video_chunk_size)) + if (av_new_packet(pkt, 8 + s->decode_map_chunk_size + s->video_chunk_size + s->skip_map_chunk_size)) return CHUNK_NOMEM; if (s->has_palette) { @@ -181,6 +183,7 @@ static int load_ipmovie_packet(IPMVEContext *s, AVIOContext *pb, AV_WL8(pkt->data + 1, s->send_buffer); AV_WL16(pkt->data + 2, s->video_chunk_size); AV_WL16(pkt->data + 4, s->decode_map_chunk_size); + AV_WL16(pkt->data + 6, s->skip_map_chunk_size); s->frame_format = 0; s->send_buffer = 0; @@ -189,24 +192,39 @@ static int load_ipmovie_packet(IPMVEContext *s, AVIOContext *pb, avio_seek(pb, s->video_chunk_offset, SEEK_SET); s->video_chunk_offset = 0; - if (avio_read(pb, pkt->data + 6, s->video_chunk_size) != + if (avio_read(pb, pkt->data + 8, s->video_chunk_size) != s->video_chunk_size) { av_packet_unref(pkt); return CHUNK_EOF; } - pkt->pos = s->decode_map_chunk_offset; - avio_seek(pb, s->decode_map_chunk_offset, SEEK_SET); - s->decode_map_chunk_offset = 0; + if (s->decode_map_chunk_size) { + pkt->pos = s->decode_map_chunk_offset; + avio_seek(pb, s->decode_map_chunk_offset, SEEK_SET); + s->decode_map_chunk_offset = 0; - if (avio_read(pb, pkt->data + 6 + s->video_chunk_size, - s->decode_map_chunk_size) != s->decode_map_chunk_size) { - av_packet_unref(pkt); - return CHUNK_EOF; + if (avio_read(pb, pkt->data + 8 + s->video_chunk_size, + s->decode_map_chunk_size) != s->decode_map_chunk_size) { + av_packet_unref(pkt); + return CHUNK_EOF; + } + } + + if (s->skip_map_chunk_size) { + pkt->pos = s->skip_map_chunk_offset; + avio_seek(pb, s->skip_map_chunk_offset, SEEK_SET); + s->skip_map_chunk_offset = 0; + + if (avio_read(pb, pkt->data + 8 + s->video_chunk_size + s->decode_map_chunk_size, + s->skip_map_chunk_size) != s->skip_map_chunk_size) { + av_packet_unref(pkt); + return CHUNK_EOF; + } } s->video_chunk_size = 0; s->decode_map_chunk_size = 0; + s->skip_map_chunk_size = 0; pkt->stream_index = s->video_stream_index; pkt->pts = s->video_pts; @@ -444,8 +462,6 @@ static int process_ipmovie_chunk(IPMVEContext *s, AVIOContext *pb, s->video_width, s->video_height); break; - case OPCODE_UNKNOWN_0E: - case OPCODE_UNKNOWN_10: case OPCODE_UNKNOWN_12: case OPCODE_UNKNOWN_13: case OPCODE_UNKNOWN_14: @@ -527,6 +543,15 @@ static int process_ipmovie_chunk(IPMVEContext *s, AVIOContext *pb, avio_skip(pb, opcode_size); break; + case OPCODE_SET_SKIP_MAP: + av_log(s->avf, AV_LOG_TRACE, "set skip map\n"); + + /* log position and move on for now */ + s->skip_map_chunk_offset = avio_tell(pb); + s->skip_map_chunk_size = opcode_size; + avio_skip(pb, opcode_size); + break; + case OPCODE_SET_DECODING_MAP: av_log(s->avf, AV_LOG_TRACE, "set decoding map\n"); @@ -546,6 +571,16 @@ static int process_ipmovie_chunk(IPMVEContext *s, AVIOContext *pb, avio_skip(pb, opcode_size); break; + case OPCODE_VIDEO_DATA_10: + av_log(s->avf, AV_LOG_TRACE, "set video data format 0x10\n"); + s->frame_format = 0x10; + + /* log position and move on for now */ + s->video_chunk_offset = avio_tell(pb); + s->video_chunk_size = opcode_size; + avio_skip(pb, opcode_size); + break; + case OPCODE_VIDEO_DATA_11: av_log(s->avf, AV_LOG_TRACE, "set video data format 0x11\n"); s->frame_format = 0x11; @@ -614,8 +649,9 @@ static int ipmovie_read_header(AVFormatContext *s) /* initialize private context members */ ipmovie->video_pts = ipmovie->audio_frame_count = 0; ipmovie->audio_chunk_offset = ipmovie->video_chunk_offset = - ipmovie->decode_map_chunk_offset = 0; - ipmovie->decode_map_chunk_size = ipmovie->video_chunk_size = 0; + ipmovie->decode_map_chunk_offset = ipmovie->skip_map_chunk_offset = 0; + ipmovie->decode_map_chunk_size = ipmovie->video_chunk_size = + ipmovie->skip_map_chunk_size = 0; ipmovie->send_buffer = ipmovie->frame_format = 0; /* on the first read, this will position the stream at the first chunk */