From patchwork Wed Jul 8 05:28:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zlomek, Josef" X-Patchwork-Id: 20892 Delivered-To: andriy.gelman@gmail.com Received: by 2002:a25:80ca:0:0:0:0:0 with SMTP id c10csp272472ybm; Tue, 7 Jul 2020 22:28:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyxi6KSfUm4Mqazo8T34iBFgYw+y4B6nbqcwNz2wut8v9D8g/mxKgm+ax6g7mx9XgeUEUFA X-Received: by 2002:a1c:4143:: with SMTP id o64mr7673152wma.28.1594186125058; Tue, 07 Jul 2020 22:28:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594186125; cv=none; d=google.com; s=arc-20160816; b=nE+NpjUiHtcSxjl4eGnxMWunSrjtuouC1UFQkFZwZU1i4Xt5MbNnvs7JjYx5N38i/E 4HHnSUMAdQynNnh5OmFIIzLaUAqggWZdlOuQ8+f7Fi+lsFAJJBJfleQlrKlUHa3uTDYI EcSnuctTOcFnL3nwCo1f8e3islgsditL7IFMip4V1tX80NC4oLFpwpDOmz48beSuxkY2 1+P7c79EYRCaBiAiIecLsRku8Ca93Y7sFTcWLgvFP3YJrktWvy6Hf8EC+E+omn+DZIfp 5Ma8/ws+JSGuYuF+yAfc1yMBm+XKAZgDqoaVUuNGg9yO1nSA1SsgWXBC4hC4NjicNyIo wNSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:delivered-to; bh=Z6a6ZTa8UnE2IndRatXoecYZ/vp5Fo5yQgGsiNgx4mc=; b=WP+fxZHBli9qMmZUQD0uecMo/wCsWuDIYGV+//QzqHIFwoYWsElkkDjjm+MfWIiYY6 2fT3WAUdqYmIlE2Hwy96yFFX3l6PJjzOHgqanyjX+z21KHmasISGA5at6XHraqgmBbkF Ev/E0pSY+UKC7p8fxeI3Gmi6Ul4aaZVP1C3cQHGdiR34AE/OcDL3Ye3SMmgSqDtMnN83 mFF/VegEn829kni1gyg/eQjoB8JQGgwvYTHZIEVF8KxaYzbpep5wmu2lIOT++ilyT3l4 7X5/Q0XQ2ldMC/wbITfgEneHdHKR2TudHh5rGSsu11v34Pkv3mh/aJjPgnDhkkgeBNpY l86Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=pex.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e1si12936052wrv.448.2020.07.07.22.28.44; Tue, 07 Jul 2020 22:28:45 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=pex.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9593D687F75; Wed, 8 Jul 2020 08:28:42 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vps.zlomek.net (vps.zlomek.net [195.181.211.90]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3E1C1687F75 for ; Wed, 8 Jul 2020 08:28:36 +0300 (EEST) Received: by vps.zlomek.net (Postfix, from userid 1000) id 96CAC603DB; Wed, 8 Jul 2020 07:28:35 +0200 (CEST) From: Josef Zlomek To: ffmpeg-devel@ffmpeg.org Date: Wed, 8 Jul 2020 07:28:23 +0200 Message-Id: <20200708052824.18582-1-josef@pex.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH 1/2] libavcodec: add support for animated WebP decoding X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Josef Zlomek MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: IyJwMBvucnxV Content-Length: 49483 Fixes: 4907 Adds support for decoding of animated WebP. The WebP parser now splits the input stream into packets containing one frame. The WebP decoder adds the animation related features according to the specs: https://developers.google.com/speed/webp/docs/riff_container#animation The frames of the animation may be smaller than the image canvas. Therefore, the frame is decoded to a temporary frame, then it is blended into the canvas, the canvas is copied to the output frame, and finally the frame is disposed from the canvas. The output to AV_PIX_FMT_YUVA420P/AV_PIX_FMT_YUV420P is still supported. The background color is specified only as BGRA in the WebP file so it is converted to YUVA if YUV formats are output. Signed-off-by: Josef Zlomek --- Changelog | 1 + libavcodec/version.h | 2 +- libavcodec/webp.c | 548 ++++++++++++++++++++++++++++++++++++--- libavcodec/webp_parser.c | 116 ++++++--- 4 files changed, 583 insertions(+), 84 deletions(-) diff --git a/Changelog b/Changelog index 1bb9931c0d..1e41040a8e 100644 --- a/Changelog +++ b/Changelog @@ -5,6 +5,7 @@ version : - AudioToolbox output device - MacCaption demuxer - PGX decoder +- animated WebP parser/decoder version 4.3: diff --git a/libavcodec/version.h b/libavcodec/version.h index 482cc6d6ba..e75891d463 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,7 +28,7 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 58 -#define LIBAVCODEC_VERSION_MINOR 94 +#define LIBAVCODEC_VERSION_MINOR 95 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ diff --git a/libavcodec/webp.c b/libavcodec/webp.c index c6d0206846..e0eb076cd1 100644 --- a/libavcodec/webp.c +++ b/libavcodec/webp.c @@ -41,6 +41,7 @@ */ #include "libavutil/imgutils.h" +#include "libswscale/swscale.h" #define BITSTREAM_READER_LE #include "avcodec.h" @@ -57,6 +58,9 @@ #define VP8X_FLAG_ALPHA 0x10 #define VP8X_FLAG_ICC 0x20 +#define ANMF_DISPOSAL_METHOD 0x01 +#define ANMF_BLENDING_METHOD 0x02 + #define MAX_PALETTE_SIZE 256 #define MAX_CACHE_BITS 11 #define NUM_CODE_LENGTH_CODES 19 @@ -188,19 +192,30 @@ typedef struct ImageContext { typedef struct WebPContext { VP8Context v; /* VP8 Context used for lossy decoding */ GetBitContext gb; /* bitstream reader for main image chunk */ + AVFrame *canvas_frame; /* AVFrame for canvas */ + AVFrame *frame; /* AVFrame for decoded frame */ AVFrame *alpha_frame; /* AVFrame for alpha data decompressed from VP8L */ AVCodecContext *avctx; /* parent AVCodecContext */ int initialized; /* set once the VP8 context is initialized */ int has_alpha; /* has a separate alpha chunk */ enum AlphaCompression alpha_compression; /* compression type for alpha chunk */ enum AlphaFilter alpha_filter; /* filtering method for alpha chunk */ + AVPacket alpha_packet; /* alpha chunk */ uint8_t *alpha_data; /* alpha chunk data */ int alpha_data_size; /* alpha chunk data size */ int has_exif; /* set after an EXIF chunk has been processed */ int has_iccp; /* set after an ICCP chunk has been processed */ - int width; /* image width */ - int height; /* image height */ + int vp8x_flags; /* flags from VP8X chunk */ + int anmf_flags; /* flags from ANMF chunk */ + int canvas_width; /* canvas width */ + int canvas_height; /* canvas height */ + int width; /* frame width */ + int height; /* frame height */ + int pos_x; /* frame position X */ + int pos_y; /* frame position Y */ int lossless; /* indicates lossless or lossy */ + uint32_t background_argb; /* background color in ARGB format */ + uint8_t background_yuva[4]; /* background color in YUVA format */ int nb_transforms; /* number of transforms */ enum TransformType transforms[4]; /* transformations used in the image, in order */ @@ -1100,7 +1115,7 @@ static int apply_color_indexing_transform(WebPContext *s) return 0; } -static void update_canvas_size(AVCodecContext *avctx, int w, int h) +static void update_frame_size(AVCodecContext *avctx, int w, int h) { WebPContext *s = avctx->priv_data; if (s->width && s->width != w) { @@ -1140,7 +1155,7 @@ static int vp8_lossless_decode_frame(AVCodecContext *avctx, AVFrame *p, w = get_bits(&s->gb, 14) + 1; h = get_bits(&s->gb, 14) + 1; - update_canvas_size(avctx, w, h); + update_frame_size(avctx, w, h); ret = ff_set_dimensions(avctx, s->width, s->height); if (ret < 0) @@ -1356,7 +1371,7 @@ static int vp8_lossy_decode_frame(AVCodecContext *avctx, AVFrame *p, if (!*got_frame) return AVERROR_INVALIDDATA; - update_canvas_size(avctx, avctx->width, avctx->height); + update_frame_size(avctx, avctx->width, avctx->height); if (s->has_alpha) { ret = vp8_lossy_decode_alpha(avctx, p, s->alpha_data, @@ -1367,6 +1382,392 @@ static int vp8_lossy_decode_frame(AVCodecContext *avctx, AVFrame *p, return ret; } +static int convert_background_color(AVCodecContext *avctx) +{ + WebPContext *s = avctx->priv_data; + int i; + int ret; + + uint32_t src_frame[2][2] = { {s->background_argb, s->background_argb}, + {s->background_argb, s->background_argb} }; + uint8_t dst_frames[4][2][2]; + + const uint8_t * const src_data[4] = { (const uint8_t *) &src_frame[0][0], + NULL, + NULL, + NULL }; + uint8_t * const dst_data[4] = { &dst_frames[0][0][0], + &dst_frames[1][0][0], + &dst_frames[2][0][0], + &dst_frames[3][0][0] }; + int src_linesize[4] = { 4, 0, 0, 0 }; + int dst_linesize[4] = { 2, 1, 1, 2 }; + + struct SwsContext *ctx = sws_getContext(2, 2, AV_PIX_FMT_ARGB, + 2, 2, AV_PIX_FMT_YUVA420P, + 0, 0, 0, 0); + if (!ctx) + return AVERROR(EINVAL); + + ret = sws_scale(ctx, src_data, src_linesize, 0, 2, dst_data, dst_linesize); + + if (ret >= 0) { + for (i = 0; i < 4; i++) + s->background_yuva[i] = dst_frames[i][0][0]; + } + + sws_freeContext(ctx); + return ret; +} + +static int init_canvas_frame(AVCodecContext *avctx) +{ + WebPContext *s = avctx->priv_data; + AVFrame *canvas; + AVFrame *frame = s->frame; + int x, y; + int width, height; + int ret; + + canvas = av_frame_alloc(); + if (!canvas) + return AVERROR(ENOMEM); + s->canvas_frame = canvas; + + /* let canvas always have alpha */ + canvas->format = frame->format == AV_PIX_FMT_YUV420P ? AV_PIX_FMT_YUVA420P : frame->format; + canvas->width = s->canvas_width; + canvas->height = s->canvas_height; + + ret = av_frame_copy_props(canvas, frame); + if (ret < 0) + return ret; + + ret = av_frame_get_buffer(canvas, 1); + if (ret < 0) + return ret; + + if (canvas->format == AV_PIX_FMT_ARGB) { + width = canvas->width; + height = canvas->height; + + for (y = 0; y < height; y++) { + uint32_t *dst = (uint32_t *) (canvas->data[0] + y * canvas->linesize[0]); + for (x = 0; x < width; x++) + dst[x] = s->background_argb; + } + } else /* if (canvas->format == AV_PIX_FMT_YUVA420P) */ { + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(canvas->format); + int component; + int plane; + + ret = convert_background_color(avctx); + if (ret < 0) + return ret; + + for (component = 0; component < desc->nb_components; component++) { + plane = desc->comp[component].plane; + height = canvas->height; + + if (component == 1 || component == 2) { + height = AV_CEIL_RSHIFT(height, desc->log2_chroma_h); + } + + memset(canvas->data[plane], s->background_yuva[component], + height * canvas->linesize[plane]); + } + } + + return 0; +} + +static int blend_frame_into_canvas(AVCodecContext *avctx) +{ + WebPContext *s = avctx->priv_data; + AVFrame *canvas = s->canvas_frame; + AVFrame *frame = s->frame; + int x, y; + int width, height; + int pos_x, pos_y; + + if ((s->anmf_flags & ANMF_BLENDING_METHOD) || frame->format == AV_PIX_FMT_YUV420P) { + /* do not blend, overwrite */ + + if (canvas->format == AV_PIX_FMT_ARGB) { + width = s->width; + height = s->height; + pos_x = s->pos_x; + pos_y = s->pos_y; + + for (y = 0; y < height; y++) { + const uint32_t *src = (uint32_t *) (frame->data[0] + y * frame->linesize[0]); + uint32_t *dst = (uint32_t *) (canvas->data[0] + (y + pos_y) * canvas->linesize[0]) + pos_x; + memcpy(dst, src, width * 4); + } + } else /* if (canvas->format == AV_PIX_FMT_YUVA420P) */ { + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(frame->format); + int component; + int plane; + + for (component = 0; component < desc->nb_components; component++) { + plane = desc->comp[component].plane; + width = s->width; + height = s->height; + pos_x = s->pos_x; + pos_y = s->pos_y; + if (component == 1 || component == 2) { + width = AV_CEIL_RSHIFT(width, desc->log2_chroma_w); + height = AV_CEIL_RSHIFT(height, desc->log2_chroma_h); + pos_x = AV_CEIL_RSHIFT(pos_x, desc->log2_chroma_w); + pos_y = AV_CEIL_RSHIFT(pos_y, desc->log2_chroma_h); + } + + for (y = 0; y < height; y++) { + const uint8_t *src = frame->data[plane] + y * frame->linesize[plane]; + uint8_t *dst = canvas->data[plane] + (y + pos_y) * canvas->linesize[plane] + pos_x; + memcpy(dst, src, width); + } + } + + if (desc->nb_components < 4) { + /* frame does not have alpha, set alpha to 255 */ + desc = av_pix_fmt_desc_get(canvas->format); + plane = desc->comp[3].plane; + width = s->width; + height = s->height; + pos_x = s->pos_x; + pos_y = s->pos_y; + + for (y = 0; y < height; y++) { + uint8_t *dst = canvas->data[plane] + (y + pos_y) * canvas->linesize[plane] + pos_x; + memset(dst, 255, width); + } + } + } + } else { + /* alpha blending */ + + if (canvas->format == AV_PIX_FMT_ARGB) { + width = s->width; + height = s->height; + pos_x = s->pos_x; + pos_y = s->pos_y; + + for (y = 0; y < height; y++) { + const uint8_t *src = frame->data[0] + y * frame->linesize[0]; + uint8_t *dst = canvas->data[0] + (y + pos_y) * canvas->linesize[0] + pos_x * 4; + for (x = 0; x < width; x++) { + int src_alpha = src[0]; + int dst_alpha = dst[0]; + int dst_alpha2 = dst_alpha - ROUNDED_DIV(src_alpha * dst_alpha, 255); + int blend_alpha = src_alpha + dst_alpha2; + + if (blend_alpha == 0) { + dst[0] = 0; + dst[1] = 0; + dst[2] = 0; + dst[3] = 0; + } else { + dst[0] = blend_alpha; + dst[1] = ROUNDED_DIV(src[1] * src_alpha + dst[1] * dst_alpha2, blend_alpha); + dst[2] = ROUNDED_DIV(src[2] * src_alpha + dst[2] * dst_alpha2, blend_alpha); + dst[3] = ROUNDED_DIV(src[3] * src_alpha + dst[3] * dst_alpha2, blend_alpha); + } + src += 4; + dst += 4; + } + } + } else /* if (canvas->format == AV_PIX_FMT_YUVA420P) */ { + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(frame->format); + int plane_y, plane_u, plane_v, plane_a; + int tile_w; + int tile_h; + int src_alpha; + int dst_alpha; + int dst_alpha2; + int blend_alpha; + + av_assert0(desc->nb_components >= 4); + plane_y = desc->comp[0].plane; + plane_u = desc->comp[1].plane; + plane_v = desc->comp[2].plane; + plane_a = desc->comp[3].plane; + + // first, blend U & V planes, because the later step modifies alpha plane + width = AV_CEIL_RSHIFT(s->width, desc->log2_chroma_w); + height = AV_CEIL_RSHIFT(s->height, desc->log2_chroma_h); + pos_x = AV_CEIL_RSHIFT(s->pos_x, desc->log2_chroma_w); + pos_y = AV_CEIL_RSHIFT(s->pos_y, desc->log2_chroma_h); + tile_w = 1 << desc->log2_chroma_w; + tile_h = 1 << desc->log2_chroma_h; + + for (y = 0; y < height; y++) { + uint8_t *src_u = frame->data[plane_u] + y * frame->linesize[plane_u]; + uint8_t *src_v = frame->data[plane_v] + y * frame->linesize[plane_v]; + uint8_t *dst_u = canvas->data[plane_u] + (y + pos_y) * canvas->linesize[plane_u] + pos_x; + uint8_t *dst_v = canvas->data[plane_v] + (y + pos_y) * canvas->linesize[plane_v] + pos_x; + for (x = 0; x < width; x++) { + int xx, yy; + + src_alpha = 0; + dst_alpha = 0; + for (yy = 0; yy < tile_h; yy++) { + for (xx = 0; xx < tile_w; xx++) { + src_alpha += frame->data[plane_a][(y * tile_h + yy) * frame->linesize[plane_a] + (x * tile_w + xx)]; + dst_alpha += canvas->data[plane_a][((y + pos_y) * tile_h + yy) * canvas->linesize[plane_a] + ((x + pos_x) * tile_w + xx)]; + } + } + src_alpha = RSHIFT(src_alpha, desc->log2_chroma_w + desc->log2_chroma_h); + dst_alpha = RSHIFT(dst_alpha, desc->log2_chroma_w + desc->log2_chroma_h); + dst_alpha2 = dst_alpha - ROUNDED_DIV(src_alpha * dst_alpha, 255); + blend_alpha = src_alpha + dst_alpha2; + + if (blend_alpha == 0) { + *dst_u = 0; + *dst_v = 0; + } else { + *dst_u = ROUNDED_DIV(*src_u * src_alpha + *dst_u * dst_alpha2, blend_alpha); + *dst_v = ROUNDED_DIV(*src_v * src_alpha + *dst_v * dst_alpha2, blend_alpha); + } + + src_u++; + src_v++; + dst_u++; + dst_v++; + } + } + + // then blend Y & A planes + width = s->width; + height = s->height; + pos_x = s->pos_x; + pos_y = s->pos_y; + + for (y = 0; y < height; y++) { + const uint8_t *src_y = frame->data[plane_y] + y * frame->linesize[plane_y]; + const uint8_t *src_a = frame->data[plane_a] + y * frame->linesize[plane_a]; + uint8_t *dst_y = canvas->data[plane_y] + (y + pos_y) * canvas->linesize[plane_y] + pos_x; + uint8_t *dst_a = canvas->data[plane_a] + (y + pos_y) * canvas->linesize[plane_a] + pos_x; + for (x = 0; x < width; x++) { + src_alpha = *src_a; + dst_alpha = *dst_a; + dst_alpha2 = dst_alpha - ROUNDED_DIV(src_alpha * dst_alpha, 255); + blend_alpha = src_alpha + dst_alpha2; + + if (blend_alpha == 0) { + *dst_y = 0; + *dst_a = 0; + } else { + *dst_y = ROUNDED_DIV(*src_y * src_alpha + *dst_y * dst_alpha2, blend_alpha); + *dst_a = blend_alpha; + } + + src_y++; + src_a++; + dst_y++; + dst_a++; + } + } + } + } + + return 0; +} + +static int copy_canvas_to_frame(AVCodecContext *avctx, AVFrame *frame) +{ + WebPContext *s = avctx->priv_data; + AVFrame *canvas = s->canvas_frame; + int ret; + + avctx->pix_fmt = canvas->format; + frame->format = canvas->format; + frame->width = canvas->width; + frame->height = canvas->height; + + ret = av_frame_get_buffer(frame, 1); + if (ret < 0) + return ret; + + ret = av_frame_copy_props(frame, canvas); + if (ret < 0) + return ret; + + ret = av_frame_copy(frame, canvas); + if (ret < 0) + return ret; + + /* VP8 decoder changed the width and height in AVCodecContext. + * Change it back to the canvas size. */ + ret = ff_set_dimensions(avctx, canvas->width, canvas->height); + if (ret < 0) + return ret; + + return 0; +} + +static int dispose_frame_in_canvas(AVCodecContext *avctx) +{ + WebPContext *s = avctx->priv_data; + AVFrame *canvas = s->canvas_frame; + int x, y; + int width, height; + int pos_x, pos_y; + + if (s->anmf_flags & ANMF_DISPOSAL_METHOD) { + /* dispose to background color */ + + if (canvas->format == AV_PIX_FMT_ARGB) { + width = s->width; + height = s->height; + pos_x = s->pos_x; + pos_y = s->pos_y; + + for (y = 0; y < height; y++) { + uint32_t *dst = (uint32_t *) (canvas->data[0] + (y + pos_y) * canvas->linesize[0]) + pos_x; + for (x = 0; x < width; x++) + dst[x] = s->background_argb; + } + } else /* if (canvas->format == AV_PIX_FMT_YUVA420P) */ { + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(canvas->format); + int component; + int plane; + + for (component = 0; component < desc->nb_components; component++) { + plane = desc->comp[component].plane; + width = s->width; + height = s->height; + pos_x = s->pos_x; + pos_y = s->pos_y; + if (component == 1 || component == 2) { + width = AV_CEIL_RSHIFT(width, desc->log2_chroma_w); + height = AV_CEIL_RSHIFT(height, desc->log2_chroma_h); + pos_x = AV_CEIL_RSHIFT(pos_x, desc->log2_chroma_w); + pos_y = AV_CEIL_RSHIFT(pos_y, desc->log2_chroma_h); + } + + for (y = 0; y < height; y++) { + uint8_t *dst = canvas->data[plane] + (y + pos_y) * canvas->linesize[plane] + pos_x; + memset(dst, s->background_yuva[component], width); + } + } + } + } + + return 0; +} + +static av_cold int webp_decode_init(AVCodecContext *avctx) +{ + WebPContext *s = avctx->priv_data; + + s->frame = av_frame_alloc(); + if (!s->frame) + return AVERROR(ENOMEM); + + return 0; +} + static int webp_decode_frame(AVCodecContext *avctx, void *data, int *got_frame, AVPacket *avpkt) { @@ -1375,33 +1776,13 @@ static int webp_decode_frame(AVCodecContext *avctx, void *data, int *got_frame, GetByteContext gb; int ret; uint32_t chunk_type, chunk_size; - int vp8x_flags = 0; s->avctx = avctx; - s->width = 0; - s->height = 0; *got_frame = 0; - s->has_alpha = 0; - s->has_exif = 0; - s->has_iccp = 0; bytestream2_init(&gb, avpkt->data, avpkt->size); - if (bytestream2_get_bytes_left(&gb) < 12) - return AVERROR_INVALIDDATA; - - if (bytestream2_get_le32(&gb) != MKTAG('R', 'I', 'F', 'F')) { - av_log(avctx, AV_LOG_ERROR, "missing RIFF tag\n"); - return AVERROR_INVALIDDATA; - } - - chunk_size = bytestream2_get_le32(&gb); - if (bytestream2_get_bytes_left(&gb) < chunk_size) - return AVERROR_INVALIDDATA; - - if (bytestream2_get_le32(&gb) != MKTAG('W', 'E', 'B', 'P')) { - av_log(avctx, AV_LOG_ERROR, "missing WEBP tag\n"); - return AVERROR_INVALIDDATA; - } + // clear the previous frame + av_frame_unref(s->frame); while (bytestream2_get_bytes_left(&gb) > 8) { char chunk_str[5] = { 0 }; @@ -1412,6 +1793,10 @@ static int webp_decode_frame(AVCodecContext *avctx, void *data, int *got_frame, return AVERROR_INVALIDDATA; chunk_size += chunk_size & 1; + /* we need to dive into RIFF chunk */ + if (chunk_type == MKTAG('R', 'I', 'F', 'F')) + chunk_size = 4; + if (bytestream2_get_bytes_left(&gb) < chunk_size) { /* we seem to be running out of data, but it could also be that the bitstream has trailing junk leading to bogus chunk_size. */ @@ -1419,9 +1804,27 @@ static int webp_decode_frame(AVCodecContext *avctx, void *data, int *got_frame, } switch (chunk_type) { + case MKTAG('R', 'I', 'F', 'F'): + if (bytestream2_get_le32(&gb) != MKTAG('W', 'E', 'B', 'P')) { + av_log(avctx, AV_LOG_ERROR, "missing WEBP tag\n"); + return AVERROR_INVALIDDATA; + } + s->vp8x_flags = 0; + s->anmf_flags = 0; + s->canvas_width = 0; + s->canvas_height = 0; + s->width = 0; + s->height = 0; + s->pos_x = 0; + s->pos_y = 0; + s->has_alpha = 0; + s->has_exif = 0; + s->has_iccp = 0; + av_packet_unref(&s->alpha_packet); + break; case MKTAG('V', 'P', '8', ' '): if (!*got_frame) { - ret = vp8_lossy_decode_frame(avctx, p, got_frame, + ret = vp8_lossy_decode_frame(avctx, s->frame, got_frame, avpkt->data + bytestream2_tell(&gb), chunk_size); if (ret < 0) @@ -1431,7 +1834,7 @@ static int webp_decode_frame(AVCodecContext *avctx, void *data, int *got_frame, break; case MKTAG('V', 'P', '8', 'L'): if (!*got_frame) { - ret = vp8_lossless_decode_frame(avctx, p, got_frame, + ret = vp8_lossless_decode_frame(avctx, s->frame, got_frame, avpkt->data + bytestream2_tell(&gb), chunk_size, 0); if (ret < 0) @@ -1441,14 +1844,16 @@ static int webp_decode_frame(AVCodecContext *avctx, void *data, int *got_frame, bytestream2_skip(&gb, chunk_size); break; case MKTAG('V', 'P', '8', 'X'): - if (s->width || s->height || *got_frame) { + if (s->canvas_width || s->canvas_height || *got_frame) { av_log(avctx, AV_LOG_ERROR, "Canvas dimensions are already set\n"); return AVERROR_INVALIDDATA; } - vp8x_flags = bytestream2_get_byte(&gb); + s->vp8x_flags = bytestream2_get_byte(&gb); bytestream2_skip(&gb, 3); s->width = bytestream2_get_le24(&gb) + 1; s->height = bytestream2_get_le24(&gb) + 1; + s->canvas_width = s->width; + s->canvas_height = s->height; ret = av_image_check_size(s->width, s->height, 0, avctx); if (ret < 0) return ret; @@ -1456,7 +1861,7 @@ static int webp_decode_frame(AVCodecContext *avctx, void *data, int *got_frame, case MKTAG('A', 'L', 'P', 'H'): { int alpha_header, filter_m, compression; - if (!(vp8x_flags & VP8X_FLAG_ALPHA)) { + if (!(s->vp8x_flags & VP8X_FLAG_ALPHA)) { av_log(avctx, AV_LOG_WARNING, "ALPHA chunk present, but alpha bit not set in the " "VP8X header\n"); @@ -1465,8 +1870,12 @@ static int webp_decode_frame(AVCodecContext *avctx, void *data, int *got_frame, av_log(avctx, AV_LOG_ERROR, "invalid ALPHA chunk size\n"); return AVERROR_INVALIDDATA; } + av_packet_unref(&s->alpha_packet); + ret = av_packet_ref(&s->alpha_packet, avpkt); + if (ret < 0) + return ret; alpha_header = bytestream2_get_byte(&gb); - s->alpha_data = avpkt->data + bytestream2_tell(&gb); + s->alpha_data = s->alpha_packet.data + bytestream2_tell(&gb); s->alpha_data_size = chunk_size - 1; bytestream2_skip(&gb, s->alpha_data_size); @@ -1493,7 +1902,7 @@ static int webp_decode_frame(AVCodecContext *avctx, void *data, int *got_frame, av_log(avctx, AV_LOG_VERBOSE, "Ignoring extra EXIF chunk\n"); goto exif_end; } - if (!(vp8x_flags & VP8X_FLAG_EXIF_METADATA)) + if (!(s->vp8x_flags & VP8X_FLAG_EXIF_METADATA)) av_log(avctx, AV_LOG_WARNING, "EXIF chunk present, but Exif bit not set in the " "VP8X header\n"); @@ -1528,13 +1937,13 @@ exif_end: bytestream2_skip(&gb, chunk_size); break; } - if (!(vp8x_flags & VP8X_FLAG_ICC)) + if (!(s->vp8x_flags & VP8X_FLAG_ICC)) av_log(avctx, AV_LOG_WARNING, "ICCP chunk present, but ICC Profile bit not set in the " "VP8X header\n"); s->has_iccp = 1; - sd = av_frame_new_side_data(p, AV_FRAME_DATA_ICC_PROFILE, chunk_size); + sd = av_frame_new_side_data(s->frame, AV_FRAME_DATA_ICC_PROFILE, chunk_size); if (!sd) return AVERROR(ENOMEM); @@ -1542,7 +1951,39 @@ exif_end: break; } case MKTAG('A', 'N', 'I', 'M'): + if (!(s->vp8x_flags & VP8X_FLAG_ANIMATION)) { + av_log(avctx, AV_LOG_WARNING, + "ANIM chunk present, but animation bit not set in the " + "VP8X header\n"); + } + /* background is stored as BGRA, we need ARGB in native endian */ + s->background_argb = av_bswap32(bytestream2_get_ne32u(&gb)); + bytestream2_skip(&gb, 2); /* loop count is ignored */ + break; case MKTAG('A', 'N', 'M', 'F'): + av_packet_unref(&s->alpha_packet); + s->has_alpha = 0; + + if (!(s->vp8x_flags & VP8X_FLAG_ANIMATION)) { + av_log(avctx, AV_LOG_WARNING, + "ANMF chunk present, but animation bit not set in the " + "VP8X header\n"); + s->vp8x_flags |= VP8X_FLAG_ANIMATION; + } + s->pos_x = bytestream2_get_le24(&gb) * 2; + s->pos_y = bytestream2_get_le24(&gb) * 2; + s->width = bytestream2_get_le24(&gb) + 1; + s->height = bytestream2_get_le24(&gb) + 1; + bytestream2_skip(&gb, 3); /* duration */ + s->anmf_flags = bytestream2_get_byte(&gb); + + if (s->width + s->pos_x > s->canvas_width || + s->height + s->pos_y > s->canvas_height) { + av_log(avctx, AV_LOG_ERROR, + "frame does not fit into canvas\n"); + return AVERROR_INVALIDDATA; + } + break; case MKTAG('X', 'M', 'P', ' '): AV_WL32(chunk_str, chunk_type); av_log(avctx, AV_LOG_WARNING, "skipping unsupported chunk: %s\n", @@ -1558,9 +1999,31 @@ exif_end: } } - if (!*got_frame) { - av_log(avctx, AV_LOG_ERROR, "image data not found\n"); - return AVERROR_INVALIDDATA; + if (*got_frame) { + if (!(s->vp8x_flags & VP8X_FLAG_ANIMATION)) { + /* no animation, output the decoded frame */ + av_frame_move_ref(p, s->frame); + } else { + if (!s->canvas_frame) { + ret = init_canvas_frame(avctx); + if (ret < 0) + return ret; + } + + ret = blend_frame_into_canvas(avctx); + if (ret < 0) + return ret; + + ret = copy_canvas_to_frame(avctx, p); + if (ret < 0) + return ret; + + ret = dispose_frame_in_canvas(avctx); + if (ret < 0) + return ret; + } + + p->pts = avpkt->pts; } return avpkt->size; @@ -1570,6 +2033,10 @@ static av_cold int webp_decode_close(AVCodecContext *avctx) { WebPContext *s = avctx->priv_data; + av_frame_free(&s->canvas_frame); + av_frame_free(&s->frame); + av_packet_unref(&s->alpha_packet); + if (s->initialized) return ff_vp8_decode_free(avctx); @@ -1582,7 +2049,8 @@ AVCodec ff_webp_decoder = { .type = AVMEDIA_TYPE_VIDEO, .id = AV_CODEC_ID_WEBP, .priv_data_size = sizeof(WebPContext), + .init = webp_decode_init, .decode = webp_decode_frame, .close = webp_decode_close, - .capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_FRAME_THREADS, + .capabilities = AV_CODEC_CAP_DR1, }; diff --git a/libavcodec/webp_parser.c b/libavcodec/webp_parser.c index fdb7c38350..f959be8520 100644 --- a/libavcodec/webp_parser.c +++ b/libavcodec/webp_parser.c @@ -25,13 +25,16 @@ #include "libavutil/bswap.h" #include "libavutil/common.h" +#include "libavutil/intreadwrite.h" #include "parser.h" typedef struct WebPParseContext { ParseContext pc; + int frame; uint32_t fsize; - uint32_t remaining_size; + uint32_t remaining_file_size; + uint32_t remaining_tag_size; } WebPParseContext; static int webp_parse(AVCodecParserContext *s, AVCodecContext *avctx, @@ -41,62 +44,89 @@ static int webp_parse(AVCodecParserContext *s, AVCodecContext *avctx, WebPParseContext *ctx = s->priv_data; uint64_t state = ctx->pc.state64; int next = END_NOT_FOUND; - int i = 0; + int i, len; - *poutbuf = NULL; - *poutbuf_size = 0; - -restart: - if (ctx->pc.frame_start_found <= 8) { - for (; i < buf_size; i++) { + for (i = 0; i < buf_size;) { + if (ctx->remaining_tag_size) { + /* consuming tag */ + len = FFMIN(ctx->remaining_tag_size, buf_size - i); + i += len; + ctx->remaining_tag_size -= len; + ctx->remaining_file_size -= len; + } else if (ctx->frame) { + /* consumed tag containing frame, flush it */ + next = i; + ctx->frame = 0; + break; + } else { + /* scan for the next tag or file */ state = (state << 8) | buf[i]; - if (ctx->pc.frame_start_found == 0) { - if ((state >> 32) == MKBETAG('R', 'I', 'F', 'F')) { - ctx->fsize = av_bswap32(state); - if (ctx->fsize > 15 && ctx->fsize <= UINT32_MAX - 10) { - ctx->pc.frame_start_found = 1; - ctx->fsize += 8; + i++; + + if (!ctx->remaining_file_size) { + /* scan for the next file */ + if (ctx->pc.frame_start_found == 4) { + ctx->pc.frame_start_found = 0; + if ((uint32_t) state == MKBETAG('W', 'E', 'B', 'P')) { + if (i != 12) { + next = i - 12; + state = 0; + break; + } + ctx->remaining_file_size = ctx->fsize - 4; + continue; } } - } else if (ctx->pc.frame_start_found == 8) { - if ((state >> 32) != MKBETAG('W', 'E', 'B', 'P')) { + if (ctx->pc.frame_start_found == 0) { + if ((state >> 32) == MKBETAG('R', 'I', 'F', 'F')) { + ctx->fsize = av_bswap32(state); + if (ctx->fsize > 15 && ctx->fsize <= UINT32_MAX - 10) { + ctx->fsize += (ctx->fsize & 1); + ctx->pc.frame_start_found = 1; + } + } + } else + ctx->pc.frame_start_found++; + } else { + /* read the next tag */ + ctx->remaining_file_size--; + if (ctx->remaining_file_size == 0) { ctx->pc.frame_start_found = 0; continue; } ctx->pc.frame_start_found++; - ctx->remaining_size = ctx->fsize + i - 15; - if (ctx->pc.index + i > 15) { - next = i - 15; - state = 0; - break; - } else { - ctx->pc.state64 = 0; - goto restart; + if (ctx->pc.frame_start_found < 8) + continue; + + switch (state >> 32) { + case MKBETAG('A', 'N', 'M', 'F'): + case MKBETAG('V', 'P', '8', ' '): + case MKBETAG('V', 'P', '8', 'L'): + ctx->frame = 1; + break; + default: + ctx->frame = 0; + break; } - } else if (ctx->pc.frame_start_found) - ctx->pc.frame_start_found++; - } - ctx->pc.state64 = state; - } else { - if (ctx->remaining_size) { - i = FFMIN(ctx->remaining_size, buf_size); - ctx->remaining_size -= i; - if (ctx->remaining_size) - goto flush; - ctx->pc.frame_start_found = 0; - goto restart; + ctx->remaining_tag_size = av_bswap32(state); + ctx->remaining_tag_size += ctx->remaining_tag_size & 1; + if (ctx->remaining_tag_size > ctx->remaining_file_size) { + /* this is probably trash at the end of file */ + ctx->remaining_tag_size = ctx->remaining_file_size; + } + ctx->pc.frame_start_found = 0; + state = 0; + } } } + ctx->pc.state64 = state; -flush: - if (ff_combine_frame(&ctx->pc, next, &buf, &buf_size) < 0) + if (ff_combine_frame(&ctx->pc, next, &buf, &buf_size) < 0) { + *poutbuf = NULL; + *poutbuf_size = 0; return buf_size; - - if (next != END_NOT_FOUND && next < 0) - ctx->pc.frame_start_found = FFMAX(ctx->pc.frame_start_found - i - 1, 0); - else - ctx->pc.frame_start_found = 0; + } *poutbuf = buf; *poutbuf_size = buf_size; From patchwork Wed Jul 8 05:28:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zlomek, Josef" X-Patchwork-Id: 20891 Delivered-To: andriy.gelman@gmail.com Received: by 2002:a25:80ca:0:0:0:0:0 with SMTP id c10csp272548ybm; Tue, 7 Jul 2020 22:28:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxpIOoxxSv4T5fAbe5/OVGRmpVX/ajlnIRr71WBMhZTbfIwfxfSrVDUNOOqA6qkWVADCEgA X-Received: by 2002:a1c:e383:: with SMTP id a125mr5899177wmh.11.1594186134799; Tue, 07 Jul 2020 22:28:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594186134; cv=none; d=google.com; s=arc-20160816; b=OsqXOvOD3koT4CekTCBjtU7LRnZLZn/IPTys4Pde14pZpN8hB1rV6dQai86JXZxn6k hSEsEgCm1h+mxmb9QYo9W+c1MPwA0qg1BTua+y5L2Fju+NpqoXdkCMFjJivYpcnPUpuG wy8sGL8eaB9fsc/4ruwEmj4cFs0THH03Q6MVJf3Ju+PaPmRIjFfWLUmNi6cKpTDiKEni ZeslR5t68qXgknNi1z0dB8ReSWLU5opZFQMbZCITfHQPSedl3znRZ7F5LzR/1JCs6P31 nGvEWuTmyT9dimZIv21VVz3qMJclGmaISb4nrMOrfdkIQCvj+2TcEbG+tQwJuRl99RMJ xajA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to; bh=BO4B6MlsxI9Pq24xFB/08cLfa6DZYIc7d5vOfKmTALc=; b=Wdv9FW3yRpDGLokPGyMwx4UDh5bkUTVlRPvh47tELj8053i2lczKAFkx/sorY9MqUA Cl2kbxraLrI8TqWD9gKYeI2T896TdspqW8BfI53rV1EpAUx2oKREEtAFdKdUhedTK/jm rXlxROlTa/bj3nM69QQ4iSWpcWLKfGdljq+VgioDPrII/LXsWwcyN5dPfAc3bRHA4DfQ wUnU/n41oyV1hiUn4S4Tyb06qeD4O3Dr83XKXKBD5ce/dsCOvIld4XkXTu+f9k/2rfmo iRWKiYvAvDKCdmW907bT2U0qZxkEAMNQA7a7sfaExblg/kyxuLVCptX0U78CEMrPqBDW WrjA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=pex.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b6si3866913wmj.108.2020.07.07.22.28.54; Tue, 07 Jul 2020 22:28:54 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=pex.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BF79168A585; Wed, 8 Jul 2020 08:28:48 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vps.zlomek.net (vps.zlomek.net [195.181.211.90]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 367F0689FFE for ; Wed, 8 Jul 2020 08:28:43 +0300 (EEST) Received: by vps.zlomek.net (Postfix, from userid 1000) id DC10E603DB; Wed, 8 Jul 2020 07:28:42 +0200 (CEST) From: Josef Zlomek To: ffmpeg-devel@ffmpeg.org Date: Wed, 8 Jul 2020 07:28:24 +0200 Message-Id: <20200708052824.18582-2-josef@pex.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200708052824.18582-1-josef@pex.com> References: <20200708052824.18582-1-josef@pex.com> Subject: [FFmpeg-devel] [PATCH 2/2] libavformat: add WebP demuxer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Josef Zlomek MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 5tMz/BKHW4ww Content-Length: 19818 Fixes: 4907 Adds support for demuxing of animated WebP. The WebP demuxer splits the input stream into packets containing one frame. It also sets the timing information properly. Signed-off-by: Josef Zlomek --- Changelog | 1 + doc/demuxers.texi | 28 ++++ libavformat/Makefile | 1 + libavformat/allformats.c | 1 + libavformat/version.h | 2 +- libavformat/webpdec.c | 322 +++++++++++++++++++++++++++++++++++++++ 6 files changed, 354 insertions(+), 1 deletion(-) create mode 100644 libavformat/webpdec.c diff --git a/Changelog b/Changelog index 1e41040a8e..fc0bbdca45 100644 --- a/Changelog +++ b/Changelog @@ -6,6 +6,7 @@ version : - MacCaption demuxer - PGX decoder - animated WebP parser/decoder +- animated WebP demuxer version 4.3: diff --git a/doc/demuxers.texi b/doc/demuxers.texi index 3c15ab9eee..9b5932308b 100644 --- a/doc/demuxers.texi +++ b/doc/demuxers.texi @@ -832,4 +832,32 @@ which in turn, acts as a ceiling for the size of scripts that can be read. Default is 1 MiB. @end table +@section webp + +Animated WebP demuxer. + +It accepts the following options: + +@table @option +@item min_delay +Set the minimum valid delay between frames in milliseconds. +Range is 0 to 60000. Default value is 10. + +@item max_webp_delay +Set the maximum valid delay between frames in milliseconds. +Range is 0 to 16777215. Default value is 16777215 (over four hours), +the maximum value allowed by the specification. + +@item default_delay +Set the default delay between frames in milliseconds. +Range is 0 to 60000. Default value is 100. + +@item ignore_loop +WebP files can contain information to loop a certain number of times (or +infinitely). If @option{ignore_loop} is set to 1, then the loop setting +from the input will be ignored and looping will not occur. If set to 0, +then looping will occur and will cycle the number of times according to +the WebP. Default value is 1. +@end table + @c man end DEMUXERS diff --git a/libavformat/Makefile b/libavformat/Makefile index 26af859a28..93793de45d 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -557,6 +557,7 @@ OBJS-$(CONFIG_WEBM_MUXER) += matroskaenc.o matroska.o \ wv.o vorbiscomment.o OBJS-$(CONFIG_WEBM_DASH_MANIFEST_MUXER) += webmdashenc.o OBJS-$(CONFIG_WEBM_CHUNK_MUXER) += webm_chunk.o +OBJS-$(CONFIG_WEBP_DEMUXER) += webpdec.o OBJS-$(CONFIG_WEBP_MUXER) += webpenc.o OBJS-$(CONFIG_WEBVTT_DEMUXER) += webvttdec.o subtitles.o OBJS-$(CONFIG_WEBVTT_MUXER) += webvttenc.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index f8527b1fd4..389273ea52 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -455,6 +455,7 @@ extern AVOutputFormat ff_webm_muxer; extern AVInputFormat ff_webm_dash_manifest_demuxer; extern AVOutputFormat ff_webm_dash_manifest_muxer; extern AVOutputFormat ff_webm_chunk_muxer; +extern AVInputFormat ff_webp_demuxer; extern AVOutputFormat ff_webp_muxer; extern AVInputFormat ff_webvtt_demuxer; extern AVOutputFormat ff_webvtt_muxer; diff --git a/libavformat/version.h b/libavformat/version.h index 75c03fde0a..33cebed85e 100644 --- a/libavformat/version.h +++ b/libavformat/version.h @@ -32,7 +32,7 @@ // Major bumping may affect Ticket5467, 5421, 5451(compatibility with Chromium) // Also please add any ticket numbers that you believe might be affected here #define LIBAVFORMAT_VERSION_MAJOR 58 -#define LIBAVFORMAT_VERSION_MINOR 48 +#define LIBAVFORMAT_VERSION_MINOR 49 #define LIBAVFORMAT_VERSION_MICRO 100 #define LIBAVFORMAT_VERSION_INT AV_VERSION_INT(LIBAVFORMAT_VERSION_MAJOR, \ diff --git a/libavformat/webpdec.c b/libavformat/webpdec.c new file mode 100644 index 0000000000..8d6e6df9c0 --- /dev/null +++ b/libavformat/webpdec.c @@ -0,0 +1,322 @@ +/* + * WebP demuxer + * Copyright (c) 2020 Pexeso Inc. + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * WebP demuxer. + */ + +#include "libavutil/intreadwrite.h" +#include "libavutil/opt.h" +#include "avformat.h" +#include "internal.h" + +typedef struct WebPDemuxContext { + const AVClass *class; + /** + * Time span in milliseconds before the next frame + * should be drawn on screen. + */ + int delay; + /** + * Minimum allowed delay between frames in milliseconds. + * Values below this threshold are considered to be invalid + * and set to value of default_delay. + */ + int min_delay; + int max_delay; + int default_delay; + + /** + * loop options + */ + int total_iter; + int iter_count; + int ignore_loop; + + int nb_frames; + int remaining_size; +} WebPDemuxContext; + +/** + * Major web browsers display WebPs at ~10-15fps when rate is not + * explicitly set or have too low values. We assume default rate to be 10. + * Default delay = 1000 microseconds / 10fps = 100 milliseconds per frame. + */ +#define WEBP_DEFAULT_DELAY 100 +/** + * By default delay values less than this threshold considered to be invalid. + */ +#define WEBP_MIN_DELAY 10 + +static int webp_probe(const AVProbeData *p) +{ + const uint8_t *b = p->buf; + + if (p->filename && ff_guess_image2_codec(p->filename)) { + if (AV_RB32(b) == MKBETAG('R', 'I', 'F', 'F') && + AV_RB32(b + 8) == MKBETAG('W', 'E', 'B', 'P')) + return AVPROBE_SCORE_MAX; + } + + return 0; +} + +static int resync(AVFormatContext *s) +{ + WebPDemuxContext *wdc = s->priv_data; + AVIOContext *pb = s->pb; + int i; + uint64_t state = 0; + + for (i = 0; i < 12; i++) { + state = (state << 8) | avio_r8(pb); + if (i == 11) { + if ((uint32_t) state == MKBETAG('W', 'E', 'B', 'P')) + return 0; + i -= 4; + } + if (i == 7) { + if ((state >> 32) != MKBETAG('R', 'I', 'F', 'F')) { + i--; + } else { + uint32_t fsize = av_bswap32(state); + if (!(fsize > 15 && fsize <= UINT32_MAX - 10)) { + i -= 4; + } else { + wdc->remaining_size = fsize - 4; + } + } + } + if (avio_feof(pb)) + return AVERROR_EOF; + } + return 0; +} + +static int webp_read_header(AVFormatContext *s) +{ + WebPDemuxContext *wdc = s->priv_data; + AVIOContext *pb = s->pb; + AVStream *st; + int ret, n; + int64_t nb_frames = 0, duration = 0; + int width = 0, height = 0; + uint32_t chunk_type, chunk_size; + + ret = resync(s); + if (ret < 0) + return ret; + + st = avformat_new_stream(s, NULL); + if (!st) + return AVERROR(ENOMEM); + + st->codecpar->width = 0; + st->codecpar->height = 0; + wdc->delay = wdc->default_delay; + + while (1) { + chunk_type = avio_rl32(pb); + chunk_size = avio_rl32(pb); + if (chunk_size == UINT32_MAX) + return AVERROR_INVALIDDATA; + chunk_size += chunk_size & 1; + if (avio_feof(pb)) + break; + + switch (chunk_type) { + case MKTAG('V', 'P', '8', 'X'): + avio_skip(pb, 4); + width = avio_rl24(pb) + 1; + height = avio_rl24(pb) + 1; + break; + case MKTAG('V', 'P', '8', ' '): + avio_skip(pb, 6); + width = avio_rl16(pb) & 0x3fff; + height = avio_rl16(pb) & 0x3fff; + duration += wdc->delay; + nb_frames++; + avio_skip(pb, chunk_size - 10); + break; + case MKTAG('V', 'P', '8', 'L'): + avio_skip(pb, 1); + n = avio_rl32(pb); + width = (n & 0x3fff) + 1; /* first 14 bits */ + height = ((n >> 14) & 0x3fff) + 1; /* next 14 bits */ + duration += wdc->delay; + nb_frames++; + avio_skip(pb, chunk_size - 5); + break; + case MKTAG('A', 'N', 'M', 'F'): + avio_skip(pb, 6); + width = avio_rl24(pb) + 1; + height = avio_rl24(pb) + 1; + wdc->delay = avio_rl24(pb); + if (wdc->delay < wdc->min_delay) + wdc->delay = wdc->default_delay; + wdc->delay = FFMIN(wdc->delay, wdc->max_delay); + duration += wdc->delay; + nb_frames++; + avio_skip(pb, chunk_size - 15); + break; + default: + avio_skip(pb, chunk_size); + } + + if (avio_feof(pb)) + break; + + if (st->codecpar->width == 0 && width > 0) + st->codecpar->width = width; + if (st->codecpar->height == 0 && height > 0) + st->codecpar->height = height; + } + + /* WebP format operates with time in "milliseconds", + * therefore timebase is 1/1000 */ + avpriv_set_pts_info(st, 64, 1, 1000); + st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; + st->codecpar->codec_id = AV_CODEC_ID_WEBP; + st->start_time = 0; + st->duration = duration; + st->nb_frames = nb_frames; + + /* jump to start because WebP decoder needs header data too */ + if (avio_seek(pb, 0, SEEK_SET) != 0) + return AVERROR(EIO); + wdc->remaining_size = 0; + + return 0; +} + +static int webp_read_packet(AVFormatContext *s, AVPacket *pkt) +{ + WebPDemuxContext *wdc = s->priv_data; + AVIOContext *pb = s->pb; + int ret; + int64_t frame_start = avio_tell(pb), frame_end; + uint32_t chunk_type, chunk_size; + int is_frame = 0; + + if (wdc->remaining_size == 0) { + ret = resync(s); + if (ret == AVERROR_EOF) { + if (!wdc->ignore_loop && avio_feof(pb) + && (wdc->total_iter < 0 || ++wdc->iter_count < wdc->total_iter)) + return avio_seek(pb, 0, SEEK_SET); + return AVERROR_EOF; + } + if (ret < 0) + return ret; + + wdc->delay = wdc->default_delay; + } + + while (!is_frame && wdc->remaining_size > 0 && !avio_feof(pb)) { + chunk_type = avio_rl32(pb); + chunk_size = avio_rl32(pb); + if (chunk_size == UINT32_MAX) + return AVERROR_INVALIDDATA; + chunk_size += chunk_size & 1; + + if (wdc->remaining_size < 8 + chunk_size) + return AVERROR_INVALIDDATA; + wdc->remaining_size -= 8 + chunk_size; + + switch (chunk_type) { + case MKTAG('A', 'N', 'I', 'M'): + avio_skip(pb, 4); + wdc->total_iter = avio_rl16(pb); + if (wdc->total_iter == 0) + wdc->total_iter = -1; + ret = avio_skip(pb, chunk_size - 6); + break; + case MKTAG('A', 'N', 'M', 'F'): + avio_skip(pb, 12); + wdc->delay = avio_rl24(pb); + if (wdc->delay < wdc->min_delay) + wdc->delay = wdc->default_delay; + wdc->delay = FFMIN(wdc->delay, wdc->max_delay); + wdc->nb_frames++; + is_frame = 1; + ret = avio_skip(pb, chunk_size - 15); + break; + case MKTAG('V', 'P', '8', ' '): + case MKTAG('V', 'P', '8', 'L'): + wdc->nb_frames++; + is_frame = 1; + /* fallthrough */ + default: + ret = avio_skip(pb, chunk_size); + break; + } + + if (ret < 0) + return ret; + } + + frame_end = avio_tell(pb); + + if (avio_seek(pb, frame_start, SEEK_SET) != frame_start) + return AVERROR(EIO); + + ret = av_get_packet(pb, pkt, frame_end - frame_start); + if (ret < 0) + return ret; + + pkt->flags |= AV_PKT_FLAG_KEY; + pkt->stream_index = 0; + pkt->duration = is_frame ? wdc->delay : 0; + + if (is_frame && wdc->nb_frames == 1) { + s->streams[0]->r_frame_rate = (AVRational) {1000, pkt->duration}; + } + + return ret; +} + +static const AVOption options[] = { + { "min_delay" , "minimum valid delay between frames (in milliseconds)", offsetof(WebPDemuxContext, min_delay) , AV_OPT_TYPE_INT, {.i64 = WEBP_MIN_DELAY} , 0, 1000 * 60, AV_OPT_FLAG_DECODING_PARAM }, + { "max_webp_delay", "maximum valid delay between frames (in milliseconds)", offsetof(WebPDemuxContext, max_delay) , AV_OPT_TYPE_INT, {.i64 = 0xffffff} , 0, 0xffffff , AV_OPT_FLAG_DECODING_PARAM }, + { "default_delay" , "default delay between frames (in milliseconds)" , offsetof(WebPDemuxContext, default_delay), AV_OPT_TYPE_INT, {.i64 = WEBP_DEFAULT_DELAY}, 0, 1000 * 60, AV_OPT_FLAG_DECODING_PARAM }, + { "ignore_loop" , "ignore loop setting" , offsetof(WebPDemuxContext, ignore_loop) , AV_OPT_TYPE_BOOL,{.i64 = 1} , 0, 1 , AV_OPT_FLAG_DECODING_PARAM }, + { NULL }, +}; + +static const AVClass demuxer_class = { + .class_name = "WebP demuxer", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, + .category = AV_CLASS_CATEGORY_DEMUXER, +}; + +AVInputFormat ff_webp_demuxer = { + .name = "webp", + .long_name = NULL_IF_CONFIG_SMALL("WebP image"), + .priv_data_size = sizeof(WebPDemuxContext), + .read_probe = webp_probe, + .read_header = webp_read_header, + .read_packet = webp_read_packet, + .flags = AVFMT_GENERIC_INDEX, + .priv_class = &demuxer_class, +};