From patchwork Tue Oct 24 16:40:22 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bjorn Roche X-Patchwork-Id: 5678 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.90 with SMTP id m26csp830636jah; Tue, 24 Oct 2017 09:40:39 -0700 (PDT) X-Google-Smtp-Source: ABhQp+RLkxq1Pvrb7ja+bRRdikcOiQRWD4piCIBaXtEjaSPikcOTTaTrDJe1MDECBWZopATU0Nl0 X-Received: by 10.223.179.1 with SMTP id j1mr5125154wrd.105.1508863239163; Tue, 24 Oct 2017 09:40:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1508863239; cv=none; d=google.com; s=arc-20160816; b=KPqk+EdD4VWXDZ7yg7L6pbzFmkofhT70tXBpMceDTd/hrq6PtuZhpWfxINo5vaHxIG ouEB1XwausmCsqbdu2TtiB+Qbc8vyA3EeGqNLawXIl3btXfPfBiGV1BUCEVV01VXyHVV FQKlryJkERPwiQ5ze2FGosmdc2N7pyWivoYkoLaDVI2hQOXI/LGaaI8+bdoy/QTg9sRQ VvkfhbSv3duPP/pTXa3GG8MGePq6QVlZOlo9yoA0cruaMhyyRC8WifeLBwI99w+DJmX7 WSKZIciGXKdXa9P/oJCMDv2lbCBs05CBJbt5thjlTqOwCWRUG6vU/VPp8yIFmtXKvZ8z 3G7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to:arc-authentication-results; bh=nn8blwWv+98Fj/NWgo0uUFuM7mwgszIfUDhKTe6TgOI=; b=fioxcrelR9GSvU1r9kJK8EF0rTSegokT6yCsQ/+8sZsRUNrBjGKSufz1mXX5Nwyrg2 J71S6VR/W5vRjEjmptlh+nXNhPxLEI46vpkDs3inYszJcyLZ+6UkRE17byVos+Gr3ErR xuWMEZxL/tpp4k3uiNviGX/ZAqYkG7xxZq2Xf1fZkJFJeJgJ9fYoGYrOGRj2Jpzz3wMS /komnVWLgnuRS1NK+5ZYj8pGM5/Lj04CCFzS7wVthPNAXlWEARcUSkUxP3FiZDX+lFsm IUOKtJ+PkY8GO38tmfk5q/oANa+HOvuU6NZ8ZnM7M8EvAkI6p6U3S2KdDxqtKXcJV0DG DTdQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@giphy-com.20150623.gappssmtp.com header.s=20150623 header.b=MHT3BAwR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r13si496687wra.497.2017.10.24.09.40.38; Tue, 24 Oct 2017 09:40:39 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@giphy-com.20150623.gappssmtp.com header.s=20150623 header.b=MHT3BAwR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8C96268A2BD; Tue, 24 Oct 2017 19:40:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk0-f180.google.com (mail-qk0-f180.google.com [209.85.220.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CE675689FF1 for ; Tue, 24 Oct 2017 19:40:21 +0300 (EEST) Received: by mail-qk0-f180.google.com with SMTP id m189so27018177qke.4 for ; Tue, 24 Oct 2017 09:40:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=giphy-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=qtpp62vsyYeX4ayOLDA3T8SlOadQmA0tgiTlFhQRgYc=; b=MHT3BAwRRx/TvgVWZpTnDcr9JsUwD67D95yixWPzVFAg4O7J7xxVu+ug27DEjsV7PL 41LaA94jAwe0LPC53gsexxMUTYwy91mlQiLbp62c9pIXkAlQoQNrCqdNsmX9cQm/H80E cekiMr0K3pduO7s/61emQBJlXF8E7KxbOTjNjU6+HGMo7tYK5xtzu6NpFshuybKdyTHh zJu4fLWoWDylVTFBQOe8ssX5Yq3P+ZrCpoNcX7ZrRXW7SgOeMbQGNNwwtft61WumZI8o JwucJFazMzZ4SFME5gRAtPQrQdA6PoZcrsY5VD1IbaON83H875/ZsBaIhmKA/81IMZLO FeYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=qtpp62vsyYeX4ayOLDA3T8SlOadQmA0tgiTlFhQRgYc=; b=MdzmAoHJm3VxMj78OTuOfpPpaKzQEqH+PvDaBd4LRHuST31TC2TPLnlCAqUHWJaGdE fxcLOapL+ZMHExhIitm0AOsOTfmLw+uDGCF9gvUDngl+QCpmrZXMXPbbLprFq8JhCHbQ Z5ia310sDRHvBtMRk7T4z2oKPXjd3On3oW+X9QkCsEFP14a9vL5jm1+dYtTPFZvSkwef Wxu21PtAEJ5xhRv6iX+Ua59qQVcXAko+f13qWycUZ+w7u9IULIUqNH8wKfjJk4cuFRiT 62/kEoN87DDmnqd2LBb9EthQpZPSvQmh5orU/GEAUivbnC+YF/VYuDRadx3DWh1USGLk 7c6A== X-Gm-Message-State: AMCzsaW6nGc5slFt3Gn++qQ7Dl3ybZQwSG4p+aOGaeMAIrF3xwDWOr9o SC0LENWKUwut62UT+laA+/AL9EWcCQI= X-Received: by 10.55.75.75 with SMTP id y72mr23597360qka.118.1508863227436; Tue, 24 Oct 2017 09:40:27 -0700 (PDT) Received: from localhost.localdomain ([208.184.100.82]) by smtp.gmail.com with ESMTPSA id 2sm508301qto.28.2017.10.24.09.40.26 (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 24 Oct 2017 09:40:26 -0700 (PDT) From: Bjorn Roche To: ffmpeg-devel@ffmpeg.org Date: Tue, 24 Oct 2017 12:40:22 -0400 Message-Id: <20171024164022.64121-1-bjorn@giphy.com> X-Mailer: git-send-email 2.14.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] fix for transparencies in animated gifs (requires feedback) X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Bjorn Roche Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Support for transparencies in animated gifs requires modifying both libavcodec/gif.c and libavformat/gif.c because both the graphics control extension (handled by libavformat/gif.c) and the raw frame data (handled by libavcodec/gif.c) must be changed. This is because transparencies in GIF can be used both to create a transparent image, and to provide optimization. How transparencies are interpreted in a given frame is controlled by the “disposal method”, which must be set appropriately in the graphics control extension. The “in place” disposal method is used when transparency indicates optimization, and the “background” disposal method is used when transparency is intended to be preserved. In order to support both disposal methods, libavcodec/gif.c must signal to libavformat/gif.c which disposal method is required for every frame. This is done with a new side data type: AV_PKT_DATA_GIF_FRAME_DISPOSAL. This requires a change to avcodec.h Unfortunately, the addition of a new side data type causes some of the FATE tests to fail. This is not addressed here. This patch assumes paletteuse has already been patched to support transparency. (e.g. lavfi/paletteuse: fix to support transparency) Feedback I definitely need: - I’ve done a fair bit of testing, but I’m sure I missed some important cases. - I don’t know if/how to update the FATE tests. --- libavcodec/avcodec.h | 6 ++ libavcodec/gif.c | 196 +++++++++++++++++++++++++++++++++++++++++++++++++-- libavformat/gif.c | 16 ++++- 3 files changed, 212 insertions(+), 6 deletions(-) diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 52cc5b0ca0..82a5328ce1 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1599,6 +1599,12 @@ enum AVPacketSideDataType { */ AV_PKT_DATA_CONTENT_LIGHT_LEVEL, + /** + * The disposal method that should be used with the frame. If missing, + * the frame will not be disposed. This contains exactly one byte. + */ + AV_PKT_DATA_GIF_FRAME_DISPOSAL, + /** * ATSC A53 Part 4 Closed Captions. This metadata should be associated with * a video stream. A53 CC bitstream is stored as uint8_t in AVPacketSideData.data. diff --git a/libavcodec/gif.c b/libavcodec/gif.c index d9c99d52cf..6c839d99d2 100644 --- a/libavcodec/gif.c +++ b/libavcodec/gif.c @@ -74,11 +74,35 @@ static int pick_palette_entry(const uint8_t *buf, int linesize, int w, int h) return -1; } -static int gif_image_write_image(AVCodecContext *avctx, - uint8_t **bytestream, uint8_t *end, - const uint32_t *palette, - const uint8_t *buf, const int linesize, - AVPacket *pkt) +// returns true if any of the pixels are transparent +static int is_image_translucent(AVCodecContext *avctx, + const uint32_t *palette, + const uint8_t *buf, const int linesize) +{ + GIFContext *s = avctx->priv_data; + int trans = s->transparent_index; + int p; + const int m = avctx->width * avctx->height ; + + if (trans < 0) { + return 0; + } + + for (p=0; ppriv_data; int len = 0, height = avctx->height, width = avctx->width, x, y; @@ -137,6 +161,11 @@ static int gif_image_write_image(AVCodecContext *avctx, width, height, x_start, y_start, avctx->width, avctx->height); } + uint8_t *frame_disposal = av_packet_new_side_data(pkt, AV_PKT_DATA_GIF_FRAME_DISPOSAL, 1); + if (!frame_disposal) + return AVERROR(ENOMEM); + *frame_disposal = GCE_DISPOSAL_INPLACE; + /* image block */ bytestream_put_byte(bytestream, GIF_IMAGE_SEPARATOR); bytestream_put_le16(bytestream, x_start); @@ -214,6 +243,163 @@ static int gif_image_write_image(AVCodecContext *avctx, return 0; } +// wrtites an image that may contain transparency +// this should work for opaque images as well, but will be less optimized. +static int gif_image_write_translucent(AVCodecContext *avctx, + uint8_t **bytestream, uint8_t *end, + const uint32_t *palette, + const uint8_t *buf, const int linesize, + AVPacket *pkt) +{ + GIFContext *s = avctx->priv_data; + int len = 0, height = avctx->height, width = avctx->width, y; + int x_start = 0, y_start = 0, trans = s->transparent_index; + const uint8_t *ptr; + + /* Crop Image */ + if ((s->flags & GF_OFFSETTING) && trans >=0) { + const int w = avctx->width; + const int h = avctx->height; + int x_end = w - 1, + y_end = h - 1; + + // crop top + while (y_start < y_end) { + int i; + int is_trans = 1; + for (i=0; iwidth, avctx->height); + } + + + uint8_t *frame_disposal = av_packet_new_side_data(pkt, AV_PKT_DATA_GIF_FRAME_DISPOSAL, 1); + if (!frame_disposal) + return AVERROR(ENOMEM); + *frame_disposal = GCE_DISPOSAL_BACKGROUND; + + /* image block */ + bytestream_put_byte(bytestream, GIF_IMAGE_SEPARATOR); + bytestream_put_le16(bytestream, x_start); + bytestream_put_le16(bytestream, y_start); + bytestream_put_le16(bytestream, width); + bytestream_put_le16(bytestream, height); + + if (!palette) { + bytestream_put_byte(bytestream, 0x00); /* flags */ + } else { + unsigned i; + bytestream_put_byte(bytestream, 1<<7 | 0x7); /* flags */ + for (i = 0; i < AVPALETTE_COUNT; i++) { + const uint32_t v = palette[i]; + bytestream_put_be24(bytestream, v); + } + } + + bytestream_put_byte(bytestream, 0x08); + + ff_lzw_encode_init(s->lzw, s->buf, s->buf_size, + 12, FF_LZW_GIF, put_bits); + + ptr = buf + y_start*linesize + x_start; + + for (y = 0; y < height; y++) { + len += ff_lzw_encode(s->lzw, ptr, width); + ptr += linesize; + } + + len += ff_lzw_encode_flush(s->lzw, flush_put_bits); + + ptr = s->buf; + while (len > 0) { + int size = FFMIN(255, len); + bytestream_put_byte(bytestream, size); + if (end - *bytestream < size) + return -1; + bytestream_put_buffer(bytestream, ptr, size); + ptr += size; + len -= size; + } + bytestream_put_byte(bytestream, 0x00); /* end of image block */ + + return 0; +} + +static int gif_image_write_image(AVCodecContext *avctx, + uint8_t **bytestream, uint8_t *end, + const uint32_t *palette, + const uint8_t *buf, const int linesize, + AVPacket *pkt) +{ + GIFContext *s = avctx->priv_data; + if (!s->last_frame) { + return gif_image_write_opaque(avctx, bytestream, end, palette, buf, linesize, pkt); + } + + if (is_image_translucent(avctx, palette, buf, linesize)) { + return gif_image_write_translucent(avctx, bytestream, end, palette, buf, linesize, pkt); + } else { + return gif_image_write_opaque(avctx, bytestream, end, palette, buf, linesize, pkt); + } +} + static av_cold int gif_encode_init(AVCodecContext *avctx) { GIFContext *s = avctx->priv_data; diff --git a/libavformat/gif.c b/libavformat/gif.c index 01d98a27b0..b4a8b6aa94 100644 --- a/libavformat/gif.c +++ b/libavformat/gif.c @@ -144,6 +144,8 @@ static int flush_packet(AVFormatContext *s, AVPacket *new) AVIOContext *pb = s->pb; const uint32_t *palette; AVPacket *pkt = gif->prev_pkt; + uint8_t *disposal; + uint8_t packed; if (!pkt) return 0; @@ -157,16 +159,28 @@ static int flush_packet(AVFormatContext *s, AVPacket *new) } bcid = get_palette_transparency_index(palette); + disposal = av_packet_get_side_data(pkt, AV_PKT_DATA_GIF_FRAME_DISPOSAL, &size); + if (disposal && size != 1) { + av_log(s, AV_LOG_ERROR, "Invalid gif frame disposal extradata\n"); + return AVERROR_INVALIDDATA; + } + if (new && new->pts != AV_NOPTS_VALUE) gif->duration = av_clip_uint16(new->pts - gif->prev_pkt->pts); else if (!new && gif->last_delay >= 0) gif->duration = gif->last_delay; /* graphic control extension block */ + if (disposal) { + packed = (0xff & (*disposal)<<2) | (bcid >= 0); + } else { + packed = 1<<2 | (bcid >= 0); + } + avio_w8(pb, 0x21); avio_w8(pb, 0xf9); avio_w8(pb, 0x04); /* block size */ - avio_w8(pb, 1<<2 | (bcid >= 0)); + avio_w8(pb, packed); avio_wl16(pb, gif->duration); avio_w8(pb, bcid < 0 ? DEFAULT_TRANSPARENCY_INDEX : bcid); avio_w8(pb, 0x00);