From patchwork Wed Apr 17 19:19:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Diego Felix de Souza via ffmpeg-devel X-Patchwork-Id: 48117 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:ce4e:b0:1a9:af23:56c1 with SMTP id id14csp1232466pzb; Wed, 17 Apr 2024 12:21:05 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCX0ngNUy76WUIDE3fJw19fXBbQVJm11U2LzPnukai6TBwSt6y6fei9X+VjELMrXk16PV3jnPILesOMD2ZOPZjZtvZMN6VwYdkOq1A== X-Google-Smtp-Source: AGHT+IG3YG7TVyL761xgnk5TQvhzpYep5BaPayTEApf3eGcsqza2yFzTZA4JQ7ZlFLjgqn3x2lhL X-Received: by 2002:a17:906:b183:b0:a55:201f:75f with SMTP id w3-20020a170906b18300b00a55201f075fmr263234ejy.36.1713381664993; Wed, 17 Apr 2024 12:21:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1713381664; cv=none; d=google.com; s=arc-20160816; b=jTc9kHSb40nWx0kCmOKzs/bTKCs4oMx4Q8Mj/prAnJTqvmbm8kqOTR991qanTahvyC FEgtadcpCUkJuTbXfryjEMoxRsxMjvXB+Z5l1z6WOEo5epQRSaHJYxKBNglIEY5KPAFB 22EzU9++Sbt/AYAn9BKKHwC2hDM7VRXXmwY2bGAGJHk7uZuiNEmQW2+mygJJX5sxjpok K06Q9MyvZWPH0JGNlQSlbTnM0xcTzwq/Cs7GURBynBfki4lb9qeRSIrMS7qR8FS377+m rOO2a7sYyOR/4pqjOgUsWvl3+KOCNH5TztdzX286X7Y1rJZ7ruP/AOQPmO8AfZabrPRN YuYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:delivered-to; bh=KsUpl1Tjifjjz25MQ7rFqyuzLMMzOgvLJeEjj03xmIk=; fh=XuleKIHF9MUFgiZ1qrMyNqDu84gapm8sSgQI0zYajBE=; b=y2JBgsL22bVtmJQ8t5iCbpxa5XZJuwqbdX3Q6blfXW0gfJ1k/S4C2YGhgGRHwj6r2T DygGQFvQzPfoylTRtmPqWhhiByTCGIxs7Qrdn64bXd1YrH6OUzy9k/izhvHO+w4wk6Cg R++SCIsh4wY45K+K5F7Gh1Ee9LeWJllhdikVZCjJChqXLj+KmSchN1ukOSTzc4PUlA86 NmHkJkzUSyOgngQqV1hC2aXoGfeT/k0poO+98TcKHGG0XseC6iWM+OVFptIwhVdIOvEE 4NG1IVQzfN8LVJC2yf0daasqfUD6vmQqgO+CJ4THY0rbyvGEChgmPJe8gdBIYlJxsCXl CNNA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qk11-20020a1709077f8b00b00a4739d9d28fsi7230891ejc.556.2024.04.17.12.21.04; Wed, 17 Apr 2024 12:21:04 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 74FBD68D3A5; Wed, 17 Apr 2024 22:20:56 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from shout01.mail.de (shout01.mail.de [62.201.172.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0713768D37F for ; Wed, 17 Apr 2024 22:20:50 +0300 (EEST) Received: from postfix01.mail.de (postfix01.bt.mail.de [10.0.121.125]) by shout01.mail.de (Postfix) with ESMTP id A264C240E50 for ; Wed, 17 Apr 2024 21:20:49 +0200 (CEST) Received: from smtp01.mail.de (smtp03.bt.mail.de [10.0.121.213]) by postfix01.mail.de (Postfix) with ESMTP id 8B03B8029D for ; Wed, 17 Apr 2024 21:20:49 +0200 (CEST) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp01.mail.de (Postfix) with ESMTPSA id 12ED624184A for ; Wed, 17 Apr 2024 21:20:44 +0200 (CEST) To: ffmpeg-devel@ffmpeg.org Date: Wed, 17 Apr 2024 12:19:59 -0700 Message-ID: <20240417192012.22436-4-thilo.borgmann@mail.de> In-Reply-To: <20240417192012.22436-1-thilo.borgmann@mail.de> References: <20240417192012.22436-1-thilo.borgmann@mail.de> MIME-Version: 1.0 X-purgate: clean X-purgate: This mail is considered clean (visit http://www.eleven.de for further information) X-purgate-type: clean X-purgate-Ad: Categorized by eleven eXpurgate (R) http://www.eleven.de X-purgate: This mail is considered clean (visit http://www.eleven.de for further information) X-purgate: clean X-purgate-size: 13324 X-purgate-ID: 154282::1713381649-551C31F9-91D3EA56/0/0 Subject: [FFmpeg-devel] [PATCH v12 3/8] avcodec/bsf: Add awebp2webp bitstream filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Thilo Borgmann via ffmpeg-devel From: Diego Felix de Souza via ffmpeg-devel Reply-To: FFmpeg development discussions and patches Cc: thilo.borgmann@mail.de Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9vwAH0jFegvf From: Thilo Borgmann via ffmpeg-devel Splits a packet containing a webp animations into one non-compliant packet per frame of the animation. Skips RIFF and WEBP chunks for those packets except for the first. Copyies ICC, EXIF and XMP chunks first into each of the packets except for the first. --- configure | 1 + libavcodec/bitstream_filters.c | 1 + libavcodec/bsf/Makefile | 1 + libavcodec/bsf/awebp2webp.c | 350 +++++++++++++++++++++++++++++++++ 4 files changed, 353 insertions(+) create mode 100644 libavcodec/bsf/awebp2webp.c diff --git a/configure b/configure index 55f1fc354d..2d08bc1fd8 100755 --- a/configure +++ b/configure @@ -3425,6 +3425,7 @@ aac_adtstoasc_bsf_select="adts_header mpeg4audio" av1_frame_merge_bsf_select="cbs_av1" av1_frame_split_bsf_select="cbs_av1" av1_metadata_bsf_select="cbs_av1" +awebp2webp_bsf_select="" dts2pts_bsf_select="cbs_h264 h264parse" eac3_core_bsf_select="ac3_parser" evc_frame_merge_bsf_select="evcparse" diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index 12860c332b..af88283a8c 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -28,6 +28,7 @@ extern const FFBitStreamFilter ff_aac_adtstoasc_bsf; extern const FFBitStreamFilter ff_av1_frame_merge_bsf; extern const FFBitStreamFilter ff_av1_frame_split_bsf; extern const FFBitStreamFilter ff_av1_metadata_bsf; +extern const FFBitStreamFilter ff_awebp2webp_bsf; extern const FFBitStreamFilter ff_chomp_bsf; extern const FFBitStreamFilter ff_dump_extradata_bsf; extern const FFBitStreamFilter ff_dca_core_bsf; diff --git a/libavcodec/bsf/Makefile b/libavcodec/bsf/Makefile index fb70ad0c21..48c67dd210 100644 --- a/libavcodec/bsf/Makefile +++ b/libavcodec/bsf/Makefile @@ -5,6 +5,7 @@ OBJS-$(CONFIG_AAC_ADTSTOASC_BSF) += bsf/aac_adtstoasc.o OBJS-$(CONFIG_AV1_FRAME_MERGE_BSF) += bsf/av1_frame_merge.o OBJS-$(CONFIG_AV1_FRAME_SPLIT_BSF) += bsf/av1_frame_split.o OBJS-$(CONFIG_AV1_METADATA_BSF) += bsf/av1_metadata.o +OBJS-$(CONFIG_AWEBP2WEBP_BSF) += bsf/awebp2webp.o OBJS-$(CONFIG_CHOMP_BSF) += bsf/chomp.o OBJS-$(CONFIG_DCA_CORE_BSF) += bsf/dca_core.o OBJS-$(CONFIG_DTS2PTS_BSF) += bsf/dts2pts.o diff --git a/libavcodec/bsf/awebp2webp.c b/libavcodec/bsf/awebp2webp.c new file mode 100644 index 0000000000..ebd123c667 --- /dev/null +++ b/libavcodec/bsf/awebp2webp.c @@ -0,0 +1,350 @@ +/* + * Animated WebP into non-compliant WebP bitstream filter + * Copyright (c) 2024 Thilo Borgmann + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Animated WebP into non-compliant WebP bitstream filter + * Splits a packet containing a webp animations into + * one non-compliant packet per frame of the animation. + * Skips RIFF and WEBP chunks for those packets except + * for the first. Copyies ICC, EXIF and XMP chunks first + * into each of the packets except for the first. + * @author Thilo Borgmann + */ + +#include +#include + +#include "codec_id.h" +#include "bytestream.h" +#include "libavutil/error.h" + +#include "bsf.h" +#include "bsf_internal.h" +#include "packet.h" + +#define VP8X_FLAG_ANIMATION 0x02 +#define VP8X_FLAG_XMP_METADATA 0x04 +#define VP8X_FLAG_EXIF_METADATA 0x08 +#define VP8X_FLAG_ALPHA 0x10 +#define VP8X_FLAG_ICC 0x20 + +typedef struct WEBPBSFContext { + const AVClass *class; + GetByteContext gb; + + AVPacket *last_pkt; + uint8_t *last_iccp; + uint8_t *last_exif; + uint8_t *last_xmp; + + int iccp_size; + int exif_size; + int xmp_size; + + int add_iccp; + int add_exif; + int add_xmp; + + uint64_t last_pts; +} WEBPBSFContext; + +static int save_chunk(WEBPBSFContext *ctx, uint8_t **buf, int *buf_size, uint32_t chunk_size) +{ + if (*buf || !buf_size || !chunk_size) + return 0; + + *buf = av_malloc(chunk_size + 8); + if (!*buf) + return AVERROR(ENOMEM); + + *buf_size = chunk_size + 8; + + bytestream2_seek(&ctx->gb, -8, SEEK_CUR); + bytestream2_get_buffer(&ctx->gb, *buf, chunk_size + 8); + + return 0; +} + +static int awebp2webp_filter(AVBSFContext *ctx, AVPacket *out) +{ + WEBPBSFContext *s = ctx->priv_data; + AVPacket *in; + uint32_t chunk_type; + uint32_t chunk_size; + int64_t packet_start; + int64_t packet_end; + int64_t out_off; + int ret = 0; + int is_frame = 0; + int key_frame = 0; + int delay = 0; + int out_size = 0; + int has_anim = 0; + + // initialize for new packet + if (!bytestream2_size(&s->gb)) { + if (s->last_pkt) + av_packet_unref(s->last_pkt); + + ret = ff_bsf_get_packet(ctx, &s->last_pkt); + if (ret < 0) + goto fail; + + bytestream2_init(&s->gb, s->last_pkt->data, s->last_pkt->size); + + av_freep(&s->last_iccp); + av_freep(&s->last_exif); + av_freep(&s->last_xmp); + + // read packet scanning for metadata && animation + while (bytestream2_get_bytes_left(&s->gb) > 0) { + chunk_type = bytestream2_get_le32(&s->gb); + chunk_size = bytestream2_get_le32(&s->gb); + + if (chunk_size == UINT32_MAX) + return AVERROR_INVALIDDATA; + chunk_size += chunk_size & 1; + + if (!bytestream2_get_bytes_left(&s->gb) || + bytestream2_get_bytes_left(&s->gb) < chunk_size) + break; + + if (chunk_type == MKTAG('R', 'I', 'F', 'F') && chunk_size > 4) { + chunk_size = 4; + } + + switch (chunk_type) { + case MKTAG('I', 'C', 'C', 'P'): + if (!s->last_iccp) { + ret = save_chunk(s, &s->last_iccp, &s->iccp_size, chunk_size); + if (ret < 0) + goto fail; + } else { + bytestream2_skip(&s->gb, chunk_size); + } + break; + + case MKTAG('E', 'X', 'I', 'F'): + if (!s->last_exif) { + ret = save_chunk(s, &s->last_exif, &s->exif_size, chunk_size); + if (ret < 0) + goto fail; + } else { + bytestream2_skip(&s->gb, chunk_size); + } + break; + + case MKTAG('X', 'M', 'P', ' '): + if (!s->last_xmp) { + ret = save_chunk(s, &s->last_xmp, &s->xmp_size, chunk_size); + if (ret < 0) + goto fail; + } else { + bytestream2_skip(&s->gb, chunk_size); + } + break; + + case MKTAG('A', 'N', 'M', 'F'): + has_anim = 1; + bytestream2_skip(&s->gb, chunk_size); + break; + + default: + bytestream2_skip(&s->gb, chunk_size); + break; + } + } + + // if no animation is found, pass-through the packet + if (!has_anim) { + av_packet_move_ref(out, s->last_pkt); + return 0; + } + + // reset bytestream to beginning of packet + bytestream2_init(&s->gb, s->last_pkt->data, s->last_pkt->size); + } + + // packet read completely, reset and ask for next packet + if (!bytestream2_get_bytes_left(&s->gb)) { + if (s->last_pkt) + av_packet_free(&s->last_pkt); + // reset to empty buffer for reinit with next real packet + bytestream2_init(&s->gb, NULL, 0); + return AVERROR(EAGAIN); + } + + // start reading from packet until sub packet ready + packet_start = bytestream2_tell(&s->gb); + s->add_iccp = 1; + s->add_exif = 1; + s->add_xmp = 1; + + while (bytestream2_get_bytes_left(&s->gb) > 0) { + chunk_type = bytestream2_get_le32(&s->gb); + chunk_size = bytestream2_get_le32(&s->gb); + + if (chunk_size == UINT32_MAX) + return AVERROR_INVALIDDATA; + chunk_size += chunk_size & 1; + + if (!bytestream2_get_bytes_left(&s->gb) || + bytestream2_get_bytes_left(&s->gb) < chunk_size) + break; + + if (chunk_type == MKTAG('R', 'I', 'F', 'F') && chunk_size > 4) { + chunk_size = 4; + key_frame = 1; + } + + switch (chunk_type) { + case MKTAG('I', 'C', 'C', 'P'): + s->add_iccp = 0; + bytestream2_skip(&s->gb, chunk_size); + break; + + case MKTAG('E', 'X', 'I', 'F'): + s->add_exif = 0; + bytestream2_skip(&s->gb, chunk_size); + break; + + case MKTAG('X', 'M', 'P', ' '): + s->add_xmp = 0; + bytestream2_skip(&s->gb, chunk_size); + break; + + case MKTAG('V', 'P', '8', ' '): + if (is_frame) { + bytestream2_seek(&s->gb, -8, SEEK_CUR); + goto flush; + } + bytestream2_skip(&s->gb, chunk_size); + is_frame = 1; + break; + + case MKTAG('V', 'P', '8', 'L'): + if (is_frame) { + bytestream2_seek(&s->gb, -8, SEEK_CUR); + goto flush; + } + bytestream2_skip(&s->gb, chunk_size); + is_frame = 1; + break; + + case MKTAG('A', 'N', 'M', 'F'): + if (is_frame) { + bytestream2_seek(&s->gb, -8, SEEK_CUR); + goto flush; + } + bytestream2_skip(&s->gb, 12); + delay = bytestream2_get_le24(&s->gb); + bytestream2_skip(&s->gb, 1); + break; + + default: + bytestream2_skip(&s->gb, chunk_size); + break; + } + + packet_end = bytestream2_tell(&s->gb); + } + +flush: + // generate packet from data read so far + out_size = packet_end - packet_start; + out_off = 0; + + if (s->add_iccp && s->last_iccp) + out_size += s->iccp_size; + if (s->add_exif && s->last_exif) + out_size += s->exif_size; + if (s->add_xmp && s->last_xmp) + out_size += s->xmp_size; + + ret = av_new_packet(out, out_size); + if (ret < 0) + goto fail; + + // copy metadata + if (s->add_iccp && s->last_iccp) { + memcpy(out->data + out_off, s->last_iccp, s->iccp_size); + out_off += s->iccp_size; + } + if (s->add_exif && s->last_exif) { + memcpy(out->data + out_off, s->last_exif, s->exif_size); + out_off += s->exif_size; + } + if (s->add_xmp && s->last_xmp) { + memcpy(out->data + out_off, s->last_xmp, s->xmp_size); + out_off += s->xmp_size; + } + + // copy frame data + memcpy(out->data + out_off, s->last_pkt->data + packet_start, packet_end - packet_start); + + if (key_frame) + out->flags |= AV_PKT_FLAG_KEY; + else + out->flags &= ~AV_PKT_FLAG_KEY; + + out->pts = s->last_pts; + out->dts = out->pts; + out->pos = packet_start; + out->duration = delay; + out->stream_index = s->last_pkt->stream_index; + out->time_base = s->last_pkt->time_base; + + s->last_pts += (delay > 0) ? delay : 1; + + key_frame = 0; + + return 0; + +fail: + if (ret < 0) { + av_packet_unref(out); + return ret; + } + av_packet_free(&in); + + return ret; +} + +static void awebp2webp_close(AVBSFContext *ctx) +{ + WEBPBSFContext *s = ctx->priv_data; + av_freep(&s->last_iccp); + av_freep(&s->last_exif); + av_freep(&s->last_xmp); +} + +static const enum AVCodecID codec_ids[] = { + AV_CODEC_ID_WEBP, AV_CODEC_ID_NONE, +}; + +const FFBitStreamFilter ff_awebp2webp_bsf = { + .p.name = "awebp2webp", + .p.codec_ids = codec_ids, + .priv_data_size = sizeof(WEBPBSFContext), + .filter = awebp2webp_filter, + .close = awebp2webp_close, +};