From patchwork Fri Jun 21 10:43:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thilo Borgmann X-Patchwork-Id: 50036 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:ae71:0:b0:482:c625:d099 with SMTP id w17csp432113vqz; Fri, 21 Jun 2024 03:44:28 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCW4Migk8Ma+G/hFZg2nVfSCFIJkf7r10sRApvLgalV4G7wVF91q+3m5+mRyYYJ/FGzcwEtAo6xzQ0K97XGc9D+gG0F1q+isXlDbQQ== X-Google-Smtp-Source: AGHT+IE6IElESc66Tpq2PioEzew6TnOVkBbbxsJCu4RuU+wMSW/wy+M2owBO8KQ249zcvJF1k+Jx X-Received: by 2002:ac2:4ecb:0:b0:52c:9421:2739 with SMTP id 2adb3069b0e04-52ccaa2a9c0mr4147752e87.9.1718966668079; Fri, 21 Jun 2024 03:44:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718966668; cv=none; d=google.com; s=arc-20160816; b=XQr6j7iw8KF5VVen8ysAb0Co0xQem5MsfNzp+VGejfkQlbB2/kciavo7WC7Bwzf0Du 7dLCDk8c8VTXlFV2XuHYaTSPyE9FFJiHkEgXCEQ9g9XN5f9mVp7xEkFMsrGhZhm/PtDx xaSERVPJEMEe75U4XZAeNrzR/O8epnsm4QVCCYwiYX4SbFn8nXWRpZn6PyHiBnFjb3sm T/t/vd/i3vVn8hBrtdjnzCg57W9MJZBzMyVXBXNufQsyefIera6XXyA5jYOtkkkOBN/H INQlDif/bYYX85f7aKf6YojzwT6eakSV2tvLa9xtQEBSKYOMVBkfFJZZpwgIzsT9oK+o Unrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:delivered-to; bh=PA2ozTsGrV9YIqumCVFirc7NaHIRr91l3xGkhyCpxT8=; fh=uZWItAgYCSLFS5Zum7Ip+PilGyWYOeCwun+We0BTHAs=; b=ZWUaNTigpxfPqQFEM2CVqsrhkNlcX57JTmQW3GMa8QFJdSuXqV/rr/wTeGqnCxtEG+ 3HOBKJeYazLP0QeuRsTbrEkvsOepQYnlJ7eTSvO9ilCTHkvtO3DrYq2toqEjKd6TlU3x M++Osi5DI3vAfkQj99ZMfNO0oY1efZSt4iW3Aiwm56TEciXq+XOV/tsSedpmtfGqjJET v8Yb/G1dlSlLjFNClS9WVZ0QxlR9doGPV2SBiRrh1RM1f1Tw2lzvZcewG27Xqh3yth3K /uZKQQZqIvgbtjhvC91r/CYLf1VHd/xUT8dQclfz123sdNzPGDA1reSp6uxtiSO9duIl VGOA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-52cd6453ecfsi357726e87.579.2024.06.21.03.44.27; Fri, 21 Jun 2024 03:44:28 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A231268D892; Fri, 21 Jun 2024 13:43:42 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from shout02.mail.de (shout02.mail.de [62.201.172.25]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1661B68D81A for ; Fri, 21 Jun 2024 13:43:31 +0300 (EEST) Received: from postfix03.mail.de (postfix03.bt.mail.de [10.0.121.127]) by shout02.mail.de (Postfix) with ESMTP id A80DE240E82 for ; Fri, 21 Jun 2024 12:43:25 +0200 (CEST) Received: from smtp01.mail.de (smtp04.bt.mail.de [10.0.121.214]) by postfix03.mail.de (Postfix) with ESMTP id 8FAA580209 for ; Fri, 21 Jun 2024 12:43:25 +0200 (CEST) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp01.mail.de (Postfix) with ESMTPSA id 43761240AD7 for ; Fri, 21 Jun 2024 12:43:25 +0200 (CEST) To: ffmpeg-devel@ffmpeg.org Date: Fri, 21 Jun 2024 12:43:18 +0200 Message-Id: <20240621104323.92453-4-thilo.borgmann@mail.de> In-Reply-To: <20240621104323.92453-1-thilo.borgmann@mail.de> References: <20240621104323.92453-1-thilo.borgmann@mail.de> MIME-Version: 1.0 X-purgate: clean X-purgate: This mail is considered clean (visit http://www.eleven.de for further information) X-purgate-type: clean X-purgate-Ad: Categorized by eleven eXpurgate (R) http://www.eleven.de X-purgate: This mail is considered clean (visit http://www.eleven.de for further information) X-purgate: clean X-purgate-size: 13444 X-purgate-ID: 154282::1718966605-E17EB338-BFA9AD90/0/0 Subject: [FFmpeg-devel] [PATCH v13 3/8] avcodec/bsf: Add awebp2webp bitstream filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Thilo Borgmann via ffmpeg-devel From: Thilo Borgmann Reply-To: FFmpeg development discussions and patches Cc: Thilo Borgmann Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: /q06qs3BJARI From: Thilo Borgmann via ffmpeg-devel Splits a packet containing a webp animations into one non-compliant packet per frame of the animation. Skips RIFF and WEBP chunks for those packets except for the first. Copyies ICC, EXIF and XMP chunks first into each of the packets except for the first. --- configure | 1 + libavcodec/bitstream_filters.c | 1 + libavcodec/bsf/Makefile | 1 + libavcodec/bsf/awebp2webp.c | 353 +++++++++++++++++++++++++++++++++ 4 files changed, 356 insertions(+) create mode 100644 libavcodec/bsf/awebp2webp.c diff --git a/configure b/configure index 3bca638459..1b6e56496f 100755 --- a/configure +++ b/configure @@ -3437,6 +3437,7 @@ aac_adtstoasc_bsf_select="adts_header mpeg4audio" av1_frame_merge_bsf_select="cbs_av1" av1_frame_split_bsf_select="cbs_av1" av1_metadata_bsf_select="cbs_av1" +awebp2webp_bsf_select="" dts2pts_bsf_select="cbs_h264 h264parse" eac3_core_bsf_select="ac3_parser" evc_frame_merge_bsf_select="evcparse" diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index 138246c50e..1f6471f4f3 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -28,6 +28,7 @@ extern const FFBitStreamFilter ff_aac_adtstoasc_bsf; extern const FFBitStreamFilter ff_av1_frame_merge_bsf; extern const FFBitStreamFilter ff_av1_frame_split_bsf; extern const FFBitStreamFilter ff_av1_metadata_bsf; +extern const FFBitStreamFilter ff_awebp2webp_bsf; extern const FFBitStreamFilter ff_chomp_bsf; extern const FFBitStreamFilter ff_dump_extradata_bsf; extern const FFBitStreamFilter ff_dca_core_bsf; diff --git a/libavcodec/bsf/Makefile b/libavcodec/bsf/Makefile index fb70ad0c21..48c67dd210 100644 --- a/libavcodec/bsf/Makefile +++ b/libavcodec/bsf/Makefile @@ -5,6 +5,7 @@ OBJS-$(CONFIG_AAC_ADTSTOASC_BSF) += bsf/aac_adtstoasc.o OBJS-$(CONFIG_AV1_FRAME_MERGE_BSF) += bsf/av1_frame_merge.o OBJS-$(CONFIG_AV1_FRAME_SPLIT_BSF) += bsf/av1_frame_split.o OBJS-$(CONFIG_AV1_METADATA_BSF) += bsf/av1_metadata.o +OBJS-$(CONFIG_AWEBP2WEBP_BSF) += bsf/awebp2webp.o OBJS-$(CONFIG_CHOMP_BSF) += bsf/chomp.o OBJS-$(CONFIG_DCA_CORE_BSF) += bsf/dca_core.o OBJS-$(CONFIG_DTS2PTS_BSF) += bsf/dts2pts.o diff --git a/libavcodec/bsf/awebp2webp.c b/libavcodec/bsf/awebp2webp.c new file mode 100644 index 0000000000..69a0156167 --- /dev/null +++ b/libavcodec/bsf/awebp2webp.c @@ -0,0 +1,353 @@ +/* + * Animated WebP into non-compliant WebP bitstream filter + * Copyright (c) 2024 Thilo Borgmann + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Animated WebP into non-compliant WebP bitstream filter + * Splits a packet containing a webp animations into + * one non-compliant packet per frame of the animation. + * Skips RIFF and WEBP chunks for those packets except + * for the first. Copyies ICC, EXIF and XMP chunks first + * into each of the packets except for the first. + * @author Thilo Borgmann + */ + +#include +#include + +#include "codec_id.h" +#include "bytestream.h" +#include "libavutil/error.h" +#include "libavutil/mem.h" + +#include "bsf.h" +#include "bsf_internal.h" +#include "packet.h" + +#define VP8X_FLAG_ANIMATION 0x02 +#define VP8X_FLAG_XMP_METADATA 0x04 +#define VP8X_FLAG_EXIF_METADATA 0x08 +#define VP8X_FLAG_ALPHA 0x10 +#define VP8X_FLAG_ICC 0x20 + +typedef struct WEBPBSFContext { + const AVClass *class; + GetByteContext gb; + + AVPacket *last_pkt; + uint8_t *last_iccp; + uint8_t *last_exif; + uint8_t *last_xmp; + + int iccp_size; + int exif_size; + int xmp_size; + + int add_iccp; + int add_exif; + int add_xmp; + + uint64_t last_pts; +} WEBPBSFContext; + +static int save_chunk(WEBPBSFContext *ctx, uint8_t **buf, int *buf_size, uint32_t chunk_size) +{ + if (*buf || !buf_size || !chunk_size) + return 0; + + *buf = av_malloc(chunk_size + 8); + if (!*buf) + return AVERROR(ENOMEM); + + *buf_size = chunk_size + 8; + + bytestream2_seek(&ctx->gb, -8, SEEK_CUR); + bytestream2_get_buffer(&ctx->gb, *buf, chunk_size + 8); + + return 0; +} + +static int awebp2webp_filter(AVBSFContext *ctx, AVPacket *out) +{ + WEBPBSFContext *s = ctx->priv_data; + AVPacket *in; + uint32_t chunk_type; + uint32_t chunk_size; + int64_t packet_start; + int64_t packet_end; + int64_t out_off; + int ret = 0; + int is_frame = 0; + int key_frame = 0; + int delay = 0; + int out_size = 0; + int has_anim = 0; + + // initialize for new packet + if (!bytestream2_size(&s->gb)) { + if (s->last_pkt) + av_packet_unref(s->last_pkt); + + ret = ff_bsf_get_packet(ctx, &s->last_pkt); + if (ret < 0) + goto fail; + + bytestream2_init(&s->gb, s->last_pkt->data, s->last_pkt->size); + + av_freep(&s->last_iccp); + av_freep(&s->last_exif); + av_freep(&s->last_xmp); + + // read packet scanning for metadata && animation + while (bytestream2_get_bytes_left(&s->gb) > 0) { + chunk_type = bytestream2_get_le32(&s->gb); + chunk_size = bytestream2_get_le32(&s->gb); + + if (chunk_size == UINT32_MAX) + return AVERROR_INVALIDDATA; + chunk_size += chunk_size & 1; + + if (!bytestream2_get_bytes_left(&s->gb) || + bytestream2_get_bytes_left(&s->gb) < chunk_size) + break; + + if (chunk_type == MKTAG('R', 'I', 'F', 'F') && chunk_size > 4) { + chunk_size = 4; + } + + switch (chunk_type) { + case MKTAG('I', 'C', 'C', 'P'): + if (!s->last_iccp) { + ret = save_chunk(s, &s->last_iccp, &s->iccp_size, chunk_size); + if (ret < 0) + goto fail; + } else { + bytestream2_skip(&s->gb, chunk_size); + } + break; + + case MKTAG('E', 'X', 'I', 'F'): + if (!s->last_exif) { + ret = save_chunk(s, &s->last_exif, &s->exif_size, chunk_size); + if (ret < 0) + goto fail; + } else { + bytestream2_skip(&s->gb, chunk_size); + } + break; + + case MKTAG('X', 'M', 'P', ' '): + if (!s->last_xmp) { + ret = save_chunk(s, &s->last_xmp, &s->xmp_size, chunk_size); + if (ret < 0) + goto fail; + } else { + bytestream2_skip(&s->gb, chunk_size); + } + break; + + case MKTAG('A', 'N', 'M', 'F'): + has_anim = 1; + bytestream2_skip(&s->gb, chunk_size); + break; + + default: + bytestream2_skip(&s->gb, chunk_size); + break; + } + } + + // if no animation is found, pass-through the packet + if (!has_anim) { + av_packet_move_ref(out, s->last_pkt); + return 0; + } + + // reset bytestream to beginning of packet + bytestream2_init(&s->gb, s->last_pkt->data, s->last_pkt->size); + } + + // packet read completely, reset and ask for next packet + if (!bytestream2_get_bytes_left(&s->gb)) { + if (s->last_pkt) + av_packet_free(&s->last_pkt); + // reset to empty buffer for reinit with next real packet + bytestream2_init(&s->gb, NULL, 0); + return AVERROR(EAGAIN); + } + + // start reading from packet until sub packet ready + packet_start = bytestream2_tell(&s->gb); + s->add_iccp = 1; + s->add_exif = 1; + s->add_xmp = 1; + + while (bytestream2_get_bytes_left(&s->gb) > 0) { + chunk_type = bytestream2_get_le32(&s->gb); + chunk_size = bytestream2_get_le32(&s->gb); + + if (chunk_size == UINT32_MAX) + return AVERROR_INVALIDDATA; + chunk_size += chunk_size & 1; + + if (!bytestream2_get_bytes_left(&s->gb) || + bytestream2_get_bytes_left(&s->gb) < chunk_size) + break; + + if (chunk_type == MKTAG('R', 'I', 'F', 'F') && chunk_size > 4) { + chunk_size = 4; + key_frame = 1; + } + + switch (chunk_type) { + case MKTAG('I', 'C', 'C', 'P'): + s->add_iccp = 0; + bytestream2_skip(&s->gb, chunk_size); + break; + + case MKTAG('E', 'X', 'I', 'F'): + s->add_exif = 0; + bytestream2_skip(&s->gb, chunk_size); + break; + + case MKTAG('X', 'M', 'P', ' '): + s->add_xmp = 0; + bytestream2_skip(&s->gb, chunk_size); + break; + + case MKTAG('V', 'P', '8', ' '): + if (is_frame) { + bytestream2_seek(&s->gb, -8, SEEK_CUR); + goto flush; + } + bytestream2_skip(&s->gb, chunk_size); + is_frame = 1; + break; + + case MKTAG('V', 'P', '8', 'L'): + if (is_frame) { + bytestream2_seek(&s->gb, -8, SEEK_CUR); + goto flush; + } + bytestream2_skip(&s->gb, chunk_size); + is_frame = 1; + break; + + case MKTAG('A', 'N', 'M', 'F'): + if (is_frame) { + bytestream2_seek(&s->gb, -8, SEEK_CUR); + goto flush; + } + bytestream2_skip(&s->gb, 12); + delay = bytestream2_get_le24(&s->gb); + if (!delay) + delay = s->last_pkt->duration; + bytestream2_skip(&s->gb, 1); + break; + + default: + bytestream2_skip(&s->gb, chunk_size); + break; + } + + packet_end = bytestream2_tell(&s->gb); + } + +flush: + // generate packet from data read so far + out_size = packet_end - packet_start; + out_off = 0; + + if (s->add_iccp && s->last_iccp) + out_size += s->iccp_size; + if (s->add_exif && s->last_exif) + out_size += s->exif_size; + if (s->add_xmp && s->last_xmp) + out_size += s->xmp_size; + + ret = av_new_packet(out, out_size); + if (ret < 0) + goto fail; + + // copy metadata + if (s->add_iccp && s->last_iccp) { + memcpy(out->data + out_off, s->last_iccp, s->iccp_size); + out_off += s->iccp_size; + } + if (s->add_exif && s->last_exif) { + memcpy(out->data + out_off, s->last_exif, s->exif_size); + out_off += s->exif_size; + } + if (s->add_xmp && s->last_xmp) { + memcpy(out->data + out_off, s->last_xmp, s->xmp_size); + out_off += s->xmp_size; + } + + // copy frame data + memcpy(out->data + out_off, s->last_pkt->data + packet_start, packet_end - packet_start); + + if (key_frame) + out->flags |= AV_PKT_FLAG_KEY; + else + out->flags &= ~AV_PKT_FLAG_KEY; + + out->pts = s->last_pts; + out->dts = out->pts; + out->pos = packet_start; + out->duration = delay; + out->stream_index = s->last_pkt->stream_index; + out->time_base = s->last_pkt->time_base; + + s->last_pts += (delay > 0) ? delay : 1; + + key_frame = 0; + + return 0; + +fail: + if (ret < 0) { + av_packet_unref(out); + return ret; + } + av_packet_free(&in); + + return ret; +} + +static void awebp2webp_close(AVBSFContext *ctx) +{ + WEBPBSFContext *s = ctx->priv_data; + av_freep(&s->last_iccp); + av_freep(&s->last_exif); + av_freep(&s->last_xmp); +} + +static const enum AVCodecID codec_ids[] = { + AV_CODEC_ID_WEBP, AV_CODEC_ID_NONE, +}; + +const FFBitStreamFilter ff_awebp2webp_bsf = { + .p.name = "awebp2webp", + .p.codec_ids = codec_ids, + .priv_data_size = sizeof(WEBPBSFContext), + .filter = awebp2webp_filter, + .close = awebp2webp_close, +};