From patchwork Thu Mar 28 14:08:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thilo Borgmann X-Patchwork-Id: 47585 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9f96:b0:1a3:b6bb:3029 with SMTP id mm22csp1349797pzb; Thu, 28 Mar 2024 07:09:53 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCW8AGPjEk/M17Nq0FyCa6o81BaY9hFrakyA1jL2H83cbjsRle8j8+DuZ6qu4hb7ZSiNIHzEHIhIcV+fX4DdVSIPJ8D+Vxh3Nfom2A== X-Google-Smtp-Source: AGHT+IHq5zM8koSmF6tBSpotQ8w08zH8yYloS7MYrrjl+8AMfrr78Mhy+lIPNl9WTbwD7Fm4T8TR X-Received: by 2002:a50:cd18:0:b0:568:32f7:6c53 with SMTP id z24-20020a50cd18000000b0056832f76c53mr2233045edi.3.1711634992973; Thu, 28 Mar 2024 07:09:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1711634992; cv=none; d=google.com; s=arc-20160816; b=bGaFqh+ltgzhC4LegbN4f0za3c1SHQetd/Y71EP+MPDiIHbQxdXiRiBIxuXqtDDoqe uB28kSf5034HF/E49hx/+lFym0+dlUZBT6vhp9Pc0cjrm4m1PoApvILDITmqx5RuYZaM rBCFATM1/J28vyn7ngKAt4Ve3URaTBzQ10l0oKsqafUVKjXx5JfbY+WGMsrvswZPhWzf X0be+G6NXkNxCLnebPeAdiHns3htz/OlC5n6su5Yo7suEBjDogfgVpu7zMoubvnk900k hjeL3xHR7WfKK79zIFQj9RVUUJASrBBRqz4TehUOSfxCQyI/uxsbbUQxFPdRO6kHtrcB exLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:delivered-to; bh=MA6k/09SWmmLpkdNQpxd1mmHqpsSsOm56kOjkReKhiE=; fh=uZWItAgYCSLFS5Zum7Ip+PilGyWYOeCwun+We0BTHAs=; b=K5GobNo0H21lK7acLnDsgzLZFUd8s8MZead5qnDf4kDr8ZjLotKKy7EF/+Ww4m90im 8g7HptwqJmcSlBZMS4qIAKKz7HVvp9oVXEFP1qcS5DPZ6d3TnB0De0EONIVdphoA9OFS 96ApMqVhl4UY5K2Yr8cZo5Cjk/KtkMyp8A6pG+42hflj29N94XsLM3Jvv8u2ms+mamFS FxBKl7Xhi+2uYI1jvXZPVqsy6CXxAKEKb2Qb7q6G4Nm2qNtesJvzO6vI8S3EunsUEczm 7FjDSITAbHgQQaSP6N37F8OntSmWWRSeOZk6/aibyJ0rgfrh096oIVlxXlvwl/O9bF2U tQcg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v28-20020a50955c000000b00568a7528b6dsi819559eda.32.2024.03.28.07.09.52; Thu, 28 Mar 2024 07:09:52 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E4C0A68D71C; Thu, 28 Mar 2024 16:09:13 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from shout02.mail.de (shout02.mail.de [62.201.172.25]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E25CB68D217 for ; Thu, 28 Mar 2024 16:09:05 +0200 (EET) Received: from postfix02.mail.de (postfix02.bt.mail.de [10.0.121.126]) by shout02.mail.de (Postfix) with ESMTP id C1098240DF9 for ; Thu, 28 Mar 2024 15:08:59 +0100 (CET) Received: from smtp01.mail.de (smtp02.bt.mail.de [10.0.121.212]) by postfix02.mail.de (Postfix) with ESMTP id 923C3A03C6 for ; Thu, 28 Mar 2024 15:08:59 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp01.mail.de (Postfix) with ESMTPSA id 625F2240CF5 for ; Thu, 28 Mar 2024 15:08:59 +0100 (CET) To: ffmpeg-devel@ffmpeg.org Date: Thu, 28 Mar 2024 15:08:53 +0100 Message-Id: <20240328140858.2521-4-thilo.borgmann@mail.de> In-Reply-To: <20240328140858.2521-1-thilo.borgmann@mail.de> References: <20240328140858.2521-1-thilo.borgmann@mail.de> MIME-Version: 1.0 X-purgate: clean X-purgate: This mail is considered clean (visit http://www.eleven.de for further information) X-purgate-type: clean X-purgate-Ad: Categorized by eleven eXpurgate (R) http://www.eleven.de X-purgate: This mail is considered clean (visit http://www.eleven.de for further information) X-purgate: clean X-purgate-size: 13273 X-purgate-ID: 154282::1711634939-FED2B878-934FB484/0/0 Subject: [FFmpeg-devel] [PATCH v11 3/8] avcodec/bsf: Add awebp2webp bitstream filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Thilo Borgmann via ffmpeg-devel From: Thilo Borgmann Reply-To: FFmpeg development discussions and patches Cc: Thilo Borgmann Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: HCyYAshcRPWu Splits a packet containing a webp animations into one non-compliant packet per frame of the animation. Skips RIFF and WEBP chunks for those packets except for the first. Copyies ICC, EXIF and XMP chunks first into each of the packets except for the first. --- configure | 1 + libavcodec/bitstream_filters.c | 1 + libavcodec/bsf/Makefile | 1 + libavcodec/bsf/awebp2webp.c | 350 +++++++++++++++++++++++++++++++++ 4 files changed, 353 insertions(+) create mode 100644 libavcodec/bsf/awebp2webp.c diff --git a/configure b/configure index 2d46ef0b9c..91813a41cd 100755 --- a/configure +++ b/configure @@ -3418,6 +3418,7 @@ aac_adtstoasc_bsf_select="adts_header mpeg4audio" av1_frame_merge_bsf_select="cbs_av1" av1_frame_split_bsf_select="cbs_av1" av1_metadata_bsf_select="cbs_av1" +awebp2webp_bsf_select="" dts2pts_bsf_select="cbs_h264 h264parse" eac3_core_bsf_select="ac3_parser" evc_frame_merge_bsf_select="evcparse" diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index 12860c332b..af88283a8c 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -28,6 +28,7 @@ extern const FFBitStreamFilter ff_aac_adtstoasc_bsf; extern const FFBitStreamFilter ff_av1_frame_merge_bsf; extern const FFBitStreamFilter ff_av1_frame_split_bsf; extern const FFBitStreamFilter ff_av1_metadata_bsf; +extern const FFBitStreamFilter ff_awebp2webp_bsf; extern const FFBitStreamFilter ff_chomp_bsf; extern const FFBitStreamFilter ff_dump_extradata_bsf; extern const FFBitStreamFilter ff_dca_core_bsf; diff --git a/libavcodec/bsf/Makefile b/libavcodec/bsf/Makefile index fb70ad0c21..48c67dd210 100644 --- a/libavcodec/bsf/Makefile +++ b/libavcodec/bsf/Makefile @@ -5,6 +5,7 @@ OBJS-$(CONFIG_AAC_ADTSTOASC_BSF) += bsf/aac_adtstoasc.o OBJS-$(CONFIG_AV1_FRAME_MERGE_BSF) += bsf/av1_frame_merge.o OBJS-$(CONFIG_AV1_FRAME_SPLIT_BSF) += bsf/av1_frame_split.o OBJS-$(CONFIG_AV1_METADATA_BSF) += bsf/av1_metadata.o +OBJS-$(CONFIG_AWEBP2WEBP_BSF) += bsf/awebp2webp.o OBJS-$(CONFIG_CHOMP_BSF) += bsf/chomp.o OBJS-$(CONFIG_DCA_CORE_BSF) += bsf/dca_core.o OBJS-$(CONFIG_DTS2PTS_BSF) += bsf/dts2pts.o diff --git a/libavcodec/bsf/awebp2webp.c b/libavcodec/bsf/awebp2webp.c new file mode 100644 index 0000000000..ebd123c667 --- /dev/null +++ b/libavcodec/bsf/awebp2webp.c @@ -0,0 +1,350 @@ +/* + * Animated WebP into non-compliant WebP bitstream filter + * Copyright (c) 2024 Thilo Borgmann + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Animated WebP into non-compliant WebP bitstream filter + * Splits a packet containing a webp animations into + * one non-compliant packet per frame of the animation. + * Skips RIFF and WEBP chunks for those packets except + * for the first. Copyies ICC, EXIF and XMP chunks first + * into each of the packets except for the first. + * @author Thilo Borgmann + */ + +#include +#include + +#include "codec_id.h" +#include "bytestream.h" +#include "libavutil/error.h" + +#include "bsf.h" +#include "bsf_internal.h" +#include "packet.h" + +#define VP8X_FLAG_ANIMATION 0x02 +#define VP8X_FLAG_XMP_METADATA 0x04 +#define VP8X_FLAG_EXIF_METADATA 0x08 +#define VP8X_FLAG_ALPHA 0x10 +#define VP8X_FLAG_ICC 0x20 + +typedef struct WEBPBSFContext { + const AVClass *class; + GetByteContext gb; + + AVPacket *last_pkt; + uint8_t *last_iccp; + uint8_t *last_exif; + uint8_t *last_xmp; + + int iccp_size; + int exif_size; + int xmp_size; + + int add_iccp; + int add_exif; + int add_xmp; + + uint64_t last_pts; +} WEBPBSFContext; + +static int save_chunk(WEBPBSFContext *ctx, uint8_t **buf, int *buf_size, uint32_t chunk_size) +{ + if (*buf || !buf_size || !chunk_size) + return 0; + + *buf = av_malloc(chunk_size + 8); + if (!*buf) + return AVERROR(ENOMEM); + + *buf_size = chunk_size + 8; + + bytestream2_seek(&ctx->gb, -8, SEEK_CUR); + bytestream2_get_buffer(&ctx->gb, *buf, chunk_size + 8); + + return 0; +} + +static int awebp2webp_filter(AVBSFContext *ctx, AVPacket *out) +{ + WEBPBSFContext *s = ctx->priv_data; + AVPacket *in; + uint32_t chunk_type; + uint32_t chunk_size; + int64_t packet_start; + int64_t packet_end; + int64_t out_off; + int ret = 0; + int is_frame = 0; + int key_frame = 0; + int delay = 0; + int out_size = 0; + int has_anim = 0; + + // initialize for new packet + if (!bytestream2_size(&s->gb)) { + if (s->last_pkt) + av_packet_unref(s->last_pkt); + + ret = ff_bsf_get_packet(ctx, &s->last_pkt); + if (ret < 0) + goto fail; + + bytestream2_init(&s->gb, s->last_pkt->data, s->last_pkt->size); + + av_freep(&s->last_iccp); + av_freep(&s->last_exif); + av_freep(&s->last_xmp); + + // read packet scanning for metadata && animation + while (bytestream2_get_bytes_left(&s->gb) > 0) { + chunk_type = bytestream2_get_le32(&s->gb); + chunk_size = bytestream2_get_le32(&s->gb); + + if (chunk_size == UINT32_MAX) + return AVERROR_INVALIDDATA; + chunk_size += chunk_size & 1; + + if (!bytestream2_get_bytes_left(&s->gb) || + bytestream2_get_bytes_left(&s->gb) < chunk_size) + break; + + if (chunk_type == MKTAG('R', 'I', 'F', 'F') && chunk_size > 4) { + chunk_size = 4; + } + + switch (chunk_type) { + case MKTAG('I', 'C', 'C', 'P'): + if (!s->last_iccp) { + ret = save_chunk(s, &s->last_iccp, &s->iccp_size, chunk_size); + if (ret < 0) + goto fail; + } else { + bytestream2_skip(&s->gb, chunk_size); + } + break; + + case MKTAG('E', 'X', 'I', 'F'): + if (!s->last_exif) { + ret = save_chunk(s, &s->last_exif, &s->exif_size, chunk_size); + if (ret < 0) + goto fail; + } else { + bytestream2_skip(&s->gb, chunk_size); + } + break; + + case MKTAG('X', 'M', 'P', ' '): + if (!s->last_xmp) { + ret = save_chunk(s, &s->last_xmp, &s->xmp_size, chunk_size); + if (ret < 0) + goto fail; + } else { + bytestream2_skip(&s->gb, chunk_size); + } + break; + + case MKTAG('A', 'N', 'M', 'F'): + has_anim = 1; + bytestream2_skip(&s->gb, chunk_size); + break; + + default: + bytestream2_skip(&s->gb, chunk_size); + break; + } + } + + // if no animation is found, pass-through the packet + if (!has_anim) { + av_packet_move_ref(out, s->last_pkt); + return 0; + } + + // reset bytestream to beginning of packet + bytestream2_init(&s->gb, s->last_pkt->data, s->last_pkt->size); + } + + // packet read completely, reset and ask for next packet + if (!bytestream2_get_bytes_left(&s->gb)) { + if (s->last_pkt) + av_packet_free(&s->last_pkt); + // reset to empty buffer for reinit with next real packet + bytestream2_init(&s->gb, NULL, 0); + return AVERROR(EAGAIN); + } + + // start reading from packet until sub packet ready + packet_start = bytestream2_tell(&s->gb); + s->add_iccp = 1; + s->add_exif = 1; + s->add_xmp = 1; + + while (bytestream2_get_bytes_left(&s->gb) > 0) { + chunk_type = bytestream2_get_le32(&s->gb); + chunk_size = bytestream2_get_le32(&s->gb); + + if (chunk_size == UINT32_MAX) + return AVERROR_INVALIDDATA; + chunk_size += chunk_size & 1; + + if (!bytestream2_get_bytes_left(&s->gb) || + bytestream2_get_bytes_left(&s->gb) < chunk_size) + break; + + if (chunk_type == MKTAG('R', 'I', 'F', 'F') && chunk_size > 4) { + chunk_size = 4; + key_frame = 1; + } + + switch (chunk_type) { + case MKTAG('I', 'C', 'C', 'P'): + s->add_iccp = 0; + bytestream2_skip(&s->gb, chunk_size); + break; + + case MKTAG('E', 'X', 'I', 'F'): + s->add_exif = 0; + bytestream2_skip(&s->gb, chunk_size); + break; + + case MKTAG('X', 'M', 'P', ' '): + s->add_xmp = 0; + bytestream2_skip(&s->gb, chunk_size); + break; + + case MKTAG('V', 'P', '8', ' '): + if (is_frame) { + bytestream2_seek(&s->gb, -8, SEEK_CUR); + goto flush; + } + bytestream2_skip(&s->gb, chunk_size); + is_frame = 1; + break; + + case MKTAG('V', 'P', '8', 'L'): + if (is_frame) { + bytestream2_seek(&s->gb, -8, SEEK_CUR); + goto flush; + } + bytestream2_skip(&s->gb, chunk_size); + is_frame = 1; + break; + + case MKTAG('A', 'N', 'M', 'F'): + if (is_frame) { + bytestream2_seek(&s->gb, -8, SEEK_CUR); + goto flush; + } + bytestream2_skip(&s->gb, 12); + delay = bytestream2_get_le24(&s->gb); + bytestream2_skip(&s->gb, 1); + break; + + default: + bytestream2_skip(&s->gb, chunk_size); + break; + } + + packet_end = bytestream2_tell(&s->gb); + } + +flush: + // generate packet from data read so far + out_size = packet_end - packet_start; + out_off = 0; + + if (s->add_iccp && s->last_iccp) + out_size += s->iccp_size; + if (s->add_exif && s->last_exif) + out_size += s->exif_size; + if (s->add_xmp && s->last_xmp) + out_size += s->xmp_size; + + ret = av_new_packet(out, out_size); + if (ret < 0) + goto fail; + + // copy metadata + if (s->add_iccp && s->last_iccp) { + memcpy(out->data + out_off, s->last_iccp, s->iccp_size); + out_off += s->iccp_size; + } + if (s->add_exif && s->last_exif) { + memcpy(out->data + out_off, s->last_exif, s->exif_size); + out_off += s->exif_size; + } + if (s->add_xmp && s->last_xmp) { + memcpy(out->data + out_off, s->last_xmp, s->xmp_size); + out_off += s->xmp_size; + } + + // copy frame data + memcpy(out->data + out_off, s->last_pkt->data + packet_start, packet_end - packet_start); + + if (key_frame) + out->flags |= AV_PKT_FLAG_KEY; + else + out->flags &= ~AV_PKT_FLAG_KEY; + + out->pts = s->last_pts; + out->dts = out->pts; + out->pos = packet_start; + out->duration = delay; + out->stream_index = s->last_pkt->stream_index; + out->time_base = s->last_pkt->time_base; + + s->last_pts += (delay > 0) ? delay : 1; + + key_frame = 0; + + return 0; + +fail: + if (ret < 0) { + av_packet_unref(out); + return ret; + } + av_packet_free(&in); + + return ret; +} + +static void awebp2webp_close(AVBSFContext *ctx) +{ + WEBPBSFContext *s = ctx->priv_data; + av_freep(&s->last_iccp); + av_freep(&s->last_exif); + av_freep(&s->last_xmp); +} + +static const enum AVCodecID codec_ids[] = { + AV_CODEC_ID_WEBP, AV_CODEC_ID_NONE, +}; + +const FFBitStreamFilter ff_awebp2webp_bsf = { + .p.name = "awebp2webp", + .p.codec_ids = codec_ids, + .priv_data_size = sizeof(WEBPBSFContext), + .filter = awebp2webp_filter, + .close = awebp2webp_close, +};