From patchwork Tue Sep 24 14:43:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51814 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:9548:0:b0:48e:c0f8:d0de with SMTP id t8csp207472vqk; Tue, 24 Sep 2024 20:39:18 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWCH1Hk16NmpnYGn/BjIJAW8BlC/Zwqx1iRrqk8RKt8flmII20DT82FksgBg3031tiQ/TPmRfIp/rUe+F0pqpIB@gmail.com X-Google-Smtp-Source: AGHT+IHMX2Yxzjh5bajK97bDe6HjWOaBUW1xk8RwU/XlIE6mRznX6FjiD0PaNkHw2bOwDaskrT50 X-Received: by 2002:a2e:a9a4:0:b0:2f7:586d:e5e7 with SMTP id 38308e7fff4ca-2f8d0b586eemr17783131fa.5.1727235557863; Tue, 24 Sep 2024 20:39:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1727235557; cv=none; d=google.com; s=arc-20240605; b=fY6FV8fjF+bMdGzJ68bVnUX+rLSPrc/9rs/dMyGeMPigs8CVOHFd/snaSWpefIIBVJ tsYFjoBTH+yKPzvACWr3JuaAm9HpDWmwOExicH789IdT25K+wVodNj3xIfCzp3Ndsgsw +NFzhTdO/kZhojhHTOx+SJxN0Ihin6nu5GR+7rn5nipEdbpjz6aYYCuot2OiZ019T1IW F1ghkVK4bmg6b/Tr8PHKEnqthDossea3JnFbnRzAi7zEHAf/hi3ELrlqV6o870UIl0A9 6m1z41IM1pWdkKt6rl7Cl5s13GzzxjkbMlRtJe4AxuSCDILVzaOtbDEdYmh7epA1rFO4 +pxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=8Q9YEK/A/hJ00L8y88LmhCyomG6m2NYMUp9Rt/0J4zY=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=Ffk+FDim+7QKDAugHOBdyzZRysK7gIOWNCi/7KkBoeGNpZKXvvTmMux1Z2prAiS0Ho ePRIPBDLJzxdVwZixc6gEeUO1DZvcChn1EU2P4wpOTmP8CdgaLtUI7fazMFFArCdovm7 CtvB8S1rPs5kOWy/yxzWpyjPURNN11sFJNfgTwLsBGc5urFbVF/WETe2IvNgwJKnxwu4 1BPNa9aUBEiZ9NgFaegOm0c+0Ve03aLwXHIMXDz7Eo0HjUZmOhGMA6c3Qfh4o9ig2P86 W/ii2+SEIkFNNGh8V3qX3bNl2E7nAWVvK7KDQ3OS+raO3oow/lOR9Vr9G9z52ZCMFte3 mfkA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=TwuIbrTk; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2f8d28b67a4si8486411fa.521.2024.09.24.20.39.17; Tue, 24 Sep 2024 20:39:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=TwuIbrTk; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 260DE68DBB0; Tue, 24 Sep 2024 17:42:53 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oo1-f50.google.com (mail-oo1-f50.google.com [209.85.161.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D412668D73B for ; Tue, 24 Sep 2024 17:42:46 +0300 (EEST) Received: by mail-oo1-f50.google.com with SMTP id 006d021491bc7-5e58237a1aaso2654305eaf.3 for ; Tue, 24 Sep 2024 07:42:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727188964; x=1727793764; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=hshZ6oo5r2MovQCL28s0a2c3YFoM24f+/V0rUaPiDt8=; b=TwuIbrTk4ILHdrhPaPqyywxdznTNLrRpxlteQHkoQQNuSod3YQPBZQRy014A/VqP77 Z0LT6L7dlWVRo/fykEtqHEweGj+QooT5U2FYjRH6X5PHLRbsk+8J51xa+ip3ncKW8vJK Ym+0Xk4uY0cZFVFbCpcDzh9TcLna4JS++eVDIQ40eS6AF5V4pGaa1Nw5tx8KG/HRBLgA uE/i7CCdjHxbK/wAYXUOsLsrt+r8vwz6/Z4q2roA0dBnrZjN3YoA2Tq343gNtOC875gQ 0sD9zxiwUjPiKcn0mHtYyZKPvgKp7WdC3mC25tTblpX3StqhfKCL82SpAMFDGA209cF0 9rlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727188964; x=1727793764; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hshZ6oo5r2MovQCL28s0a2c3YFoM24f+/V0rUaPiDt8=; b=j5a0rGdSQH/dOv66NVoQjF1YfGX0M4xa3Tl1agRFqqWIhfELX0ODWoyolbwut8nNHB aCMIhlCzah0+ipZHN5k3SAvc7SGx+Hhoo9KlodcjPrNBjQ07rKf2aNCw/m2okZrbxWU7 zxgS799j/lKwqEwh2wAHw5ONoeh458SBUsPN+CAS+B/fsoxSKyZVK1vrs6qWKk5xytdF sENavHC4YwgUWUfePf1dfsg9UIUJ4it+bOoVMHIVOocTSWecBHmFomBPRA/NPyBhAvFA 19lmXXzMXQ6eJBzQKKc/hivKUBaArwHhEIiedRuHb87uwRpbH0+BVvc2dd55ZLX9OQbQ NAzQ== X-Gm-Message-State: AOJu0YzUALuWNI+YNKu2Cp9pR7mN7L2fb4sZ8Pl8dsBz5lYoXQKNWprU mXjVD42mNstHlt/erwj6vHzEe+kb62nsSQ94OKZI4FwaSNs47RfjUoC9YQ== X-Received: by 2002:a05:6358:2809:b0:1b8:40e5:4237 with SMTP id e5c5f4694b2df-1bc975d468amr657282455d.10.1727188964423; Tue, 24 Sep 2024 07:42:44 -0700 (PDT) Received: from localhost.localdomain ([181.92.233.116]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cb0f4c1b4fsm7152716d6.43.2024.09.24.07.42.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Sep 2024 07:42:43 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Tue, 24 Sep 2024 11:43:06 -0300 Message-ID: <20240924144308.1196-1-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/3] avcodec: add an LCEVC merger bsf X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: W4shfvLF/WNK Signed-off-by: James Almer --- libavcodec/bitstream_filters.c | 1 + libavcodec/bsf/Makefile | 1 + libavcodec/bsf/lcevc_merge_bsf.c | 273 +++++++++++++++++++++++++++++++ 3 files changed, 275 insertions(+) create mode 100644 libavcodec/bsf/lcevc_merge_bsf.c diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index f923411bee..fdd4fcf01b 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -45,6 +45,7 @@ extern const FFBitStreamFilter ff_hapqa_extract_bsf; extern const FFBitStreamFilter ff_hevc_metadata_bsf; extern const FFBitStreamFilter ff_hevc_mp4toannexb_bsf; extern const FFBitStreamFilter ff_imx_dump_header_bsf; +extern const FFBitStreamFilter ff_lcevc_merge_bsf; extern const FFBitStreamFilter ff_media100_to_mjpegb_bsf; extern const FFBitStreamFilter ff_mjpeg2jpeg_bsf; extern const FFBitStreamFilter ff_mjpega_dump_header_bsf; diff --git a/libavcodec/bsf/Makefile b/libavcodec/bsf/Makefile index 40b7fc6e9b..3d28da206b 100644 --- a/libavcodec/bsf/Makefile +++ b/libavcodec/bsf/Makefile @@ -22,6 +22,7 @@ OBJS-$(CONFIG_HEVC_METADATA_BSF) += bsf/h265_metadata.o OBJS-$(CONFIG_DOVI_RPU_BSF) += bsf/dovi_rpu.o OBJS-$(CONFIG_HEVC_MP4TOANNEXB_BSF) += bsf/hevc_mp4toannexb.o OBJS-$(CONFIG_IMX_DUMP_HEADER_BSF) += bsf/imx_dump_header.o +OBJS-$(CONFIG_LCEVC_MERGE_BSF) += bsf/lcevc_merge_bsf.o OBJS-$(CONFIG_MEDIA100_TO_MJPEGB_BSF) += bsf/media100_to_mjpegb.o OBJS-$(CONFIG_MJPEG2JPEG_BSF) += bsf/mjpeg2jpeg.o OBJS-$(CONFIG_MJPEGA_DUMP_HEADER_BSF) += bsf/mjpega_dump_header.o diff --git a/libavcodec/bsf/lcevc_merge_bsf.c b/libavcodec/bsf/lcevc_merge_bsf.c new file mode 100644 index 0000000000..269597bdf0 --- /dev/null +++ b/libavcodec/bsf/lcevc_merge_bsf.c @@ -0,0 +1,273 @@ +/* + * Copyright (c) 2024 James Almer + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/avassert.h" +#include "libavutil/attributes.h" +#include "libavutil/mem.h" +#include "libavutil/opt.h" +#include "libavutil/tree.h" + +#include "bsf.h" +#include "bsf_internal.h" +#include "container_fifo.h" + +typedef struct LCEVCMergeContext { + const AVClass *class; + + struct AVTreeNode *base; + struct AVTreeNode *enhancement; + ContainerFifo *nodes; + + int base_idx, enhancement_idx; +} LCEVCMergeContext; + +// AVTreeNode callbacks +static int cmp_insert(const void *_key, const void *_node) +{ + const AVPacket *key = _key, *node = _node; + + return FFDIFFSIGN(key->pts, node->pts); +} + +static int cmp_find(const void *_key, const void *_node) +{ + int64_t key = *(const int64_t *)_key; + const AVPacket *node = _node; + + return FFDIFFSIGN(key, node->pts); +} + +#define WARN_BUFFERED(type) \ +static int warn_##type##_buffered(void *logctx, void *elem) \ +{ \ + const AVPacket *pkt = (const AVPacket *)elem; \ + av_log(logctx, AV_LOG_WARNING, #type" packet with PTS %"PRId64 \ + " left buffered at EOF\n", pkt->pts); \ + return 0; \ +} + +WARN_BUFFERED(base) +WARN_BUFFERED(enhanced) + +static void *node_alloc(void) +{ + return av_tree_node_alloc(); +} + +static void node_reset(void *obj) +{ + memset(obj, 0, av_tree_node_size); +} + +static void node_free(void *obj) +{ + av_free(obj); +} + +static int node_write(void *dst, void *src) +{ + memcpy(dst, &src, sizeof(src)); + return 0; +} + +static int lcevc_merge_init(AVBSFContext *ctx) +{ + LCEVCMergeContext *s = ctx->priv_data; + + if (s->base_idx < 0 || s->enhancement_idx < 0) { + av_log(ctx, AV_LOG_ERROR, "Both base and enhancement stream index must be set\n"); + return AVERROR(EINVAL); + } + + s->nodes = ff_container_fifo_alloc(node_alloc, node_reset, node_free, node_write, node_write); + if (!s->nodes) + return AVERROR(ENOMEM); + + return 0; +} + +static int merge_packet(AVBSFContext *ctx, struct AVTreeNode **root, + AVPacket *out, AVPacket *in) +{ + LCEVCMergeContext *s = ctx->priv_data; + struct AVTreeNode *node = NULL; + uint8_t *side_data; + int ret; + + // It doesn't matter if the packet is from the base or enhancement stream + // as both share the pts cmp_insert() will look for to remove the element. + av_tree_insert(root, in, cmp_insert, &node); + ret = ff_container_fifo_write(s->nodes, node); + if (ret < 0) { + av_free(node); + return ret; + } + + side_data = av_packet_new_side_data(out, AV_PKT_DATA_LCEVC, in->size); + if (!side_data) + return AVERROR(ENOMEM); + + memcpy(side_data, in->data, in->size); + + return 0; +} + +static int buffer_packet(AVBSFContext *ctx, struct AVTreeNode **root, + AVPacket **p_in) +{ + LCEVCMergeContext *s = ctx->priv_data; + AVPacket *in = *p_in, *pkt; + struct AVTreeNode *node = NULL; + + if (ff_container_fifo_can_read(s->nodes)) + ff_container_fifo_read(s->nodes, &node); + else + node = av_tree_node_alloc(); + if (!node) + return AVERROR(ENOMEM); + + pkt = av_tree_insert(root, in, cmp_insert, &node); + if (pkt && pkt != in) { + av_log(ctx, AV_LOG_ERROR, "Duplicate packet with PTS %"PRId64 + " for stream_index %d \n", in->pts, in->stream_index); + av_free(node); + return AVERROR_INVALIDDATA; + } + *p_in = NULL; + + return 0; +} + +#define HANDLE_PACKET(type1, type2, pkt1, pkt2) \ +static int handle_##type1##_packet(AVBSFContext *ctx, AVPacket *out, AVPacket **p_in) \ +{ \ + LCEVCMergeContext *s = ctx->priv_data; \ + AVPacket *in = *p_in, *pkt; \ + int ret; \ + \ + pkt = av_tree_find(s->type2, &in->pts, cmp_find, NULL); \ + if (pkt) { \ + ret = merge_packet(ctx, &s->type2, pkt1, pkt2); \ + if (!ret) \ + av_packet_move_ref(out, pkt1); \ + av_packet_free(&pkt); \ + av_packet_free(p_in); \ + return ret; \ + } \ + \ + return buffer_packet(ctx, &s->type1, p_in); \ +} + +HANDLE_PACKET(base, enhancement, in, pkt) +HANDLE_PACKET(enhancement, base, pkt, in) + +static int lcevc_merge_filter(AVBSFContext *ctx, AVPacket *out) +{ + LCEVCMergeContext *s = ctx->priv_data; + AVPacket *in; + int ret; + + do { + ret = ff_bsf_get_packet(ctx, &in); + if (ret < 0) { + if (ret == AVERROR_EOF) { + av_tree_enumerate(s->base, ctx, NULL, warn_base_buffered); + av_tree_enumerate(s->enhancement, ctx, NULL, warn_enhanced_buffered); + } + return ret; + } + + if (!in->size || in->pts < 0) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + + if (in->stream_index == s->base_idx) + ret = handle_base_packet(ctx, out, &in); + else if (in->stream_index == s->enhancement_idx) + ret = handle_enhancement_packet(ctx, out, &in); + else { + av_log(ctx, AV_LOG_ERROR, "Input packet neither base or enhacement\n"); + ret = AVERROR(EINVAL); + } + if (ret < 0) + goto fail; + } while (!out->data); + + ret = 0; +fail: + if (ret < 0) + av_packet_free(&in); + + return ret; +} + +static int free_node(void *opaque, void *elem) +{ + AVPacket *pkt = elem; + av_packet_free(&pkt); + return 0; +} + +static void lcevc_merge_flush(AVBSFContext *ctx) +{ + LCEVCMergeContext *s = ctx->priv_data; + + av_tree_enumerate(s->base, NULL, NULL, free_node); + av_tree_enumerate(s->enhancement, NULL, NULL, free_node); + av_tree_destroy(s->base); + av_tree_destroy(s->enhancement); + s->base = NULL; + s->enhancement = NULL; +} + +static void lcevc_merge_close(AVBSFContext *ctx) +{ + LCEVCMergeContext *s = ctx->priv_data; + + lcevc_merge_flush(ctx); + + ff_container_fifo_free(&s->nodes); +} + +#define OFFSET(x) offsetof(LCEVCMergeContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_BSF_PARAM) +static const AVOption lcevc_merge_options[] = { + { "base_idx", NULL, OFFSET(base_idx), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, FLAGS }, + { "enhancement_idx", NULL, OFFSET(enhancement_idx), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, FLAGS }, + { NULL } +}; + +static const AVClass lcevc_merge_class = { + .class_name = "lcevc_merge_bsf", + .item_name = av_default_item_name, + .option = lcevc_merge_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFBitStreamFilter ff_lcevc_merge_bsf = { + .p.name = "lcevc_merge", + .p.priv_class = &lcevc_merge_class, + .priv_data_size = sizeof(LCEVCMergeContext), + .init = lcevc_merge_init, + .flush = lcevc_merge_flush, + .close = lcevc_merge_close, + .filter = lcevc_merge_filter, +};