From patchwork Sat Aug 31 16:31:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51256 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1020762vqb; Sat, 31 Aug 2024 09:40:12 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUJXOH7Cf5KjbHHEfvjyp76a4046qe/v+H81gGdLgECnk/PFz9yCtoDnKbH72x+clux7HKY92qJqFu6+kEKLy5d@gmail.com X-Google-Smtp-Source: AGHT+IFhe19ZkWwb39+s1XcqhULcmhvN+nxXE04i5pXCn7JqEVxYdQUvuhl4GHeCWGgcovY4p9zU X-Received: by 2002:a05:651c:1991:b0:2f5:10c5:9111 with SMTP id 38308e7fff4ca-2f61e0aaa72mr22355721fa.8.1725122411724; Sat, 31 Aug 2024 09:40:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725122411; cv=none; d=google.com; s=arc-20240605; b=J0oLhMSOXMCZdN6Ru5jLZ7UP1kASfXUGQnD4PAxESs94nQkyOIaV5aeAEAZUQj/keQ Va0IsJbWbd3To9x/yDoOcFQJI8tPj3EcmDxiyxE8yu1fEstKiv6xufmV8xxOoBIDdCKa BNkY1NXlObl6ixPly86MnKA/+xq6N9MX54kJcTdTA4m6I4DWvrimFLLKOPUeBBuna2eg 4FB2ks7SfPCUWXTo3l7iq+fW1owFWz0kvtwryP3TamlDvIvjY1uW8WfF/CMVRhnu15tv aW7p9EYuantY6CxN6QOJ2qXLu0D4U4fQ1nen6uZJ5tmz5F2ToycbXicyEkqHPip9xxe2 xkmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=broJgnM8Nbdiv+MEDqH2cWgXndW8dWWY1Fh4ejEsnE8=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=ZY5rt4vR9DP9x0SMzAR1YtK6ZfZRIzEofPF3g2SY7WLPyV3S1EJkmOChafVXxtjLR/ D/C5dlMLAuO+JcmLMv4q4874licRb+9XihfAnHMG+XXKGhCe9m4zSos0HLNWDVY35Yq8 g+BV3nHJp1GS9UFXP6B0l0isdlyhm9nM2ucRGN9WWyLGH3h29QLirMvRpzMOKuGFpCq5 YhlMAkEej2tmsFVhu56NUAtokRoPR9TrytMNhGsJ7fFVQWYhEY7S16yYqyXJiHN3khrU GQKu5jeTC4267JzxfhQCnti5tk2TM1Jv1RiuxlL7fflw4bSiM8hXGgiaRuJrVzRKrEAd DZKA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=TgxzmUPl; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5c226d6d241si4840579a12.631.2024.08.31.09.40.10; Sat, 31 Aug 2024 09:40:11 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=TgxzmUPl; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2DF6B68DE5E; Sat, 31 Aug 2024 19:31:11 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5EC5568DDEA for ; Sat, 31 Aug 2024 19:31:07 +0300 (EEST) Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-2d3b5f2f621so2072955a91.1 for ; Sat, 31 Aug 2024 09:31:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725121865; x=1725726665; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=4RStph+HDRefuNP4WS4Gtca82902/H+C8j9HlpzwFZw=; b=TgxzmUPlRpVtwxFQIPc0t5R0KmnnID9r56HfFLGljFQBthczqk2rssR3sGw5e0YzZ6 5VpRaTS2Wsh3MyxYYvBuFHiCrL5tILIXVeP7tEkNA8hJP/pfz0nafNeNfajGUcS8X3gg Gh7KrefB9yhuRBBeZgVWAz317niYfbxB2VsTyRzsDUbW6TPzo2bsHa5+tmYnzRPZes/4 1AtL6VF/W/z77+laJe1DtnV+OCxt1Xi2mAEcN8JsZNhm0hgiKITyees5KRxGoUcCdJFh lTzngdWQPfcIZcEeXNlfLSxFdAuvYuGyMzlX+Tzf/RimUAOpygRtPK9rO5OPRuVhoehz pq/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725121865; x=1725726665; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4RStph+HDRefuNP4WS4Gtca82902/H+C8j9HlpzwFZw=; b=CUHiP6fkq+QzAXzjSMf2TXw3K7gcr8hGGhP6AlhMswvYgCFunQiTvQHA0Xxm2yF6y8 90pZpa5u9pD09BWIl8oHz+L8LEBEAHsscgghL2bh6F+jZBibp7jxfINVNpqEmDHlUpCs XVw0/Orh6clFk+LX/xF6O0xS4OEehvdxVaK3a3QmFgatxeKbUeG5GEi/J43Q3h6sv5LC WoziMJSl4NfQCrQCapWKa1RRQ+4td8Kz+B1h6pe10zDoN0mMNGf88PgakIltSdqd5C71 Z7ZkC1LiWJWSSyeAypL/lrUHoKLHn6k+5UtUwJqBx+84xU52tiiLL9eHT1IY5CDbQMkY hTQg== X-Gm-Message-State: AOJu0YylRzVdBqQo/7cM+x8jeyK7J1on998kxDrupa9jsft1jxOkKUp3 kpldnwT3S2ZuFGrfK0KhnEV/mc9olLY22lriL6gpserXJ1SyUUf4crF8ug== X-Received: by 2002:a17:90a:fe18:b0:2cd:1e60:9c31 with SMTP id 98e67ed59e1d1-2d894453fd1mr2065079a91.30.1725121864466; Sat, 31 Aug 2024 09:31:04 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d85b3b9522sm5935328a91.50.2024.08.31.09.31.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 31 Aug 2024 09:31:03 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Aug 2024 13:31:13 -0300 Message-ID: <20240831163114.4197-12-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240831163114.4197-1-jamrial@gmail.com> References: <20240831163114.4197-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 12/13] avcodec: add an LCEVC merger bsf X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ecloHbC5BIhR Signed-off-by: James Almer --- libavcodec/bitstream_filters.c | 1 + libavcodec/bsf/Makefile | 3 +- libavcodec/bsf/lcevc_merge_bsf.c | 265 +++++++++++++++++++++++++++++++ 3 files changed, 268 insertions(+), 1 deletion(-) create mode 100644 libavcodec/bsf/lcevc_merge_bsf.c diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index f923411bee..fdd4fcf01b 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -45,6 +45,7 @@ extern const FFBitStreamFilter ff_hapqa_extract_bsf; extern const FFBitStreamFilter ff_hevc_metadata_bsf; extern const FFBitStreamFilter ff_hevc_mp4toannexb_bsf; extern const FFBitStreamFilter ff_imx_dump_header_bsf; +extern const FFBitStreamFilter ff_lcevc_merge_bsf; extern const FFBitStreamFilter ff_media100_to_mjpegb_bsf; extern const FFBitStreamFilter ff_mjpeg2jpeg_bsf; extern const FFBitStreamFilter ff_mjpega_dump_header_bsf; diff --git a/libavcodec/bsf/Makefile b/libavcodec/bsf/Makefile index 40b7fc6e9b..850bcaf377 100644 --- a/libavcodec/bsf/Makefile +++ b/libavcodec/bsf/Makefile @@ -1,4 +1,4 @@ -clean:: +\clean:: $(RM) $(CLEANSUFFIXES:%=libavcodec/bsf/%) OBJS-$(CONFIG_AAC_ADTSTOASC_BSF) += bsf/aac_adtstoasc.o @@ -22,6 +22,7 @@ OBJS-$(CONFIG_HEVC_METADATA_BSF) += bsf/h265_metadata.o OBJS-$(CONFIG_DOVI_RPU_BSF) += bsf/dovi_rpu.o OBJS-$(CONFIG_HEVC_MP4TOANNEXB_BSF) += bsf/hevc_mp4toannexb.o OBJS-$(CONFIG_IMX_DUMP_HEADER_BSF) += bsf/imx_dump_header.o +OBJS-$(CONFIG_LCEVC_MERGE_BSF) += bsf/lcevc_merge_bsf.o OBJS-$(CONFIG_MEDIA100_TO_MJPEGB_BSF) += bsf/media100_to_mjpegb.o OBJS-$(CONFIG_MJPEG2JPEG_BSF) += bsf/mjpeg2jpeg.o OBJS-$(CONFIG_MJPEGA_DUMP_HEADER_BSF) += bsf/mjpega_dump_header.o diff --git a/libavcodec/bsf/lcevc_merge_bsf.c b/libavcodec/bsf/lcevc_merge_bsf.c new file mode 100644 index 0000000000..fcaeb7ef7f --- /dev/null +++ b/libavcodec/bsf/lcevc_merge_bsf.c @@ -0,0 +1,265 @@ +/* + * Copyright (c) 2022 James Almer + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Derive PTS by reordering DTS from supported streams + */ + +#include "libavutil/avassert.h" +#include "libavutil/attributes.h" +#include "libavutil/fifo.h" +#include "libavutil/mem.h" +#include "libavutil/opt.h" +#include "libavutil/tree.h" + +#include "bsf.h" +#include "bsf_internal.h" + +typedef struct LCEVCMergeContext { + const AVClass *class; + + struct AVTreeNode *base; + struct AVTreeNode *enhancement; + AVFifo *nodes; + + int base_idx, enhancement_idx; +} LCEVCMergeContext; + +// AVTreeNode callbacks +static int cmp_insert(const void *_key, const void *_node) +{ + const AVPacket *key = _key, *node = _node; + + return FFDIFFSIGN(key->pts, node->pts); +} + +static int cmp_find(const void *_key, const void *_node) +{ + int64_t key = *(const int64_t *)_key; + const AVPacket *node = _node; + + return FFDIFFSIGN(key, node->pts); +} + +#define WARN_BUFFERED(type) \ +static int warn_##type##_buffered(void *logctx, void *elem) \ +{ \ + const AVPacket *pkt = (const AVPacket *)elem; \ + av_log(logctx, AV_LOG_WARNING, #type" packet with PTS %"PRId64 \ + " left buffered at EOF\n", pkt->pts); \ + return 0; \ +} + +WARN_BUFFERED(base) +WARN_BUFFERED(enhanced) + +static int lcevc_merge_init(AVBSFContext *ctx) +{ + LCEVCMergeContext *s = ctx->priv_data; + + if (s->base_idx < 0 || s->enhancement_idx < 0) { + av_log(ctx, AV_LOG_ERROR, "Both base and enhancement stream index must be set\n"); + return AVERROR(EINVAL); + } + + s->nodes = av_fifo_alloc2(1, sizeof(struct AVTreeNode*), AV_FIFO_FLAG_AUTO_GROW); + if (!s->nodes) + return AVERROR(ENOMEM); + + return 0; +} + +static int merge_packet(AVBSFContext *ctx, struct AVTreeNode **root, + AVPacket *out, AVPacket *in) +{ + LCEVCMergeContext *s = ctx->priv_data; + struct AVTreeNode *node = NULL; + uint8_t *side_data; + int ret; + + // It doesn't matter if the packet is from the base or enhancement stream + // as both share the pts cmp_insert() will look for to remove the element. + av_tree_insert(root, in, cmp_insert, &node); + memset(node, 0, av_tree_node_size); + ret = av_fifo_write(s->nodes, &node, 1); + if (ret < 0) { + av_free(node); + return ret; + } + + side_data = av_packet_new_side_data(out, AV_PKT_DATA_LCEVC, in->size); + if (!side_data) + return AVERROR(ENOMEM); + + memcpy(side_data, in->data, in->size); + + return 0; +} + +static int buffer_packet(AVBSFContext *ctx, struct AVTreeNode **root, + AVPacket **p_in) +{ + LCEVCMergeContext *s = ctx->priv_data; + AVPacket *in = *p_in, *pkt; + struct AVTreeNode *node = NULL; + + if (av_fifo_can_read(s->nodes)) { + int av_unused ret = av_fifo_read(s->nodes, &node, 1); + av_assert2(ret >= 0); + } else + node = av_tree_node_alloc(); + if (!node) + return AVERROR(ENOMEM); + + pkt = av_tree_insert(root, in, cmp_insert, &node); + if (pkt && pkt != in) { + av_log(ctx, AV_LOG_ERROR, "Duplicate packet with PTS %"PRId64 + " for stream_index %d \n", in->pts, in->stream_index); + av_free(node); + return AVERROR_INVALIDDATA; + } + *p_in = NULL; + + return 0; +} + +#define HANDLE_PACKET(type1, type2, pkt1, pkt2) \ +static int handle_##type1##_packet(AVBSFContext *ctx, AVPacket *out, AVPacket **p_in) \ +{ \ + LCEVCMergeContext *s = ctx->priv_data; \ + AVPacket *in = *p_in, *pkt; \ + int ret; \ + \ + pkt = av_tree_find(s->type2, &in->pts, cmp_find, NULL); \ + if (pkt) { \ + ret = merge_packet(ctx, &s->type2, pkt1, pkt2); \ + if (!ret) \ + av_packet_move_ref(out, pkt1); \ + av_packet_free(&pkt); \ + av_packet_free(p_in); \ + return ret; \ + } \ + \ + return buffer_packet(ctx, &s->type1, p_in); \ +} + +HANDLE_PACKET(base, enhancement, in, pkt) +HANDLE_PACKET(enhancement, base, pkt, in) + +static int lcevc_merge_filter(AVBSFContext *ctx, AVPacket *out) +{ + LCEVCMergeContext *s = ctx->priv_data; + AVPacket *in; + int ret; + + do { + ret = ff_bsf_get_packet(ctx, &in); + if (ret < 0) { + if (ret == AVERROR_EOF) { + av_tree_enumerate(s->base, ctx, NULL, warn_base_buffered); + av_tree_enumerate(s->enhancement, ctx, NULL, warn_enhanced_buffered); + } + return ret; + } + + if (!in->size || in->pts < 0) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + + if (in->stream_index == s->base_idx) + ret = handle_base_packet(ctx, out, &in); + else if (in->stream_index == s->enhancement_idx) + ret = handle_enhancement_packet(ctx, out, &in); + else { + av_log(ctx, AV_LOG_ERROR, "Input packet neither base or enhacement\n"); + ret = AVERROR(EINVAL); + } + if (ret < 0) + goto fail; + } while (!out->data); + + ret = 0; +fail: + if (ret < 0) + av_packet_free(&in); + + return ret; +} + +static int free_node(void *opaque, void *elem) +{ + AVPacket *pkt = elem; + av_packet_free(&pkt); + return 0; +} + +static void lcevc_merge_flush(AVBSFContext *ctx) +{ + LCEVCMergeContext *s = ctx->priv_data; + + av_tree_enumerate(s->base, NULL, NULL, free_node); + av_tree_enumerate(s->enhancement, NULL, NULL, free_node); + av_tree_destroy(s->base); + av_tree_destroy(s->enhancement); + s->base = NULL; + s->enhancement = NULL; +} + +static void lcevc_merge_close(AVBSFContext *ctx) +{ + LCEVCMergeContext *s = ctx->priv_data; + + lcevc_merge_flush(ctx); + + if (s->nodes) { + struct AVTreeNode *node; + while (av_fifo_read(s->nodes, &node, 1) >= 0) + av_free(node); + } + + av_fifo_freep2(&s->nodes); +} + +#define OFFSET(x) offsetof(LCEVCMergeContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_BSF_PARAM) +static const AVOption lcevc_merge_options[] = { + { "base_idx", NULL, OFFSET(base_idx), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, FLAGS }, + { "enhancement_idx", NULL, OFFSET(enhancement_idx), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, FLAGS }, + { NULL } +}; + +static const AVClass lcevc_merge_class = { + .class_name = "lcevc_merge_bsf", + .item_name = av_default_item_name, + .option = lcevc_merge_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFBitStreamFilter ff_lcevc_merge_bsf = { + .p.name = "lcevc_merge", + .p.priv_class = &lcevc_merge_class, + .priv_data_size = sizeof(LCEVCMergeContext), + .init = lcevc_merge_init, + .flush = lcevc_merge_flush, + .close = lcevc_merge_close, + .filter = lcevc_merge_filter, +};