From patchwork Fri Jan 6 15:52:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 39891 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:bc95:b0:ad:ade2:bfd2 with SMTP id fx21csp957960pzb; Thu, 5 Jan 2023 23:52:51 -0800 (PST) X-Google-Smtp-Source: AMrXdXt+puttJnDJ+ifB+tmego0DYFoyPKdsXZeB7hztvGtA/P6PrpOBmii/7D84MmE+o2aFZr33 X-Received: by 2002:a05:6402:375:b0:46d:ea1e:fed0 with SMTP id s21-20020a056402037500b0046dea1efed0mr58116353edw.40.1672991571157; Thu, 05 Jan 2023 23:52:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672991571; cv=none; d=google.com; s=arc-20160816; b=FDl4IEOQhG6aE3fFiA7mkCtWgNbRDAp8x80GLFGDvqi/zuAy5VGP+IXaJlajnp5X0A ea3lP9zhkaI1o7ON0dQ8NzGpbZCmGcJjGPHcqrMsUL8AaHq7f9bKDItQderVLYprmI5B L/3K1obyYVc8vPDJEbo/rdNC4b1UwoOo1BzCHixW7rmEQVs0iDFAZ7ZQIW0z/JrfUEHz a8hDxqzrWzK7JvnWvfHgCyoYMWSjW/w8/hDQrKLJht0bDYjOfiPjfsdrZAPeDim9Xuu9 rm4ijVt3f/wAQZmBVT1zTRjKN/mxsViCHEDMgA2ireLjcvozbb2yC/OPHFaByaGjQjRm nj8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=bfkP3cJ5PMVXVjjuiG/mgn35e6h/WggfN8lIq4bO2Xc=; b=Wd08Rp7U2TIKQoRGejK9moPf5xPM5Swe0K91DIheRboxK3w2gv52NySWdT3Vtn9T+m br8ieHRJlpcL+reWOJ2jqAjtODGPKaxVw22R+2nUp0FABYnhsx8T6pe02IrRBkIomXwp M+35qwLlpPHNgx8Fql/On9WrDIanxfju9NrIE6QMRNTUibto2KJdvMLUuHeaeHlPUPcE XwgFAyMgTnr6X2rg7gcuHaXcEg4e7fXFSohIs+nhc0/GNdB/g+0NVZb41Dw4QkuNTPXu CTwWDsfixfe451qWW4jsL0qJkUbDwobwfILEulBYBBYxC3faskjy1i1cdPV3xUfzOMgE Ma8w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=Oq865TkO; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y5-20020aa7ccc5000000b0048bffdbb775si918016edt.202.2023.01.05.23.52.50; Thu, 05 Jan 2023 23:52:51 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=Oq865TkO; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5C63268BC9F; Fri, 6 Jan 2023 09:52:45 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-57-49.mail.qq.com (out162-62-57-49.mail.qq.com [162.62.57.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A7AF268BC64 for ; Fri, 6 Jan 2023 09:52:36 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1672991553; bh=edK49j2aZ/mQow0f+ysgHUbxV746OBNgBAYorHabn4k=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Oq865TkOya0d+r6hEv5ROVfACo60vVH2AQ2kNosg7ku7sj4UhP40odRJUOqaeadMD Hwo7uncuwtE1DntWrYnRu0khTFoSOr/3UDXRcKxLbUbWiALdZyd801g6MM4gJ/22yx 4MPHcI9hB10uyI6eVxSCiyvMcF7/1wEPaHkgL41s= Received: from localhost.localdomain ([59.37.125.102]) by newxmesmtplogicsvrsza2-0.qq.com (NewEsmtp) with SMTP id D1E058B1; Fri, 06 Jan 2023 15:52:30 +0800 X-QQ-mid: xmsmtpt1672991552tmxcfj693 Message-ID: X-QQ-XMAILINFO: MFWpArBVhhGTqY4JTEdMnSnl7C+f4XzRtqbaz/EJk2klYXVDthzhBaKn9uvsdq ivuiykr8aRRsqpb1xLaLcmLUnmjIZKXHbWMwcIwiW33cQ+820LZJpF6wfP0k7FEC948vc4NWoXXm dTfz1s+SuKawlMN4x7LOPjc5OuupIRwEXq0SJeLwo2l0vkzFMR6PE1d/hnI3saCfpT3tOtHmmdrc IeXIyKJPpMYJEHlOlulgmHZwLzPqB2IjxKrU4YQPEdeh3vtbrOlMQ00NLtpsq+SjkyYzvZsPieTE 6684qrhPnsQ5EY2+AIY/sVywyJB1/YV6ka8wSe5FPltui9vbTjFaQpXplOhILO77xpI5NIQGjka/ 8uCmOBxTG8W3KOzRr6gShj9CcqODKWKpZpXWoF6udpRA9YvMJ0UV/Dp2DveRr6pwExH989HbXFqZ JLNZPsZ8EP7w6Jt0w1AntHIyUifQ6ZsFh81HzENS2HSxSR71tUCMTrRfTH/UugSWtCvATWkiQW6c OqhbL7iwL8mxnkVRDCGGiz+VLNCfhQV9g7LatoAEFIxl88R4ZX+Yddoe9OEn8ax1suJ12WXCOBaj +PR4PGSKY3AvZyNPBHgYzgx+4eKZYNkYuhIAAmP07w9jkxSPR3x6G9BxqoCLrBBhuWhaybkYgKVf NXv80CEgSnz7ggJEdD/6cZysb9w8f0cMbT/IyVVeEDL8hZhxY/1AqPXOqFl4iRba520M8C7JPixG +WLaYarRVAgHVP7MmGczQf7NRzvRVFBo7g6vW+EWCYrsDA01RrNbOg6X6HPjej7wMWuv1Cy5vVuo j6mR3qFC7ZSzJP2S++Uz5sc5lNB2/ubUQ7/zSNqYCSOOEgwO7Rf3bJ2kMnX0Ei/ycqV8d3z6YIEO hku8Co4R0MddJB+Ea3UMcYkVVoXt1JF72mXMj7ycN+74Bo+0E1PtyWQWGwrM5F1eQ17919gEmyLH fGeGnpHDm3/1SldL5AxfZvcrdwbtX80ylUn17hKbIn8hokPp0ttPTJ9VNGC6JfanhPT6ljnWs= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Fri, 6 Jan 2023 23:52:30 +0800 X-OQ-MSGID: <20230106155230.487282-5-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230106155230.487282-1-quinkblack@foxmail.com> References: <20230106155230.487282-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 4/4] avcodec: add hevc_extract_layer bsf X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: /nWOQ5F6v2nI From: Zhao Zhili For example, to extract alpha layer with nuh_layer_id equal to 1: ./ffmpeg -i alpha.mp4 \ -an -c:v copy \ -bsf:v hevc_extract_layer=nuh_layer_id=1 \ output.mp4 Signed-off-by: Zhao Zhili --- configure | 1 + doc/bitstream_filters.texi | 12 +++ libavcodec/Makefile | 1 + libavcodec/bitstream_filters.c | 1 + libavcodec/hevc_extract_layer_bsf.c | 126 ++++++++++++++++++++++++++++ libavcodec/version.h | 4 +- 6 files changed, 143 insertions(+), 2 deletions(-) create mode 100644 libavcodec/hevc_extract_layer_bsf.c diff --git a/configure b/configure index 870d426b0e..a398b28790 100755 --- a/configure +++ b/configure @@ -3278,6 +3278,7 @@ filter_units_bsf_select="cbs" h264_metadata_bsf_deps="const_nan" h264_metadata_bsf_select="cbs_h264" h264_redundant_pps_bsf_select="cbs_h264" +hevc_extract_layer_bsf_select="cbs_h265" hevc_metadata_bsf_select="cbs_h265" mjpeg2jpeg_bsf_select="jpegtables" mpeg2_metadata_bsf_select="cbs_mpeg2" diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index c63c20370f..55ed8d91f9 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -382,6 +382,18 @@ This applies a specific fixup to some Blu-ray streams which contain redundant PPSs modifying irrelevant parameters of the stream which confuse other transformations which require correct extradata. +@section hevc_extract_layer + +Extract NALUs with the specified nuh_layer_id and rewrite as base +layer. Only works with INBL (independent non-base layer). + +@table @option + +@item nuh_layer_id +Which layer to extract. + +@end table + @section hevc_metadata Modify metadata embedded in an HEVC stream. diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 3ab448dd49..f27d169741 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1208,6 +1208,7 @@ OBJS-$(CONFIG_H264_METADATA_BSF) += h264_metadata_bsf.o h264_levels.o \ OBJS-$(CONFIG_H264_MP4TOANNEXB_BSF) += h264_mp4toannexb_bsf.o OBJS-$(CONFIG_H264_REDUNDANT_PPS_BSF) += h264_redundant_pps_bsf.o OBJS-$(CONFIG_HAPQA_EXTRACT_BSF) += hapqa_extract_bsf.o hap.o +OBJS-$(CONFIG_HEVC_EXTRACT_LAYER_BSF) += hevc_extract_layer_bsf.o OBJS-$(CONFIG_HEVC_METADATA_BSF) += h265_metadata_bsf.o h265_profile_level.o \ h2645data.o OBJS-$(CONFIG_HEVC_MP4TOANNEXB_BSF) += hevc_mp4toannexb_bsf.o diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index a3bebefe5f..ac013e8072 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -40,6 +40,7 @@ extern const FFBitStreamFilter ff_h264_metadata_bsf; extern const FFBitStreamFilter ff_h264_mp4toannexb_bsf; extern const FFBitStreamFilter ff_h264_redundant_pps_bsf; extern const FFBitStreamFilter ff_hapqa_extract_bsf; +extern const FFBitStreamFilter ff_hevc_extract_layer_bsf; extern const FFBitStreamFilter ff_hevc_metadata_bsf; extern const FFBitStreamFilter ff_hevc_mp4toannexb_bsf; extern const FFBitStreamFilter ff_imx_dump_header_bsf; diff --git a/libavcodec/hevc_extract_layer_bsf.c b/libavcodec/hevc_extract_layer_bsf.c new file mode 100644 index 0000000000..84c1c87a80 --- /dev/null +++ b/libavcodec/hevc_extract_layer_bsf.c @@ -0,0 +1,126 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/common.h" +#include "libavutil/opt.h" + +#include "bsf.h" +#include "bsf_internal.h" +#include "cbs.h" +#include "cbs_bsf.h" +#include "cbs_h265.h" +#include "hevc.h" + +typedef struct HevcExtractLayerContext { + CBSBSFContext common; + + int nuh_layer_id; +} HevcExtractLayerContext; + +static int hevc_update_vps(AVBSFContext *bsf, H265RawVPS *vps) +{ + HevcExtractLayerContext *ctx = bsf->priv_data; + + if (ctx->nuh_layer_id > 0) { + if (vps->vps_max_layers_minus1 == 0) + av_log(bsf, AV_LOG_ERROR, + "vps_max_layers_minus1 is zero, only base layer is available.\n"); + else if (ctx->nuh_layer_id > vps->vps_max_layer_id) + // It's a known issue that Apple videotoolbox encoder doesn't set + // vps_max_layer_id correctly. So this can be a false positive. + av_log(bsf, AV_LOG_ERROR, "Specified nuh_layer_id %d is larger than vps_max_layer_id %d, " + "this might leading to empty output\n", + ctx->nuh_layer_id, vps->vps_max_layer_id); + } + + vps->vps_max_layers_minus1 = 0; + vps->vps_max_layer_id = 0; + // TODO: update vps_extension to reflect the layer changes, other than + // drop it entirely. + vps->vps_extension_flag = 0; + + return 0; +} + +static int hevc_update_fragment(AVBSFContext *bsf, AVPacket *pkt, + CodedBitstreamFragment *au) +{ + HevcExtractLayerContext *ctx = bsf->priv_data; + H265RawNALUnitHeader *header; + int ret; + + for (int i = 0; i < au->nb_units; i++) { + if (au->units[i].type == HEVC_NAL_VPS) { + ret = hevc_update_vps(bsf, au->units[i].content); + if (ret) + return ret; + continue; + } + + header = au->units[i].content; + if (header->nuh_layer_id != ctx->nuh_layer_id) { + ff_cbs_delete_unit(au, i); + i--; + continue; + } + header->nuh_layer_id = 0; + } + + return 0; +} + +static const CBSBSFType hevc_extract_layer_type = { + .codec_id = AV_CODEC_ID_HEVC, + .fragment_name = "access unit", + .unit_name = "NAL unit", + .update_fragment = &hevc_update_fragment, +}; + +static int hevc_extract_layer_init(AVBSFContext *bsf) +{ + return ff_cbs_bsf_generic_init(bsf, &hevc_extract_layer_type); +} + +#define OFFSET(x) offsetof(HevcExtractLayerContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_BSF_PARAM) +static const AVOption hevc_extract_layer_options[] = { + { "nuh_layer_id", "Extract NALUs with the specified nuh_layer_id and rewrite as base layer", + OFFSET(nuh_layer_id), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 62, FLAGS }, + { NULL } +}; + +static const AVClass hevc_extract_layer_class = { + .class_name = "hevc_extract_layer_bsf", + .item_name = av_default_item_name, + .option = hevc_extract_layer_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +static const enum AVCodecID hevc_extract_layer_codec_ids[] = { + AV_CODEC_ID_HEVC, AV_CODEC_ID_NONE, +}; + +const FFBitStreamFilter ff_hevc_extract_layer_bsf = { + .p.name = "hevc_extract_layer", + .p.codec_ids = hevc_extract_layer_codec_ids, + .p.priv_class = &hevc_extract_layer_class, + .priv_data_size = sizeof(HevcExtractLayerContext), + .init = &hevc_extract_layer_init, + .close = &ff_cbs_bsf_generic_close, + .filter = &ff_cbs_bsf_generic_filter, +}; diff --git a/libavcodec/version.h b/libavcodec/version.h index 15f7c3fe3d..dfd3d5d7e5 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,8 +29,8 @@ #include "version_major.h" -#define LIBAVCODEC_VERSION_MINOR 56 -#define LIBAVCODEC_VERSION_MICRO 101 +#define LIBAVCODEC_VERSION_MINOR 57 +#define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \