From patchwork Fri May 26 10:31:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dawid Kozinski X-Patchwork-Id: 41839 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c51c:b0:10c:5e6f:955f with SMTP id gm28csp1219942pzb; Fri, 26 May 2023 03:32:06 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7swuylXUkB684UN8b9PvFBM2ed0XFImgP1fo/Rn30m6ypG8Mo0/IPE8EsuToPP8KF3vpNm X-Received: by 2002:a17:907:16a3:b0:968:4d51:800b with SMTP id hc35-20020a17090716a300b009684d51800bmr1865766ejc.1.1685097125823; Fri, 26 May 2023 03:32:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685097125; cv=none; d=google.com; s=arc-20160816; b=QjLXKZ+jlgfrUDZh+JwQlgcLs+wOjkQj6lQ+Lvt13zleLsneNlAM6NQc7KHBRuahel FMTRfUFhvBMebH9c53xbqDOpI9aYUwuEzxcFIngOT47Ok4fo/JQS8QMbUms8ijgsTm9S sD9L0HSu/LdjmC70OReQuJ+j8p67dMUQ16nAe9g6ivd3zBUY9f+mCstfwTTX/lpW50ew XeAnLS4CNoV3c+2ka5V3VvS5KnCIe7bpd+oaJQOvsq0fPErCQgTw2WLhPzIXkejnb5gq vlEzjMxYaZ1UkXLmksUnzZubUj8LsSHaHP4UBtReA4L10V7yceSqRTMprI6kEa9BbzBO 2lCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:cms-type:mime-version :message-id:date:to:from:dkim-signature:dkim-filter:delivered-to; bh=AYKoBvVmjEb0o7moqjtch8/MxXH0bGr5JLdlkL+5J/U=; b=fI5q+5Ar/UTxVC4XcQ5x6hiP8DcK0ZVKNHi0nJ8tDzHTPsmmYQ4EyaNvN/fMvlttFI odvSGywSUX4rxH6lVp0miJ6XZvzpvWmt4tJ1ExBGO1AFvKfdZbdd2Y3wqLINKsUt+c4d fZr2G5+Qke+EdSVhICS5O+Yd07eP63WLBgPGOilJq/C9sQPaTozPNDeXIZco1iVjuDgC 7A0rBki2nOiViKkA4TAmt/xX6Zh53jR0GwmjJ+vFG4jhD2FkHi6+Glixjxl4wxHd6yuL EW6X+3LGM4BTW0mShyFg4mO3fz6SgboAgsJCei6fFBQTuwwtFm8j2hEwejo45H1F+D07 YN9Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@samsung.com header.s=mail20170921 header.b=h6PIhPb+; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=samsung.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ku16-20020a170907789000b0096f79e9c5efsi597314ejc.590.2023.05.26.03.32.05; Fri, 26 May 2023 03:32:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@samsung.com header.s=mail20170921 header.b=h6PIhPb+; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=samsung.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1241D68C1F1; Fri, 26 May 2023 13:32:02 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4E52068C1CA for ; Fri, 26 May 2023 13:32:01 +0300 (EEST) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20230526103159euoutp02de65c37ea1bd222a04772d2756727ea6~iqucLIf3D1295712957euoutp02K for ; Fri, 26 May 2023 10:31:59 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20230526103159euoutp02de65c37ea1bd222a04772d2756727ea6~iqucLIf3D1295712957euoutp02K DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1685097119; bh=aznFdHwft41x8mEYpJMPcsDZSaFI4kX03BcI9R3M0zU=; h=From:To:Cc:Subject:Date:References:From; b=h6PIhPb+6itN14PGqo9HFzpg+JqUSE7fYKl7l8AGFtQnJi10ozU8T7Aw7xiKfAca1 6/x4pNDmHv8tgmc4mDOklFSsb/yJ+AEcxcFNCzWARunYBJ6l5V+mNpJ5aj5b441usq i97lDkGN32O078m9WmurtVoUKlEt9EKFeMVnHoxU= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20230526103159eucas1p25cb44060d7782eb53e1d3044dd53e357~iqubqzx_P1323313233eucas1p2S; Fri, 26 May 2023 10:31:59 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 89.51.11320.F9A80746; Fri, 26 May 2023 11:31:59 +0100 (BST) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20230526103158eucas1p1e8b2adfc43aaf57c62e23e78571df817~iqubRAe432944329443eucas1p1L; Fri, 26 May 2023 10:31:58 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20230526103158eusmtrp125432883c559388cad9ac16c7f55c6d2~iqubQc_rV0398603986eusmtrp1Z; Fri, 26 May 2023 10:31:58 +0000 (GMT) X-AuditID: cbfec7f4-97dff70000022c38-a9-64708a9f9f5b Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 7D.01.10549.E9A80746; Fri, 26 May 2023 11:31:58 +0100 (BST) Received: from AMDN5164.EU.corp.samsungelectronics.net (unknown [106.210.132.171]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20230526103158eusmtip1a3a93584e71ece46ff8e5c99dc9cdd6f~iquawCYzM1180611806eusmtip1t; Fri, 26 May 2023 10:31:58 +0000 (GMT) From: Dawid Kozinski To: d.frankiewic@samsung.com, ffmpeg-devel@ffmpeg.org Date: Fri, 26 May 2023 12:31:45 +0200 Message-Id: <20230526103145.1277-1-d.kozinski@samsung.com> X-Mailer: git-send-email 2.37.3.windows.1 MIME-Version: 1.0 X-Unsent: 1 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrDIsWRmVeSWpSXmKPExsWy7djP87rzuwpSDC6+4bL4cu0Ls8XKb79Z LL59OsPswOzxZ9FmFo++LasYA5iiuGxSUnMyy1KL9O0SuDJeLe5gLFh2i6li1dqtjA2MM78w djFyckgImEjc720Dsrk4hARWMErsmvsTyvnCKPH3P0zmM6PEobvPmGFatu84CNYuJLCcUeL9 FlmIonYmif131rGBJNgEdCVefnjEBGKLCJhKnGzeCxZnFtCWWHf3O5DNwSEskCRxcpoDSJhF QFXiy/+jYDN5BawkVp5cBnWepsSELXOYIOKCEidnPmGBiHNLrJt1iRVipLxE89bZzCA3SAj8 ZJd4OrUBqtlF4vjRU+wQtrDEq+NboGwZidOTe1hAbpAQKJY41O8AYdZIHPqRDlFhLfG28Tgj SJgZ6IT1u/Qhwo4STx+fZYeo5pO48VYQ4gA+iUnbpjNDhHklOtqEIEwVib5OMYhGKYmny+ZA g89D4tW+RpYJjIqzkHw1C8knsxDWLmBkXsUonlpanJueWmyUl1quV5yYW1yal66XnJ+7iRGY IE7/O/5lB+PyVx/1DjEycTAeYpTgYFYS4d2Qk58ixJuSWFmVWpQfX1Sak1p8iFGag0VJnFfb 9mSykEB6YklqdmpqQWoRTJaJg1OqgSnacGmz/4oJi3ia9Lv/LrwzKy9kf/gXkbCgkxPepPMd 7hIQ2z89/f/m6wLMl5fv3T/l/U7/uOTtsZKR3P7ea3er1ZubKlpseajGaL4qMHjZzdn+//0v TuHNbbztf8Svaer1/JXHSpdo1ezz4ZDvtLbZde1R/pv2zR6ROv+5dmTevDpD0eXfTrGCOz7L fL6ZvjX8wa79Q9wnTXxtiDLb/oUu6rJb6uYv3tTg/TbzFeNh8Yzas3uDFY//nHLDoqY2d/6j CQ+u1z5WlsvZYTRvzpxdCfJNXTdUC9dPYHkxs+fP3Dkngqc3RvO0Vm2+Zxjz/K7A4yWBa1ue 6/b1/D2e1jVzVyWvxOPpE87UlByenzpNiaU4I9FQi7moOBEAlEhPTn8DAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrMLMWRmVeSWpSXmKPExsVy+t/xu7rzugpSDHa9MrL4cu0Ls8XKb79Z LL59OsPswOzxZ9FmFo++LasYA5ii9GyK8ktLUhUy8otLbJWiDS2M9AwtLfSMTCz1DI3NY62M TJX07WxSUnMyy1KL9O0S9DJeLe5gLFh2i6li1dqtjA2MM78wdjFyckgImEhs33EQyObiEBJY yijxYsMDVoiElMTSpYugioQl/lzrYgOxhQRamSQmrAwEsdkEdCVefnjEBGKLCJhLXFx3C6ye WUBbYt3d72D1wgIJEt+nbACLswioSnz5fxTM5hWwklh5chnUfE2JCVvmMEHEBSVOznzCAhHn llg36xIrxEx5ieats5knMPLPQlI2C0lqASPTKkaR1NLi3PTcYkO94sTc4tK8dL3k/NxNjMDQ 3Xbs5+YdjPNefdQ7xMjEwXiIUYKDWUmEd0NOfooQb0piZVVqUX58UWlOavEhRlOgWycyS4km 5wOjJ68k3tDMwNTQxMzSwNTSzFhJnNezoCNRSCA9sSQ1OzW1ILUIpo+Jg1OqgYl31/Kn9uoJ WpMftEq/nvzV/oEg851nhZ9iAna9m+RblHXEKWDVTz5thcvnlOdziMy5w32Q6dattCPf584p 78xtv/PQd1JRufXR1d6aTruOZxS9fhckuWr5Q3fZlJaegqpJj9+zdiRV6hxIXP+jwojxa88K pobm5qzPwQ8Cp246JRaYcYXN8GxIMtuMv0IGRw9O7w8Kk/Dd/UViisX+6gMS27Y7mItJuz1d +kjsl2w9V50gm7zkvs2PTJc8FT/79IjctqrD+21a5qUetbBhnXoi+3LE9thI7r0NiuYTFrl5 2xuazt654OZH3o//M1V3/J/6LDZgsfRirYT/8/+9//+y/E78doafJQ9n88++8jNBiaU4I9FQ i7moOBEAMcFHZOYCAAA= X-CMS-MailID: 20230526103158eucas1p1e8b2adfc43aaf57c62e23e78571df817 X-Msg-Generator: CA X-RootMTR: 20230526103158eucas1p1e8b2adfc43aaf57c62e23e78571df817 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20230526103158eucas1p1e8b2adfc43aaf57c62e23e78571df817 References: Subject: [FFmpeg-devel] [PATCH v23 04/10] avformat/evc_demuxer: Added demuxer to handle reading EVC video files X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Dawid Kozinski Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: kVZq2FZcWaKs - Provided AVInputFormat struct describing EVC input format (ff_evc_demuxer) Signed-off-by: Dawid Kozinski --- libavcodec/Makefile | 1 + libavcodec/evc_frame_merge_bsf.c | 857 +++++++++++++++++++++++++++++++ libavformat/Makefile | 1 + libavformat/allformats.c | 1 + libavformat/evcdec.c | 334 ++++++++++++ 5 files changed, 1194 insertions(+) create mode 100644 libavcodec/evc_frame_merge_bsf.c create mode 100644 libavformat/evcdec.c diff --git a/libavcodec/Makefile b/libavcodec/Makefile index c598e8d6ca..a5a0c58750 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1251,6 +1251,7 @@ OBJS-$(CONFIG_VP9_METADATA_BSF) += vp9_metadata_bsf.o OBJS-$(CONFIG_VP9_RAW_REORDER_BSF) += vp9_raw_reorder_bsf.o OBJS-$(CONFIG_VP9_SUPERFRAME_BSF) += vp9_superframe_bsf.o OBJS-$(CONFIG_VP9_SUPERFRAME_SPLIT_BSF) += vp9_superframe_split_bsf.o +OBJS-$(CONFIG_EVC_FRAME_MERGE_BSF) += evc_frame_merge_bsf.o # thread libraries OBJS-$(HAVE_LIBC_MSVCRT) += file_open.o diff --git a/libavcodec/evc_frame_merge_bsf.c b/libavcodec/evc_frame_merge_bsf.c new file mode 100644 index 0000000000..7cfcfaa335 --- /dev/null +++ b/libavcodec/evc_frame_merge_bsf.c @@ -0,0 +1,857 @@ +/* + * Copyright (c) 2019 James Almer + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ +#include "get_bits.h" +#include "golomb.h" +#include "bsf.h" +#include "bsf_internal.h" +#include "avcodec.h" + +#include "evc.h" + +#define INIT_AU_BUF_CAPACITY 1024 + +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.1 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +typedef struct EVCParserSPS { + int sps_seq_parameter_set_id; // ue(v) + int profile_idc; // u(8) + + // int level_idc; // u(8) + // int toolset_idc_h; // u(32) + // int toolset_idc_l; // u(32) + + int chroma_format_idc; // ue(v) + + // int pic_width_in_luma_samples; // ue(v) + // int pic_height_in_luma_samples; // ue(v) + // int bit_depth_luma_minus8; // ue(v) + // int bit_depth_chroma_minus8; // ue(v) + + int sps_btt_flag; // u(1) + // int log2_ctu_size_minus5; // ue(v) + // int log2_min_cb_size_minus2; // ue(v) + // int log2_diff_ctu_max_14_cb_size; // ue(v) + // int log2_diff_ctu_max_tt_cb_size; // ue(v) + // int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) + + int sps_suco_flag; // u(1) + // int log2_diff_ctu_size_max_suco_cb_size; // ue(v) + // int log2_diff_max_suco_min_suco_cb_size; // ue(v) + + int sps_admvp_flag; // u(1) + // int sps_affine_flag; // u(1) + // int sps_amvr_flag; // u(1) + // int sps_dmvr_flag; // u(1) + int sps_mmvd_flag; // u(1) + // int sps_hmvp_flag; // u(1) + + int sps_eipd_flag; // u(1) + int sps_ibc_flag; // u(1) + // int log2_max_ibc_cand_size_minus2; // ue(v) + + int sps_cm_init_flag; // u(1) + int sps_adcc_flag; // u(1) + + int sps_iqt_flag; // u(1) + // int sps_ats_flag; // u(1) + + // int sps_addb_flag; // u(1) + int sps_alf_flag; // u(1) + // int sps_htdf_flag; // u(1) + int sps_rpl_flag; // u(1) + int sps_pocs_flag; // u(1) + // int sps_dquant_flag; // u(1) + // int sps_dra_flag; // u(1) + + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) + int log2_sub_gop_length; // ue(v) + + // int log2_ref_pic_gap_length; // ue(v) + + // int max_num_tid0_ref_pics; // ue(v) + + // int sps_max_dec_pic_buffering_minus1; // ue(v) + // int long_term_ref_pic_flag; // u(1) + // int rpl1_same_as_rpl0_flag; // u(1) + // int num_ref_pic_list_in_sps[2]; // ue(v) + // struct RefPicListStruct rpls[2][EVC_MAX_NUM_RPLS]; + + // int picture_cropping_flag; // u(1) + // int picture_crop_left_offset; // ue(v) + // int picture_crop_right_offset; // ue(v) + // int picture_crop_top_offset; // ue(v) + // int picture_crop_bottom_offset; // ue(v) + + // struct ChromaQpTable chroma_qp_table_struct; + + // int vui_parameters_present_flag; // u(1) + + // struct VUIParameters vui_parameters; +} EVCParserSPS; + +typedef struct EVCParserPPS { + int pps_pic_parameter_set_id; // ue(v) + int pps_seq_parameter_set_id; // ue(v) + // int num_ref_idx_default_active_minus1[2]; // ue(v) + // int additional_lt_poc_lsb_len; // ue(v) + // int rpl1_idx_present_flag; // u(1) + int single_tile_in_pic_flag; // u(1) + int num_tile_columns_minus1; // ue(v) + int num_tile_rows_minus1; // ue(v) + int uniform_tile_spacing_flag; // u(1) + // int tile_column_width_minus1[EVC_MAX_TILE_ROWS]; // ue(v) + // int tile_row_height_minus1[EVC_MAX_TILE_COLUMNS]; // ue(v) + int loop_filter_across_tiles_enabled_flag; // u(1) + // int tile_offset_len_minus1; // ue(v) + int tile_id_len_minus1; // ue(v) + int explicit_tile_id_flag; // u(1) + // int tile_id_val[EVC_MAX_TILE_ROWS][EVC_MAX_TILE_COLUMNS]; // u(v) + int pic_dra_enabled_flag; // u(1) + int pic_dra_aps_id; // u(5) + int arbitrary_slice_present_flag; // u(1) + // int constrained_intra_pred_flag; // u(1) + // int cu_qp_delta_enabled_flag; // u(1) + // int log2_cu_qp_delta_area_minus6; // ue(v) + +} EVCParserPPS; + +// The sturcture reflects Slice Header RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.6 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +// u(n) - unsigned integer using n bits. +// When n is "v" in the syntax table, the number of bits varies in a manner dependent on the value of other syntax elements. +typedef struct EVCParserSliceHeader { + int slice_pic_parameter_set_id; // ue(v) + int single_tile_in_slice_flag; // u(1) + // int first_tile_id; // u(v) + int arbitrary_slice_flag; // u(1) + // int last_tile_id; // u(v) + int num_remaining_tiles_in_slice_minus1; // ue(v) + // int delta_tile_id_minus1[EVC_MAX_TILE_ROWS * EVC_MAX_TILE_COLUMNS]; // ue(v) + + int slice_type; // ue(v) + // int no_output_of_prior_pics_flag; // u(1) + // int mmvd_group_enable_flag; // u(1) + int slice_alf_enabled_flag; // u(1) + + int slice_alf_luma_aps_id; // u(5) + int slice_alf_map_flag; // u(1) + int slice_alf_chroma_idc; // u(2) + int slice_alf_chroma_aps_id; // u(5) + // int slice_alf_chroma_map_flag; // u(1) + // int slice_alf_chroma2_aps_id; // u(5) + // int slice_alf_chroma2_map_flag; // u(1) + int slice_pic_order_cnt_lsb; // u(v) OK + + // @note + // Currently the structure does not reflect the entire Slice Header RBSP layout. + // It contains only the fields that are necessary to read from the NAL unit all the values + // necessary for the correct initialization of the AVCodecContext structure. + + // @note + // If necessary, add the missing fields to the structure to reflect + // the contents of the entire NAL unit of the SPS type + +} EVCParserSliceHeader; + +typedef struct EVCParserPoc { + int PicOrderCntVal; // current picture order count value + int prevPicOrderCntVal; // the picture order count of the previous Tid0 picture + int DocOffset; // the decoding order count of the previous picture +} EVCParserPoc; + +// Access unit data +typedef struct AccessUnitBuffer { + uint8_t *data; // the data buffer + size_t data_size; // size of data in bytes + size_t capacity; // buffer capacity +} AccessUnitBuffer; + +typedef struct EVCMergeContext { + AVPacket *in; + + EVCParserSPS *sps[EVC_MAX_SPS_COUNT]; + EVCParserPPS *pps[EVC_MAX_PPS_COUNT]; + EVCParserSliceHeader *slice_header[EVC_MAX_PPS_COUNT]; + + EVCParserPoc poc; + + // the Baseline profile is indicated by profile eqal to 0 + // the Main profile is indicated by profile eqal to 1 + int profile; + int nalu_type; // the current NALU type + int nalu_size; // the current NALU size + + int key_frame; + + AccessUnitBuffer au_buffer; + +} EVCMergeContext; + +// nuh_temporal_id specifies a temporal identifier for the NAL unit +static int get_temporal_id(const uint8_t *bits, int bits_size) +{ + int temporal_id = 0; + short t = 0; + + if (bits_size >= EVC_NALU_HEADER_SIZE) { + unsigned char *p = (unsigned char *)bits; + // forbidden_zero_bit + if ((p[0] & 0x80) != 0) + return -1; + + for (int i = 0; i < EVC_NALU_HEADER_SIZE; i++) + t = (t << 8) | p[i]; + + temporal_id = (t >> 6) & 0x0007; + } + + return temporal_id; +} + +static int get_nalu_type(const uint8_t *bits, int bits_size) +{ + int unit_type_plus1 = 0; + + if (bits_size >= EVC_NALU_HEADER_SIZE) { + unsigned char *p = (unsigned char *)bits; + // forbidden_zero_bit + if ((p[0] & 0x80) != 0) // Cannot get bitstream information. Malformed bitstream. + return -1; + + // nal_unit_type + unit_type_plus1 = (p[0] >> 1) & 0x3F; + } + + return unit_type_plus1 - 1; +} + +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size) +{ + uint32_t nalu_len = 0; + + if (bits_size >= EVC_NALU_LENGTH_PREFIX_SIZE) { + + int t = 0; + unsigned char *p = (unsigned char *)bits; + + for (int i = 0; i < EVC_NALU_LENGTH_PREFIX_SIZE; i++) + t = (t << 8) | p[i]; + + nalu_len = t; + if (nalu_len == 0) // Invalid bitstream size + return 0; + } + + return nalu_len; +} + +static int end_of_access_unit_found(AVBSFContext *s) +{ + EVCMergeContext *ctx = s->priv_data; + + if (ctx->profile == 0) { // BASELINE profile + if (ctx->nalu_type == EVC_NOIDR_NUT || ctx->nalu_type == EVC_IDR_NUT) + return 1; + } else { // MAIN profile + if (ctx->nalu_type == EVC_NOIDR_NUT) { + if (ctx->poc.PicOrderCntVal != ctx->poc.prevPicOrderCntVal) + return 1; + } else if (ctx->nalu_type == EVC_IDR_NUT) + return 1; + } + return 0; +} + +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) +static EVCParserSPS *parse_sps(const uint8_t *bs, int bs_size, EVCMergeContext *ctx) +{ + GetBitContext gb; + EVCParserSPS *sps; + int sps_seq_parameter_set_id; + + if (init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + sps_seq_parameter_set_id = get_ue_golomb(&gb); + + if (sps_seq_parameter_set_id >= EVC_MAX_SPS_COUNT) + return NULL; + + if(!ctx->sps[sps_seq_parameter_set_id]) { + if((ctx->sps[sps_seq_parameter_set_id] = av_malloc(sizeof(EVCParserSPS))) == NULL) + return NULL; + } + + sps = ctx->sps[sps_seq_parameter_set_id]; + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; + + // the Baseline profile is indicated by profile_idc eqal to 0 + // the Main profile is indicated by profile_idc eqal to 1 + sps->profile_idc = get_bits(&gb, 8); + + skip_bits_long(&gb, 8); // sps->level_idc = get_bits(&gb, 8); + + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ + + // 0 - monochrome + // 1 - 4:2:0 + // 2 - 4:2:2 + // 3 - 4:4:4 + sps->chroma_format_idc = get_ue_golomb(&gb); + + get_ue_golomb(&gb); /* skip pic_width_in_luma_samples */ + get_ue_golomb(&gb); /* skip pic_height_in_luma_samples */ + + get_ue_golomb(&gb); /* skip bit_depth_luma_minus8 = */ + get_ue_golomb(&gb); /* skip bit_depth_chroma_minus8 = */ + + sps->sps_btt_flag = get_bits(&gb, 1); + if (sps->sps_btt_flag) { + get_ue_golomb(&gb); /* skip log2_ctu_size_minus5 */ + get_ue_golomb(&gb); /* skip log2_min_cb_size_minus2 */ + get_ue_golomb(&gb); /* skip log2_diff_ctu_max_14_cb_size */ + get_ue_golomb(&gb); /* skip log2_diff_ctu_max_tt_cb_size */ + get_ue_golomb(&gb); /* skip log2_diff_min_cb_min_tt_cb_size_minus2 */ + } + + sps->sps_suco_flag = get_bits(&gb, 1); + if (sps->sps_suco_flag) { + get_ue_golomb(&gb); /* skip log2_diff_ctu_size_max_suco_cb_size = */ + get_ue_golomb(&gb); /* skip log2_diff_max_suco_min_suco_cb_size = */ + } + + sps->sps_admvp_flag = get_bits(&gb, 1); + if (sps->sps_admvp_flag) { + skip_bits_long(&gb, 1); /* skip sps_affine_flag */ + skip_bits_long(&gb, 1); /* skip sps_amvr_flag */ + skip_bits_long(&gb, 1); /* skip sps_dmvr_flag */ + + sps->sps_mmvd_flag = get_bits(&gb, 1); + + skip_bits_long(&gb, 1); /* skip sps_hmvp_flag */ + } + + sps->sps_eipd_flag = get_bits(&gb, 1); + if (sps->sps_eipd_flag) { + sps->sps_ibc_flag = get_bits(&gb, 1); + if (sps->sps_ibc_flag) + get_ue_golomb(&gb); /* skip log2_max_ibc_cand_size_minus2 */ + } + + sps->sps_cm_init_flag = get_bits(&gb, 1); + if (sps->sps_cm_init_flag) + sps->sps_adcc_flag = get_bits(&gb, 1); + + sps->sps_iqt_flag = get_bits(&gb, 1); + if (sps->sps_iqt_flag) + skip_bits_long(&gb, 1); /* skip sps_ats_flag */ + + skip_bits_long(&gb, 1); /* skip sps_addb_flag */ + + sps->sps_alf_flag = get_bits(&gb, 1); + + skip_bits_long(&gb, 1); /* skip sps->sps_htdf_flag */ + + sps->sps_rpl_flag = get_bits(&gb, 1); + sps->sps_pocs_flag = get_bits(&gb, 1); + + skip_bits_long(&gb, 1); /* skip sps_dquant_flag */ + skip_bits_long(&gb, 1); /* skip sps_dra_flag */ + + if (sps->sps_pocs_flag) + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); + + if (!sps->sps_pocs_flag || !sps->sps_rpl_flag) + sps->log2_sub_gop_length = get_ue_golomb(&gb); + + return sps; +} + +// @see ISO_IEC_23094-1 (7.3.2.2 SPS RBSP syntax) +// +// @note +// The current implementation of parse_sps function doesn't handle VUI parameters parsing. +// If it will be needed, parse_sps function could be extended to handle VUI parameters parsing +// to initialize fields of the AVCodecContex i.e. color_primaries, color_trc,color_range +// +static EVCParserPPS *parse_pps(const uint8_t *bs, int bs_size, EVCMergeContext *ctx) +{ + GetBitContext gb; + EVCParserPPS *pps; + + int pps_pic_parameter_set_id; + + if (init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + pps_pic_parameter_set_id = get_ue_golomb(&gb); + if (pps_pic_parameter_set_id > EVC_MAX_PPS_COUNT) + return NULL; + + if(!ctx->pps[pps_pic_parameter_set_id]) { + if((ctx->pps[pps_pic_parameter_set_id] = av_malloc(sizeof(EVCParserSPS))) == NULL) + return NULL; + } + + pps = ctx->pps[pps_pic_parameter_set_id]; + + pps->pps_pic_parameter_set_id = pps_pic_parameter_set_id; + + pps->pps_seq_parameter_set_id = get_ue_golomb(&gb); + if (pps->pps_seq_parameter_set_id >= EVC_MAX_SPS_COUNT) + return NULL; + + get_ue_golomb(&gb); /* skip num_ref_idx_default_active_minus1[0] */ + get_ue_golomb(&gb); /* skip num_ref_idx_default_active_minus1[1] */ + get_ue_golomb(&gb); /* skip additional_lt_poc_lsb_len */ + skip_bits_long(&gb, 1); /* skip rpl1_idx_present_flag */ + pps->single_tile_in_pic_flag = get_bits(&gb, 1); + + if (!pps->single_tile_in_pic_flag) { + pps->num_tile_columns_minus1 = get_ue_golomb(&gb); + pps->num_tile_rows_minus1 = get_ue_golomb(&gb); + pps->uniform_tile_spacing_flag = get_bits(&gb, 1); + + if (!pps->uniform_tile_spacing_flag) { + for (int i = 0; i < pps->num_tile_columns_minus1; i++) + get_ue_golomb(&gb); /* skip tile_column_width_minus1[i] */ + + for (int i = 0; i < pps->num_tile_rows_minus1; i++) + get_ue_golomb(&gb); /* skip tile_row_height_minus1[i] */ + } + skip_bits_long(&gb, 1); /* skip loop_filter_across_tiles_enabled_flag */ + get_ue_golomb(&gb); /* skip tile_offset_len_minus1 */ + } + + pps->tile_id_len_minus1 = get_ue_golomb(&gb); + pps->explicit_tile_id_flag = get_bits(&gb, 1); + + if (pps->explicit_tile_id_flag) { + for (int i = 0; i <= pps->num_tile_rows_minus1; i++) { + for (int j = 0; j <= pps->num_tile_columns_minus1; j++) + skip_bits_long(&gb, pps->tile_id_len_minus1 + 1); /* skip tile_id_val[i][j] */; + } + } + + pps->pic_dra_enabled_flag = 0; + pps->pic_dra_enabled_flag = get_bits(&gb, 1); + + if (pps->pic_dra_enabled_flag) + skip_bits_long(&gb, 5); /* skip pic_dra_aps_id */ + + pps->arbitrary_slice_present_flag = get_bits(&gb, 1); + + return pps; +} + +// @see ISO_IEC_23094-1 (7.3.2.6 Slice layer RBSP syntax) +static EVCParserSliceHeader *parse_slice_header(const uint8_t *bs, int bs_size, EVCMergeContext *ctx) +{ + GetBitContext gb; + EVCParserSliceHeader *sh; + EVCParserPPS *pps; + EVCParserSPS *sps; + + int num_tiles_in_slice = 0; + int slice_pic_parameter_set_id; + + if (init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + slice_pic_parameter_set_id = get_ue_golomb(&gb); + + if (slice_pic_parameter_set_id < 0 || slice_pic_parameter_set_id >= EVC_MAX_PPS_COUNT) + return NULL; + + if(!ctx->slice_header[slice_pic_parameter_set_id]) { + if((ctx->slice_header[slice_pic_parameter_set_id] = av_malloc(sizeof(EVCParserSliceHeader))) == NULL) + return NULL; + } + + sh = ctx->slice_header[slice_pic_parameter_set_id]; + + pps = ctx->pps[slice_pic_parameter_set_id]; + if(!pps) + return NULL; + + sps = ctx->sps[slice_pic_parameter_set_id]; + if(!sps) + return NULL; + + sh->slice_pic_parameter_set_id = slice_pic_parameter_set_id; + + if (!pps->single_tile_in_pic_flag) { + sh->single_tile_in_slice_flag = get_bits(&gb, 1); + skip_bits_long(&gb, pps->tile_id_len_minus1 + 1); /* skip first_tile_id */ + } else + sh->single_tile_in_slice_flag = 1; + + if (!sh->single_tile_in_slice_flag) { + if (pps->arbitrary_slice_present_flag) + sh->arbitrary_slice_flag = get_bits(&gb, 1); + + if (!sh->arbitrary_slice_flag) + skip_bits_long(&gb, pps->tile_id_len_minus1 + 1); /* skip last_tile_id */ + else { + sh->num_remaining_tiles_in_slice_minus1 = get_ue_golomb(&gb); + num_tiles_in_slice = sh->num_remaining_tiles_in_slice_minus1 + 2; + for (int i = 0; i < num_tiles_in_slice - 1; ++i) + get_ue_golomb(&gb); /* skip delta_tile_id_minus1[i] */ + } + } + + sh->slice_type = get_ue_golomb(&gb); + + if (ctx->nalu_type == EVC_IDR_NUT) + skip_bits_long(&gb, 1); /* skip no_output_of_prior_pics_flag */ + + if (sps->sps_mmvd_flag && ((sh->slice_type == EVC_SLICE_TYPE_B) || (sh->slice_type == EVC_SLICE_TYPE_P))) + skip_bits_long(&gb, 1); /* skip mmvd_group_enable_flag */ + // else + // sh->mmvd_group_enable_flag = 0; + + if (sps->sps_alf_flag) { + int ChromaArrayType = sps->chroma_format_idc; + + sh->slice_alf_enabled_flag = get_bits(&gb, 1); + + if (sh->slice_alf_enabled_flag) { + sh->slice_alf_luma_aps_id = get_bits(&gb, 5); + sh->slice_alf_map_flag = get_bits(&gb, 1); + sh->slice_alf_chroma_idc = get_bits(&gb, 2); + + if ((ChromaArrayType == 1 || ChromaArrayType == 2) && sh->slice_alf_chroma_idc > 0) + sh->slice_alf_chroma_aps_id = get_bits(&gb, 5); + } + if (ChromaArrayType == 3) { + int sliceChromaAlfEnabledFlag = 0; + int sliceChroma2AlfEnabledFlag = 0; + + if (sh->slice_alf_chroma_idc == 1) { // @see ISO_IEC_23094-1 (7.4.5) + sliceChromaAlfEnabledFlag = 1; + sliceChroma2AlfEnabledFlag = 0; + } else if (sh->slice_alf_chroma_idc == 2) { + sliceChromaAlfEnabledFlag = 0; + sliceChroma2AlfEnabledFlag = 1; + } else if (sh->slice_alf_chroma_idc == 3) { + sliceChromaAlfEnabledFlag = 1; + sliceChroma2AlfEnabledFlag = 1; + } else { + sliceChromaAlfEnabledFlag = 0; + sliceChroma2AlfEnabledFlag = 0; + } + + if (!sh->slice_alf_enabled_flag) + sh->slice_alf_chroma_idc = get_bits(&gb, 2); + + if (sliceChromaAlfEnabledFlag) { + skip_bits_long(&gb, 5); /* skip slice_alf_chroma_aps_id */ + skip_bits_long(&gb, 1); /* skip slice_alf_chroma_map_flag */ + } + + if (sliceChroma2AlfEnabledFlag) { + skip_bits_long(&gb, 5); /* skip slice_alf_chroma2_aps_id */ + skip_bits_long(&gb, 1); /* skip slice_alf_chroma2_map_flag */ + } + } + } + + if (ctx->nalu_type != EVC_IDR_NUT) { + if (sps->sps_pocs_flag) + sh->slice_pic_order_cnt_lsb = get_bits(&gb, sps->log2_max_pic_order_cnt_lsb_minus4 + 4); + } + + // @note + // If necessary, add the missing fields to the EVCParserSliceHeader structure + // and then extend parser implementation + + return sh; +} + +static int parse_nal_unit(const uint8_t *buf, size_t buf_size, AVBSFContext *s) +{ + int nalu_type, nalu_size; + int tid; + const uint8_t *data = buf; + int data_size = buf_size; + EVCMergeContext *ctx = s->priv_data; + + nalu_size = buf_size; + if (nalu_size <= 0) { + av_log(s, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); + return AVERROR_INVALIDDATA; + } + + // @see ISO_IEC_23094-1_2020, 7.4.2.2 NAL unit header semantic (Table 4 - NAL unit type codes and NAL unit type classes) + // @see enum EVCNALUnitType in evc.h + nalu_type = get_nalu_type(data, data_size); + if (nalu_type < EVC_NOIDR_NUT || nalu_type > EVC_UNSPEC_NUT62) { + av_log(s, AV_LOG_ERROR, "Invalid NAL unit type: (%d)\n", nalu_type); + return AVERROR_INVALIDDATA; + } + ctx->nalu_type = nalu_type; + + tid = get_temporal_id(data, data_size); + if (tid < 0) { + av_log(s, AV_LOG_ERROR, "Invalid temporial id: (%d)\n", tid); + return AVERROR_INVALIDDATA; + } + + data += EVC_NALU_HEADER_SIZE; + data_size -= EVC_NALU_HEADER_SIZE; + + switch(nalu_type) { + case EVC_SPS_NUT: { + EVCParserSPS *sps; + + sps = parse_sps(data, nalu_size, ctx); + if (!sps) { + av_log(s, AV_LOG_ERROR, "SPS parsing error\n"); + return AVERROR_INVALIDDATA; + } + + if (sps->profile_idc == 1) ctx->profile = FF_PROFILE_EVC_MAIN; + else ctx->profile = FF_PROFILE_EVC_BASELINE; + + break; + } + case EVC_PPS_NUT: { + EVCParserPPS *pps; + + pps = parse_pps(data, nalu_size, ctx); + if (!pps) { + av_log(s, AV_LOG_ERROR, "PPS parsing error\n"); + return AVERROR_INVALIDDATA; + } + break; + } + case EVC_SEI_NUT: // Supplemental Enhancement Information + case EVC_APS_NUT: // Adaptation parameter set + case EVC_FD_NUT: // Filler data + break; + case EVC_IDR_NUT: // Coded slice of a IDR or non-IDR picture + case EVC_NOIDR_NUT: { + EVCParserSliceHeader *sh; + EVCParserSPS *sps; + int slice_pic_parameter_set_id; + + sh = parse_slice_header(data, nalu_size, ctx); + if (!sh) { + av_log(s, AV_LOG_ERROR, "Slice header parsing error\n"); + return AVERROR_INVALIDDATA; + } + + ctx->key_frame = (nalu_type == EVC_IDR_NUT) ? 1 : 0; + + // POC (picture order count of the current picture) derivation + // @see ISO/IEC 23094-1:2020(E) 8.3.1 Decoding process for picture order count + slice_pic_parameter_set_id = sh->slice_pic_parameter_set_id; + sps = ctx->sps[slice_pic_parameter_set_id]; + + if (sps && sps->sps_pocs_flag) { + + int PicOrderCntMsb = 0; + ctx->poc.prevPicOrderCntVal = ctx->poc.PicOrderCntVal; + + if (nalu_type == EVC_IDR_NUT) + PicOrderCntMsb = 0; + else { + int MaxPicOrderCntLsb = 1 << (sps->log2_max_pic_order_cnt_lsb_minus4 + 4); + + int prevPicOrderCntLsb = ctx->poc.PicOrderCntVal & (MaxPicOrderCntLsb - 1); + int prevPicOrderCntMsb = ctx->poc.PicOrderCntVal - prevPicOrderCntLsb; + + + if ((sh->slice_pic_order_cnt_lsb < prevPicOrderCntLsb) && + ((prevPicOrderCntLsb - sh->slice_pic_order_cnt_lsb) >= (MaxPicOrderCntLsb / 2))) + + PicOrderCntMsb = prevPicOrderCntMsb + MaxPicOrderCntLsb; + + else if ((sh->slice_pic_order_cnt_lsb > prevPicOrderCntLsb) && + ((sh->slice_pic_order_cnt_lsb - prevPicOrderCntLsb) > (MaxPicOrderCntLsb / 2))) + + PicOrderCntMsb = prevPicOrderCntMsb - MaxPicOrderCntLsb; + + else + PicOrderCntMsb = prevPicOrderCntMsb; + } + ctx->poc.PicOrderCntVal = PicOrderCntMsb + sh->slice_pic_order_cnt_lsb; + + } else { + if (nalu_type == EVC_IDR_NUT) { + ctx->poc.PicOrderCntVal = 0; + ctx->poc.DocOffset = -1; + } else { + int SubGopLength = (int)pow(2.0, sps->log2_sub_gop_length); + if (tid == 0) { + ctx->poc.PicOrderCntVal = ctx->poc.prevPicOrderCntVal + SubGopLength; + ctx->poc.DocOffset = 0; + ctx->poc.prevPicOrderCntVal = ctx->poc.PicOrderCntVal; + } else { + int ExpectedTemporalId; + int PocOffset; + int prevDocOffset = ctx->poc.DocOffset; + + ctx->poc.DocOffset = (prevDocOffset + 1) % SubGopLength; + if (ctx->poc.DocOffset == 0) { + ctx->poc.prevPicOrderCntVal += SubGopLength; + ExpectedTemporalId = 0; + } else + ExpectedTemporalId = 1 + (int)log2(ctx->poc.DocOffset); + while (tid != ExpectedTemporalId) { + ctx->poc.DocOffset = (ctx->poc.DocOffset + 1) % SubGopLength; + if (ctx->poc.DocOffset == 0) + ExpectedTemporalId = 0; + else + ExpectedTemporalId = 1 + (int)log2(ctx->poc.DocOffset); + } + PocOffset = (int)(SubGopLength * ((2.0 * ctx->poc.DocOffset + 1) / (int)pow(2.0, tid) - 2)); + ctx->poc.PicOrderCntVal = ctx->poc.prevPicOrderCntVal + PocOffset; + } + } + } + + break; + } + } + + return 0; +} + +static void evc_frame_merge_flush(AVBSFContext *bsf) +{ + EVCMergeContext *ctx = bsf->priv_data; + + av_packet_unref(ctx->in); +} + +static int evc_frame_merge_filter(AVBSFContext *bsf, AVPacket *out) +{ + EVCMergeContext *ctx = bsf->priv_data; + AVPacket *in = ctx->in; + + int free_space = 0; + size_t nalu_size = 0; + uint8_t *nalu = NULL; + int au_end_found = 0; + int err; + + err = ff_bsf_get_packet_ref(bsf, in); + if (err < 0) + return err; + + nalu_size = read_nal_unit_length(in->data, EVC_NALU_LENGTH_PREFIX_SIZE); + if(nalu_size <= 0) { + av_packet_unref(in); + return AVERROR_INVALIDDATA; + } + + nalu = in->data + EVC_NALU_LENGTH_PREFIX_SIZE; + nalu_size = in->size - EVC_NALU_LENGTH_PREFIX_SIZE; + + // NAL unit parsing needed to determine if end of AU was found + err = parse_nal_unit(nalu, nalu_size, bsf); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "NAL Unit parsing error\n"); + av_packet_unref(in); + + return err; + } + + au_end_found = end_of_access_unit_found(bsf); + + free_space = ctx->au_buffer.capacity - ctx->au_buffer.data_size; + while( free_space < in->size ) { + ctx->au_buffer.capacity *= 2; + free_space = ctx->au_buffer.capacity - ctx->au_buffer.data_size; + + if(free_space >= in->size) { + ctx->au_buffer.data = av_realloc(ctx->au_buffer.data, ctx->au_buffer.capacity); + } + } + + memcpy(ctx->au_buffer.data + ctx->au_buffer.data_size, in->data, in->size); + + ctx->au_buffer.data_size += in->size; + + av_packet_unref(in); + + if(au_end_found) { + uint8_t *data = av_memdup(ctx->au_buffer.data, ctx->au_buffer.data_size); + err = av_packet_from_data(out, data, ctx->au_buffer.data_size); + + ctx->au_buffer.data_size = 0; + } else + err = AVERROR(EAGAIN); + + if (err < 0 && err != AVERROR(EAGAIN)) + evc_frame_merge_flush(bsf); + + return err; +} + +static int evc_frame_merge_init(AVBSFContext *bsf) +{ + EVCMergeContext *ctx = bsf->priv_data; + + ctx->in = av_packet_alloc(); + if (!ctx->in) + return AVERROR(ENOMEM); + + ctx->au_buffer.capacity = INIT_AU_BUF_CAPACITY; + ctx->au_buffer.data = av_malloc(INIT_AU_BUF_CAPACITY); + ctx->au_buffer.data_size = 0; + + return 0; +} + +static void evc_frame_merge_close(AVBSFContext *bsf) +{ + EVCMergeContext *ctx = bsf->priv_data; + + av_packet_free(&ctx->in); + + ctx->au_buffer.capacity = 0; + av_freep(&ctx->au_buffer.data); + ctx->au_buffer.data_size = 0; +} + +static const enum AVCodecID evc_frame_merge_codec_ids[] = { + AV_CODEC_ID_EVC, AV_CODEC_ID_NONE, +}; + +const FFBitStreamFilter ff_evc_frame_merge_bsf = { + .p.name = "evc_frame_merge", + .p.codec_ids = evc_frame_merge_codec_ids, + .priv_data_size = sizeof(EVCMergeContext), + .init = evc_frame_merge_init, + .flush = evc_frame_merge_flush, + .close = evc_frame_merge_close, + .filter = evc_frame_merge_filter, +}; \ No newline at end of file diff --git a/libavformat/Makefile b/libavformat/Makefile index a734ad64ca..bd717dc6b5 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -251,6 +251,7 @@ OBJS-$(CONFIG_HCOM_DEMUXER) += hcom.o pcm.o OBJS-$(CONFIG_HDS_MUXER) += hdsenc.o OBJS-$(CONFIG_HEVC_DEMUXER) += hevcdec.o rawdec.o OBJS-$(CONFIG_HEVC_MUXER) += rawenc.o +OBJS-$(CONFIG_EVC_DEMUXER) += evcdec.o rawdec.o OBJS-$(CONFIG_EVC_MUXER) += rawenc.o OBJS-$(CONFIG_HLS_DEMUXER) += hls.o hls_sample_encryption.o OBJS-$(CONFIG_HLS_MUXER) += hlsenc.o hlsplaylist.o avc.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index 2ad142d8f7..eb14c79bfa 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -154,6 +154,7 @@ extern const AVInputFormat ff_ea_cdata_demuxer; extern const AVInputFormat ff_eac3_demuxer; extern const FFOutputFormat ff_eac3_muxer; extern const AVInputFormat ff_epaf_demuxer; +extern const AVInputFormat ff_evc_demuxer; extern const FFOutputFormat ff_evc_muxer; extern const FFOutputFormat ff_f4v_muxer; extern const AVInputFormat ff_ffmetadata_demuxer; diff --git a/libavformat/evcdec.c b/libavformat/evcdec.c new file mode 100644 index 0000000000..b619576128 --- /dev/null +++ b/libavformat/evcdec.c @@ -0,0 +1,334 @@ +/* + * RAW EVC video demuxer + * + * Copyright (c) 2021 Dawid Kozinski + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavcodec/get_bits.h" +#include "libavcodec/golomb.h" +#include "libavcodec/internal.h" +#include "libavcodec/evc.h" +#include "libavcodec/bsf.h" + +#include "libavutil/opt.h" + +#include "rawdec.h" +#include "avformat.h" +#include "internal.h" + + +#define RAW_PACKET_SIZE 1024 + +typedef struct EVCParserContext { + int got_sps; + int got_pps; + int got_idr; + int got_nonidr; + +} EVCParserContext; + +typedef struct EVCDemuxContext { + const AVClass *class; + AVRational framerate; + + AVBSFContext *bsf; + +} EVCDemuxContext; + +#define DEC AV_OPT_FLAG_DECODING_PARAM +#define OFFSET(x) offsetof(EVCDemuxContext, x) +static const AVOption evc_options[] = { + { "framerate", "", OFFSET(framerate), AV_OPT_TYPE_VIDEO_RATE, {.str = "25"}, 0, INT_MAX, DEC}, + { NULL }, +}; +#undef OFFSET + +static const AVClass evc_demuxer_class = { + .class_name = "EVC Annex B demuxer", + .item_name = av_default_item_name, + .option = evc_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +static int get_nalu_type(const uint8_t *bits, int bits_size) +{ + int unit_type_plus1 = 0; + + if (bits_size >= EVC_NALU_HEADER_SIZE) { + unsigned char *p = (unsigned char *)bits; + // forbidden_zero_bit + if ((p[0] & 0x80) != 0) // Cannot get bitstream information. Malformed bitstream. + return -1; + + // nal_unit_type + unit_type_plus1 = (p[0] >> 1) & 0x3F; + } + + return unit_type_plus1 - 1; +} + +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size) +{ + uint32_t nalu_len = 0; + + if (bits_size >= EVC_NALU_LENGTH_PREFIX_SIZE) { + + int t = 0; + unsigned char *p = (unsigned char *)bits; + + for (int i = 0; i < EVC_NALU_LENGTH_PREFIX_SIZE; i++) + t = (t << 8) | p[i]; + + nalu_len = t; + if (nalu_len == 0) // Invalid bitstream size + return 0; + } + + return nalu_len; +} + +static int parse_nal_units(const AVProbeData *p, EVCParserContext *ev) +{ + int nalu_type; + size_t nalu_size; + unsigned char *bits = (unsigned char *)p->buf; + int bytes_to_read = p->buf_size; + + while (bytes_to_read > EVC_NALU_LENGTH_PREFIX_SIZE) { + + nalu_size = read_nal_unit_length(bits, EVC_NALU_LENGTH_PREFIX_SIZE); + if (nalu_size == 0) break; + + bits += EVC_NALU_LENGTH_PREFIX_SIZE; + bytes_to_read -= EVC_NALU_LENGTH_PREFIX_SIZE; + + if(bytes_to_read < nalu_size) break; + + nalu_type = get_nalu_type(bits, bytes_to_read); + + if (nalu_type == EVC_SPS_NUT) + ev->got_sps++; + else if (nalu_type == EVC_PPS_NUT) + ev->got_pps++; + else if (nalu_type == EVC_IDR_NUT ) + ev->got_idr++; + else if (nalu_type == EVC_NOIDR_NUT) + ev->got_nonidr++; + + bits += nalu_size; + bytes_to_read -= nalu_size; + } + + return 0; +} + +static int annexb_probe(const AVProbeData *p) +{ + EVCParserContext ev = {0}; + int ret = parse_nal_units(p, &ev); + + if (ret == 0 && ev.got_sps && ev.got_pps && (ev.got_idr || ev.got_nonidr > 3)) + return AVPROBE_SCORE_EXTENSION + 1; // 1 more than .mpg + + return 0; +} + +static int annexb_read_header(AVFormatContext *s) +{ + AVStream *st; + FFStream *sti; + EVCDemuxContext *c = s->priv_data; + int ret = 0; + + st = avformat_new_stream(s, NULL); + if (!st) { + ret = AVERROR(ENOMEM); + goto fail; + } + sti = ffstream(st); + + st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; + st->codecpar->codec_id = AV_CODEC_ID_EVC; + sti->need_parsing = AVSTREAM_PARSE_FULL_RAW; + + st->avg_frame_rate = c->framerate; + sti->avctx->framerate = c->framerate; + + // taken from rawvideo demuxers + avpriv_set_pts_info(st, 64, 1, 1200000); + +fail: + return ret; +} + +static int evc_read_header(AVFormatContext *s) +{ + AVStream *st; + FFStream *sti; + const AVBitStreamFilter *filter = av_bsf_get_by_name("evc_frame_merge"); + EVCDemuxContext *c = s->priv_data; + int ret = 0; + + st = avformat_new_stream(s, NULL); + if (!st) { + ret = AVERROR(ENOMEM); + goto fail; + } + sti = ffstream(st); + + st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; + st->codecpar->codec_id = AV_CODEC_ID_EVC; + + // This causes sending to the parser full frames, not chunks of data + // The flag PARSER_FLAG_COMPLETE_FRAMES will be set in demux.c (demux.c: 1316) + sti->need_parsing = AVSTREAM_PARSE_HEADERS; + + st->avg_frame_rate = c->framerate; + sti->avctx->framerate = c->framerate; + + // taken from rawvideo demuxers + avpriv_set_pts_info(st, 64, 1, 1200000); + + ret = av_bsf_alloc(filter, &c->bsf); + if (ret < 0) + return ret; + + ret = avcodec_parameters_copy(c->bsf->par_in, st->codecpar); + if (ret < 0) + return ret; + + ret = av_bsf_init(c->bsf); + if (ret < 0) + return ret; + +fail: + return ret; +} + +static int annexb_read_packet(AVFormatContext *s, AVPacket *pkt) +{ + int ret, pkt_size; + int eof; + + pkt_size = RAW_PACKET_SIZE; + + eof = avio_feof (s->pb); + if(eof) { + av_packet_unref(pkt); + return AVERROR_EOF; + } + + if ((ret = av_new_packet(pkt, pkt_size)) < 0) + return ret; + + pkt->pos = avio_tell(s->pb); + pkt->stream_index = 0; + ret = avio_read_partial(s->pb, pkt->data, pkt_size); + if (ret < 0) { + av_packet_unref(pkt); + return ret; + } + av_shrink_packet(pkt, ret); + + av_log(s, AV_LOG_ERROR, "annexb_read_packet: %d\n", pkt_size); + + return ret; +} + +static int evc_read_packet(AVFormatContext *s, AVPacket *pkt) +{ + int ret; + int32_t nalu_size; + int au_end_found; + + EVCDemuxContext *const c = s->priv_data; + + int eof = avio_feof (s->pb); + if(eof) { + av_packet_unref(pkt); + return AVERROR_EOF; + } + + au_end_found = 0; + + while(!au_end_found) { + + uint8_t buf[EVC_NALU_LENGTH_PREFIX_SIZE]; + ret = avio_read(s->pb, (unsigned char *)&buf, EVC_NALU_LENGTH_PREFIX_SIZE); + if (ret < 0) { + av_packet_unref(pkt); + return ret; + } + + nalu_size = read_nal_unit_length((const uint8_t *)&buf, EVC_NALU_LENGTH_PREFIX_SIZE); + if(nalu_size <= 0) { + av_packet_unref(pkt); + return -1; + } + + avio_seek(s->pb, -EVC_NALU_LENGTH_PREFIX_SIZE, SEEK_CUR); + + ret = av_get_packet(s->pb, pkt, nalu_size + EVC_NALU_LENGTH_PREFIX_SIZE); + if (ret < 0) + return ret; + if (ret != (nalu_size + EVC_NALU_LENGTH_PREFIX_SIZE)) + return AVERROR(EIO); + + ret = av_bsf_send_packet(c->bsf, pkt); + if (ret < 0) { + av_log(s, AV_LOG_ERROR, "Failed to send packet to " + "evc_frame_merge filter\n"); + return ret; + } + + ret = av_bsf_receive_packet(c->bsf, pkt); + if (ret < 0 && ret != AVERROR(EAGAIN) && ret != AVERROR_EOF) + av_log(s, AV_LOG_ERROR, "evc_frame_merge filter failed to " + "send output packet\n"); + + au_end_found = 1; + if (ret == AVERROR(EAGAIN)) + au_end_found = 0; + } + + return ret; +} + +static int evc_read_close(AVFormatContext *s) +{ + EVCDemuxContext *const c = s->priv_data; + + av_bsf_free(&c->bsf); + return 0; +} + +const AVInputFormat ff_evc_demuxer = { + .name = "evc", + .long_name = NULL_IF_CONFIG_SMALL("EVC Annex B"), + .read_probe = annexb_probe, + .read_header = evc_read_header, // annexb_read_header + .read_packet = evc_read_packet, // annexb_read_packet + .read_close = evc_read_close, + .extensions = "evc", + .flags = AVFMT_GENERIC_INDEX, + .flags_internal = FF_FMT_INIT_CLEANUP, + .raw_codec_id = AV_CODEC_ID_EVC, + .priv_data_size = sizeof(EVCDemuxContext), + .priv_class = &evc_demuxer_class, +};