From patchwork Thu Aug 11 12:36:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dawid Kozinski X-Patchwork-Id: 37236 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3d0d:b0:8d:a68e:8a0e with SMTP id y13csp335258pzi; Thu, 11 Aug 2022 05:37:12 -0700 (PDT) X-Google-Smtp-Source: AA6agR5WFcvB21PXriLnAc5bNM1WxHyMI9FzWNCXO1QKC7IrbtvCFoVxHhtJ7A9/CElvHBGVwqUI X-Received: by 2002:a05:6402:294c:b0:43a:91a9:a691 with SMTP id ed12-20020a056402294c00b0043a91a9a691mr31614771edb.182.1660221432395; Thu, 11 Aug 2022 05:37:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1660221432; cv=none; d=google.com; s=arc-20160816; b=ouRl/eJtuEylsZO4bwrWa8dV9Gl5wEuOSdt1InW1R/QQvKDLRAGnr6/jxfoPopJze0 o1hvn/v+DwxoH/CgmjhtWpdkDGWqkhZmkgRWwRPJpxWpVz30dKILFjUuDlHMlCiddiU4 dPrqNiu1kpezm/v30E+DP+yy26/EFvuoC6GybAPxrcODFkLNBYws16tMN3zW9yymNcPr uRj07y+b0UNIseNB0SoQtgcAPXStiaFMfspGgi8R0LSRzdmc+w77G/cL0GKW7URpeljU vzU43cMkvlOzav/D31gQkUljOp6DLJ3M1qECQbA07n/pF10mw2rCrhdqQ8NzhhJ0Hq6q 0bUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:references:cms-type:content-language :thread-index:mime-version:message-id:date:to:from:dkim-signature :dkim-filter:delivered-to; bh=4kEouZ8FkFtL6XfBQioRBb5c7V1o5+SrCj9c/9oNbj8=; b=0h01ppaYEaCTaGJoP1FBc24ydps9ieuTwfT5qV+K/1zloSRTOYBeNxTBRJKvdlnGP7 ggbVAW9cw0EB7KezA7KMMsaotnzB9l/UMUuStdTo9LkwhayE1BYTKP76MBBup65e+vRy ws0q1uIoBNqjfWdNf87y5kF8k5ECokRL6wU5qpjvz5WtHEnAdQGuWLCWG8BtGhwg72gD xceTF2WShFCDo6DKRpTRj4bC/1ERY0LGC+KtwYvANs9pGCTr7qXs3fGCUfKvlsYlwwB8 qiGDrLr2EN3IznpGFkGr2o0Tre4XIQn0H76n+qqNCdxeGoNy4EwdME5eJ8mVrAmJKHiG LURA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@samsung.com header.s=mail20170921 header.b=AHQimgte; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=samsung.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o11-20020a170906974b00b00730a4246dd0si5665956ejy.593.2022.08.11.05.37.11; Thu, 11 Aug 2022 05:37:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@samsung.com header.s=mail20170921 header.b=AHQimgte; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=samsung.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 53A6068B939; Thu, 11 Aug 2022 15:36:56 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0E6C968B926 for ; Thu, 11 Aug 2022 15:36:50 +0300 (EEST) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20220811123648euoutp01f9437187fc3c14e32bc3bf7d64f50c46~KSpMy0VPZ2740327403euoutp01g for ; Thu, 11 Aug 2022 12:36:48 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20220811123648euoutp01f9437187fc3c14e32bc3bf7d64f50c46~KSpMy0VPZ2740327403euoutp01g DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1660221408; bh=5IWcM4DgRo3ixgJlwGNrtVWHNFVwkmLrfhdSM+thFes=; h=From:To:Subject:Date:References:From; b=AHQimgteKOlkhjjIPWPhi8QdlH7Q1VxE2xjaPNZi/F7UN/hvnF65n05O9nG4PZr68 m0MRtWubluTFaijgdQsdmrinVpgl6RMsdc6gKcmSUdHfpQZP7sAbP+cowzyRbRv8hg xq8IPwLblgwLqDY4CQDWKg7IKvTnBDqgDwzD6/X8= Received: from eusmges3new.samsung.com (unknown [203.254.199.245]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20220811123648eucas1p285c87b51dcabfa57c71e5f60ce24a082~KSpMq50NP2127321273eucas1p2i for ; Thu, 11 Aug 2022 12:36:48 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges3new.samsung.com (EUCPMTA) with SMTP id AE.95.09580.0E7F4F26; Thu, 11 Aug 2022 13:36:48 +0100 (BST) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20220811123648eucas1p1336fd2c107be031a4a274057bf20084f~KSpMXpoZ01459614596eucas1p19 for ; Thu, 11 Aug 2022 12:36:48 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20220811123648eusmtrp153bfff7fd9efeb796d6df27cb252e8eb~KSpMXGAdD1618816188eusmtrp1K for ; Thu, 11 Aug 2022 12:36:48 +0000 (GMT) X-AuditID: cbfec7f5-9c3ff7000000256c-75-62f4f7e054e4 Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id C1.6E.09038.0E7F4F26; Thu, 11 Aug 2022 13:36:48 +0100 (BST) Received: from AMDN3260 (unknown [106.210.132.171]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20220811123647eusmtip1e49b17c611a45bd46f45bafa4ed6bee3~KSpMDlcMD2469924699eusmtip1Y for ; Thu, 11 Aug 2022 12:36:47 +0000 (GMT) From: "Dawid Kozinski" To: Date: Thu, 11 Aug 2022 14:36:47 +0200 Message-ID: <006a01d8ad7f$05f788f0$11e69ad0$@samsung.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 Thread-Index: Aditfo7ChTWvHrYNSWeR75mqCNLGwQ== Content-Language: pl X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrBIsWRmVeSWpSXmKPExsWy7djP87oPvn9JMjg3XcHi26czzA6MHn8W bWYJYIzisklJzcksSy3St0vgylj8/i9bwYkbTBXn1i9hbWA8eZKpi5GTQ0LARGLjuglANheH kMAKRolZv7qhnElMEgumHGeFcCYySezevpQFpuV15242iMRyRol5zcsZIZw2JolNrVMYQarY BPQlZi76xAZiiwjISqz+NwXMFhYIlZj8+CeYzSKgKjF90RmgfRwcvAKWEj1H4kDCvAKCEidn PgFbxiygLbFs4WtmiMUKEj+fLmMFKRcR0JOYuT8cokRE4sajFkaIkp/sEp2N1hC2i8TGq31Q cWGJV8e3sEPYMhKnJ/ewgIyRECiWONTvAGHWSBz6kQ5RYS3xtvE4I0iYWUBTYv0ufYiwo8SO rhNsENV8EjfeCkLs55OYtG06M0SYV6KjTQjCVJHo6xSDaJSSeLpsDtQTHhI3vp9lnMCoOAvJ r7OQ/DoLyVOzEE5YwMiyilE8tbQ4Nz212DgvtVyvODG3uDQvXS85P3cTIzAxnP53/OsOxhWv PuodYmTiYDzEKMHBrCTCW7boc5IQb0piZVVqUX58UWlOavEhRmkOFiVx3uTMDYlCAumJJanZ qakFqUUwWSYOTqkGJjVWtvCIsjjOdcLczkFLGb9vbTY4stF71jnuPZKuT9rZ1tm+P8+6Q3Df 1TONJ5x+TtSV6Fr7Ib/tZMcZwT+iGcd+XKl45z5LbqX6xrbV2qdZuHfV/vm5fltncsGK/kmF wv/0ue1KdaaErsi6e3ibUIL6jsB/N94xnfl2lKH0+4TrVy2jz7FoZF2sWPBnbtGLmcUhNc0P NXcG9XAwlU5/qXH+a/P9aR2Kb/IMbm6elVasaOHHUf7r8D6nz5+z5zb1OXh4rP4s0L+Ei5GT 9eOEmksVydGHjynF895/ceFp+553E1fNX6csJHnQZovEn7v7Db/0yW520Ltz5m6u67SHYWu8 dqwMXXrv8fQju0s957oqsRRnJBpqMRcVJwIAItBR3XsDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrGLMWRmVeSWpSXmKPExsVy+t/xu7oPvn9JMtg/kd3i26czzA6MHn8W bWYJYIzSsynKLy1JVcjILy6xVYo2tDDSM7S00DMysdQzNDaPtTIyVdK3s0lJzcksSy3St0vQ yzh/9ytjwfRbjBWTlss3MB6ZzdjFyMkhIWAi8bpzN1sXIxeHkMBSRon/ax6wQSSkJJYuXQRV JCzx51oXVFELk0TzoudMIAk2AX2JmYs+gTWICMhKrP43BcwWFgiWeHzmC1gzi4CqxPRFZ4Dq OTh4BSwleo7EgYR5BQQlTs58wgJiMwtoSzy9+RTOXrbwNTPEXgWJn0+XsYK0igjoSczcHw5R IiJx41EL4wRGgVlIJs1CMmkWkkmzkLQsYGRZxSiSWlqcm55bbKRXnJhbXJqXrpecn7uJERjg 24793LKDceWrj3qHGJk4GA8xSnAwK4nwli36nCTEm5JYWZValB9fVJqTWnyI0RTos4nMUqLJ +cAYyyuJNzQzMDU0MbM0MLU0M1YS5/Us6EgUEkhPLEnNTk0tSC2C6WPi4JRqYJoZ+YS17Ppv pj9qsQ1FzZuP5u1rY8zf5ODxSuqbr0RwYr/TxcU5Bknftzw17vonNOswexLLi3s9ebI/W81m HHP8eHy7zlO9LVt96tOjCpc9m95vE+X/Wsl/WlSfIKuo1/tq2XqOJ0kMhz4r3L6gHZbv8GWG 0sbOD6pOLBK9fsefn3PaP+3dZF+bZVoXtp0LVbJcmftk6z7x1k0qIe9zN2WION5x6FiRtWpl wxo+M1HpM4dEo1XjL+yO/ffD0PDtvrs571e4H2SJmz9x84N526+8maowa1Pjjz92YlMXbLLb 7zNfcJ+duOnOL1ZvHr5R53g2d0PL9rtf5h9udEmu2lBX7n3zjeT1jaGmLhqShd5KLMUZiYZa zEXFiQD8hRc1+QIAAA== X-CMS-MailID: 20220811123648eucas1p1336fd2c107be031a4a274057bf20084f X-Msg-Generator: CA X-RootMTR: 20220811123648eucas1p1336fd2c107be031a4a274057bf20084f X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20220811123648eucas1p1336fd2c107be031a4a274057bf20084f References: Subject: [FFmpeg-devel] [PATCH 2/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Qs2HaIntfYu1 - Added muxer for EVC format (MP4, raw) - Added demuxer for EVC format (MP4) - Added evc extension to the list of extensions for ff_mov_demuxer - Added information to moov atom Signed-off-by: Dawid Kozinski --- libavformat/evc.c | 461 +++++++++++++++++++++++++++++++++++++++++++ libavformat/evc.h | 147 ++++++++++++++ libavformat/evcdec.c | 142 +++++++++++++ 3 files changed, 750 insertions(+) create mode 100644 libavformat/evc.c create mode 100644 libavformat/evc.h create mode 100644 libavformat/evcdec.c -- 2.17.1 diff --git a/libavformat/evc.c b/libavformat/evc.c new file mode 100644 index 0000000000..9297fa5d5c --- /dev/null +++ b/libavformat/evc.c @@ -0,0 +1,461 @@ +/* + * EVC helper functions for muxers + * Copyright (c) 2022 Dawid Kozinski + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/intreadwrite.h" +#include "libavcodec/get_bits.h" +#include "libavcodec/golomb.h" +#include "avformat.h" +#include "avio.h" +#include "evc.h" +#include "avio_internal.h" + +// The length field that indicates the length in bytes of the following NAL unit is configured to be of 4 bytes +#define EVC_NAL_UNIT_LENGTH_BYTE (4) /* byte */ +#define EVC_NAL_HEADER_SIZE (2) /* byte */ + +// rpl structure +typedef struct RefPicListStruct { + int poc; + int tid; + int ref_pic_num; + int ref_pic_active_num; + int ref_pics[EVC_MAX_NUM_REF_PICS]; + char pic_type; + +} RefPicListStruct; + +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.1 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +typedef struct EVCSPS { + int sps_seq_parameter_set_id; // ue(v) + int profile_idc; // u(8) + int level_idc; // u(8) + int toolset_idc_h; // u(32) + int toolset_idc_l; // u(32) + int chroma_format_idc; // ue(v) + int pic_width_in_luma_samples; // ue(v) + int pic_height_in_luma_samples; // ue(v) + int bit_depth_luma_minus8; // ue(v) + int bit_depth_chroma_minus8; // ue(v) + + // @note + // Currently the structure does not reflect the entire SPS RBSP layout. + // It contains only the fields that are necessary to read from the NAL unit all the values + // necessary for the correct initialization of EVCDecoderConfigurationRecord + + // @note + // If necessary, add the missing fields to the structure to reflect + // the contents of the entire NAL unit of the SPS type + +} EVCSPS; + +typedef struct EVCNALUnitArray { + uint8_t array_completeness; + uint8_t NAL_unit_type; + uint16_t numNalus; + uint16_t *nalUnitLength; + uint8_t **nalUnit; +} EVCNALUnitArray; + +/** + * @brief Specifies the decoder configuration information for ISO/IEC 23094-1 video content. + * @see ISO/IEC 14496-15:2021 Coding of audio-visual objects — Part 15: + * Carriage of network abstraction layer (NAL) unit structured video in the ISO base media file format + */ +typedef struct EVCDecoderConfigurationRecord { + uint8_t configurationVersion; // 8 bits + uint8_t profile_idc; // 8 bits + uint8_t level_idc; // 8 bits + uint32_t toolset_idc_h; // 32 bits + uint32_t toolset_idc_l; // 32 bits + uint8_t chroma_format_idc; // 2 bits + uint8_t bit_depth_luma_minus8; // 3 bits + uint8_t bit_depth_chroma_minus8; // 3 bits + uint16_t pic_width_in_luma_samples; // 16 bits + uint16_t pic_height_in_luma_samples; // 16 bits + uint8_t reserved; // 6 bits '000000'b + uint8_t lengthSizeMinusOne; // 2 bits + uint8_t num_of_arrays; // 8 bits + EVCNALUnitArray *array; +} EVCDecoderConfigurationRecord; + +typedef struct NALU { + int offset; + uint32_t size; +} NALU; + +typedef struct NALUList { + NALU *nalus; + unsigned nalus_array_size; + unsigned nb_nalus; ///< valid entries in nalus +} NALUList; + +static int get_nalu_type(const uint8_t *bits, int bits_size) +{ + int unit_type_plus1 = 0; + + if (bits_size >= EVC_NAL_HEADER_SIZE) { + unsigned char *p = (unsigned char *)bits; + // forbidden_zero_bit + if ((p[0] & 0x80) != 0) + return -1; + + // nal_unit_type + unit_type_plus1 = (p[0] >> 1) & 0x3F; + } + + return unit_type_plus1 - 1; +} + +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size) +{ + uint32_t nalu_len = 0; + + if (bits_size >= EVC_NAL_UNIT_LENGTH_BYTE) { + + int t = 0; + unsigned char *p = (unsigned char *)bits; + + for (int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) + t = (t << 8) | p[i]; + + nalu_len = t; + if (nalu_len == 0) + return 0; + } + + return nalu_len; +} + +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) +static int evcc_parse_sps(const uint8_t *bs, int bs_size, EVCDecoderConfigurationRecord *evcc) +{ + GetBitContext gb; + int sps_seq_parameter_set_id; + EVCSPS sps; + + if (init_get_bits8(&gb, bs, bs_size) < 0) + return 0; + + sps.sps_seq_parameter_set_id = get_ue_golomb(&gb); + + if (sps_seq_parameter_set_id >= EVC_MAX_SPS_COUNT) + return 0; + + // the Baseline profile is indicated by profile_idc eqal to 0 + // the Main profile is indicated by profile_idc eqal to 1 + sps.profile_idc = get_bits(&gb, 8); + + sps.level_idc = get_bits(&gb, 8); + + sps.toolset_idc_h = get_bits(&gb, 32); + sps.toolset_idc_l = get_bits(&gb, 32); + + // 0 - monochrome + // 1 - 4:2:0 + // 2 - 4:2:2 + // 3 - 4:4:4 + sps.chroma_format_idc = get_ue_golomb(&gb); + + sps.pic_width_in_luma_samples = get_ue_golomb(&gb); + sps.pic_height_in_luma_samples = get_ue_golomb(&gb); + + sps.bit_depth_luma_minus8 = get_ue_golomb(&gb); + sps.bit_depth_chroma_minus8 = get_ue_golomb(&gb); + + evcc->profile_idc = sps.profile_idc; + evcc->level_idc = sps.level_idc; + evcc->toolset_idc_h = sps.toolset_idc_h; + evcc->toolset_idc_l = sps.toolset_idc_l; + evcc->chroma_format_idc = sps.chroma_format_idc; + evcc->bit_depth_luma_minus8 = sps.bit_depth_luma_minus8; + evcc->bit_depth_chroma_minus8 = sps.bit_depth_chroma_minus8; + evcc->pic_width_in_luma_samples = sps.pic_width_in_luma_samples; + evcc->pic_height_in_luma_samples = sps.pic_height_in_luma_samples; + + return 0; +} + +static int evcc_array_add_nal_unit(uint8_t *nal_buf, uint32_t nal_size, + uint8_t nal_type, int ps_array_completeness, + EVCDecoderConfigurationRecord *evcc) +{ + int ret; + uint8_t index; + uint16_t numNalus; + EVCNALUnitArray *array; + + for (index = 0; index < evcc->num_of_arrays; index++) + if (evcc->array[index].NAL_unit_type == nal_type) + break; + + if (index >= evcc->num_of_arrays) { + uint8_t i; + + ret = av_reallocp_array(&evcc->array, index + 1, sizeof(EVCNALUnitArray)); + if (ret < 0) + return ret; + + for (i = evcc->num_of_arrays; i <= index; i++) + memset(&evcc->array[i], 0, sizeof(EVCNALUnitArray)); + evcc->num_of_arrays = index + 1; + } + + array = &evcc->array[index]; + numNalus = array->numNalus; + + ret = av_reallocp_array(&array->nalUnit, numNalus + 1, sizeof(uint8_t *)); + if (ret < 0) + return ret; + + ret = av_reallocp_array(&array->nalUnitLength, numNalus + 1, sizeof(uint16_t)); + if (ret < 0) + return ret; + + array->nalUnit [numNalus] = nal_buf; + array->nalUnitLength[numNalus] = nal_size; + array->NAL_unit_type = nal_type; + array->numNalus++; + + /* + * When the sample entry name is 'evc1', the default and mandatory value of + * array_completeness is 1 for arrays of all types of parameter sets, and 0 + * for all other arrays. + */ + if (nal_type == EVC_SPS_NUT || nal_type == EVC_PPS_NUT) + array->array_completeness = ps_array_completeness; + + return 0; +} + +static void evcc_init(EVCDecoderConfigurationRecord *evcc) +{ + memset(evcc, 0, sizeof(EVCDecoderConfigurationRecord)); + evcc->configurationVersion = 1; + evcc->lengthSizeMinusOne = 3; // 4 bytes +} + +static void evcc_close(EVCDecoderConfigurationRecord *evcc) +{ + uint8_t i; + + for (i = 0; i < evcc->num_of_arrays; i++) { + evcc->array[i].numNalus = 0; + av_freep(&evcc->array[i].nalUnit); + av_freep(&evcc->array[i].nalUnitLength); + } + + evcc->num_of_arrays = 0; + av_freep(&evcc->array); +} + +static int evcc_write(AVIOContext *pb, EVCDecoderConfigurationRecord *evcc) +{ + uint8_t i; + uint16_t j, aps_count = 0, sps_count = 0, pps_count = 0; + + av_log(NULL, AV_LOG_TRACE, "configurationVersion: %"PRIu8"\n", + evcc->configurationVersion); + av_log(NULL, AV_LOG_TRACE, "profile_idc: %"PRIu8"\n", + evcc->profile_idc); + av_log(NULL, AV_LOG_TRACE, "level_idc: %"PRIu8"\n", + evcc->level_idc); + av_log(NULL, AV_LOG_TRACE, "toolset_idc_h: %"PRIu32"\n", + evcc->toolset_idc_h); + av_log(NULL, AV_LOG_TRACE, "toolset_idc_l: %"PRIu32"\n", + evcc->toolset_idc_l); + av_log(NULL, AV_LOG_TRACE, "chroma_format_idc: %"PRIu8"\n", + evcc->chroma_format_idc); + av_log(NULL, AV_LOG_TRACE, "bit_depth_luma_minus8: %"PRIu8"\n", + evcc->bit_depth_luma_minus8); + av_log(NULL, AV_LOG_TRACE, "bit_depth_chroma_minus8: %"PRIu8"\n", + evcc->bit_depth_chroma_minus8); + av_log(NULL, AV_LOG_TRACE, "pic_width_in_luma_samples: %"PRIu16"\n", + evcc->pic_width_in_luma_samples); + av_log(NULL, AV_LOG_TRACE, "pic_height_in_luma_samples: %"PRIu16"\n", + evcc->pic_height_in_luma_samples); + av_log(NULL, AV_LOG_TRACE, "lengthSizeMinusOne: %"PRIu8"\n", + evcc->lengthSizeMinusOne); + av_log(NULL, AV_LOG_TRACE, "num_of_arrays: %"PRIu8"\n", + evcc->num_of_arrays); + for (i = 0; i < evcc->num_of_arrays; i++) { + av_log(NULL, AV_LOG_TRACE, "array_completeness[%"PRIu8"]: %"PRIu8"\n", + i, evcc->array[i].array_completeness); + av_log(NULL, AV_LOG_TRACE, "NAL_unit_type[%"PRIu8"]: %"PRIu8"\n", + i, evcc->array[i].NAL_unit_type); + av_log(NULL, AV_LOG_TRACE, "numNalus[%"PRIu8"]: %"PRIu16"\n", + i, evcc->array[i].numNalus); + for (j = 0; j < evcc->array[i].numNalus; j++) + av_log(NULL, AV_LOG_TRACE, + "nalUnitLength[%"PRIu8"][%"PRIu16"]: %"PRIu16"\n", + i, j, evcc->array[i].nalUnitLength[j]); + } + + /* + * We need at least one SPS. + */ + for (i = 0; i < evcc->num_of_arrays; i++) + switch (evcc->array[i].NAL_unit_type) { + case EVC_APS_NUT: + aps_count += evcc->array[i].numNalus; + break; + case EVC_SPS_NUT: + sps_count += evcc->array[i].numNalus; + break; + case EVC_PPS_NUT: + pps_count += evcc->array[i].numNalus; + break; + default: + break; + } + if (!sps_count || sps_count > EVC_MAX_SPS_COUNT) + return AVERROR_INVALIDDATA; + + /* unsigned int(8) configurationVersion = 1; */ + avio_w8(pb, evcc->configurationVersion); + + /* unsigned int(8) profile_idc */ + avio_w8(pb, evcc->profile_idc); + + /* unsigned int(8) profile_idc */ + avio_w8(pb, evcc->level_idc); + + /* unsigned int(32) toolset_idc_h */ + avio_wb32(pb, evcc->toolset_idc_h); + + /* unsigned int(32) toolset_idc_l */ + avio_wb32(pb, evcc->toolset_idc_l); + + /* + * unsigned int(2) chroma_format_idc; + * unsigned int(3) bit_depth_luma_minus8; + * unsigned int(3) bit_depth_chroma_minus8; + */ + avio_w8(pb, evcc->chroma_format_idc << 6 | + evcc->bit_depth_luma_minus8 << 3 | + evcc->bit_depth_chroma_minus8); + + /* unsigned int(16) pic_width_in_luma_samples; */ + avio_wb16(pb, evcc->pic_width_in_luma_samples); + + /* unsigned int(16) pic_width_in_luma_samples; */ + avio_wb16(pb, evcc->pic_height_in_luma_samples); + + /* + * bit(6) reserved = '111111'b; + * unsigned int(2) chromaFormat; + */ + avio_w8(pb, evcc->lengthSizeMinusOne | 0xfc); + + /* unsigned int(8) numOfArrays; */ + avio_w8(pb, evcc->num_of_arrays); + + for (i = 0; i < evcc->num_of_arrays; i++) { + /* + * bit(1) array_completeness; + * unsigned int(1) reserved = 0; + * unsigned int(6) NAL_unit_type; + */ + avio_w8(pb, evcc->array[i].array_completeness << 7 | + evcc->array[i].NAL_unit_type & 0x3f); + + /* unsigned int(16) numNalus; */ + avio_wb16(pb, evcc->array[i].numNalus); + + for (j = 0; j < evcc->array[i].numNalus; j++) { + /* unsigned int(16) nalUnitLength; */ + avio_wb16(pb, evcc->array[i].nalUnitLength[j]); + + /* bit(8*nalUnitLength) nalUnit; */ + avio_write(pb, evcc->array[i].nalUnit[j], + evcc->array[i].nalUnitLength[j]); + } + } + + return 0; +} + +int ff_isom_write_evcc(AVIOContext *pb, const uint8_t *data, + int size, int ps_array_completeness) +{ + EVCDecoderConfigurationRecord evcc; + int ret = 0; + + if (size < 8) { + /* We can't write a valid evcC from the provided data */ + return AVERROR_INVALIDDATA; + } else if (*data == 1) { + /* Data is already evcC-formatted */ + avio_write(pb, data, size); + return 0; + } + + evcc_init(&evcc); + + int nalu_type; + size_t nalu_size; + unsigned char *bits = (unsigned char *)data; + int bytes_to_read = size; + + while (bytes_to_read > EVC_NAL_UNIT_LENGTH_BYTE) { + + nalu_size = read_nal_unit_length(bits, EVC_NAL_UNIT_LENGTH_BYTE); + if (nalu_size == 0) break; + + bits += EVC_NAL_UNIT_LENGTH_BYTE; + bytes_to_read -= EVC_NAL_UNIT_LENGTH_BYTE; + + if (bytes_to_read < nalu_size) break; + + nalu_type = get_nalu_type(bits, bytes_to_read); + uint8_t *nalu_buf = bits; + + switch (nalu_type) { + case EVC_APS_NUT: + case EVC_SPS_NUT: + case EVC_PPS_NUT: + ret = evcc_array_add_nal_unit(nalu_buf, nalu_size, nalu_type, ps_array_completeness, &evcc); + if (ret < 0) + goto end; + else if (nalu_type == EVC_SPS_NUT) + ret = evcc_parse_sps(nalu_buf, nalu_size, &evcc); + if (ret < 0) + goto end; + break; + default: + break; + } + + bits += nalu_size; + bytes_to_read -= nalu_size; + } + + ret = evcc_write(pb, &evcc); + +end: + evcc_close(&evcc); + return ret; +} + diff --git a/libavformat/evc.h b/libavformat/evc.h new file mode 100644 index 0000000000..d7b93d521d --- /dev/null +++ b/libavformat/evc.h @@ -0,0 +1,147 @@ +/* + * EVC helper functions for muxers + * Copyright (c) 2022 Dawid Kozinski + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFORMAT_EVC_H +#define AVFORMAT_EVC_H + +#include +#include "libavutil/rational.h" +#include "avio.h" + +/** + * @see ISO_IEC_23094-1_2020, 7.4.2.2 NAL unit header semantic + * Table 4 – NAL unit type codes and NAL unit type classes + */ +enum EVCNALUnitType { + EVC_NOIDR_NUT = 0, + EVC_IDR_NUT = 1, + EVC_RSV_VCL_NUT02 = 2, + EVC_RSV_VCL_NUT03 = 3, + EVC_RSV_VCL_NUT04 = 4, + EVC_RSV_VCL_NUT05 = 5, + EVC_RSV_VCL_NUT06 = 6, + EVC_RSV_VCL_NUT07 = 7, + EVC_RSV_VCL_NUT08 = 8, + EVC_RSV_VCL_NUT09 = 9, + EVC_RSV_VCL_NUT10 = 10, + EVC_RSV_VCL_NUT11 = 11, + EVC_RSV_VCL_NUT12 = 12, + EVC_RSV_VCL_NUT13 = 13, + EVC_RSV_VCL_NUT14 = 14, + EVC_RSV_VCL_NUT15 = 15, + EVC_RSV_VCL_NUT16 = 16, + EVC_RSV_VCL_NUT17 = 17, + EVC_RSV_VCL_NUT18 = 18, + EVC_RSV_VCL_NUT19 = 19, + EVC_RSV_VCL_NUT20 = 20, + EVC_RSV_VCL_NUT21 = 21, + EVC_RSV_VCL_NUT22 = 22, + EVC_RSV_VCL_NUT23 = 23, + EVC_SPS_NUT = 24, + EVC_PPS_NUT = 25, + EVC_APS_NUT = 26, + EVC_FD_NUT = 27, + EVC_SEI_NUT = 28, + EVC_RSV_NONVCL29 = 29, + EVC_RSV_NONVCL30 = 30, + EVC_RSV_NONVCL31 = 31, + EVC_RSV_NONVCL32 = 32, + EVC_RSV_NONVCL33 = 33, + EVC_RSV_NONVCL34 = 34, + EVC_RSV_NONVCL35 = 35, + EVC_RSV_NONVCL36 = 36, + EVC_RSV_NONVCL37 = 37, + EVC_RSV_NONVCL38 = 38, + EVC_RSV_NONVCL39 = 39, + EVC_RSV_NONVCL40 = 40, + EVC_RSV_NONVCL41 = 41, + EVC_RSV_NONVCL42 = 42, + EVC_RSV_NONVCL43 = 43, + EVC_RSV_NONVCL44 = 44, + EVC_RSV_NONVCL45 = 45, + EVC_RSV_NONVCL46 = 46, + EVC_RSV_NONVCL47 = 47, + EVC_RSV_NONVCL48 = 48, + EVC_RSV_NONVCL49 = 49, + EVC_RSV_NONVCL50 = 50, + EVC_RSV_NONVCL51 = 51, + EVC_RSV_NONVCL52 = 52, + EVC_RSV_NONVCL53 = 53, + EVC_RSV_NONVCL54 = 54, + EVC_RSV_NONVCL55 = 55, + EVC_UNSPEC_NUT56 = 56, + EVC_UNSPEC_NUT57 = 57, + EVC_UNSPEC_NUT58 = 58, + EVC_UNSPEC_NUT59 = 59, + EVC_UNSPEC_NUT60 = 60, + EVC_UNSPEC_NUT61 = 61, + EVC_UNSPEC_NUT62 = 62 +}; + +enum { + // 7.4.3.2: aps_video_parameter_set_id is u(4). + EVC_MAX_APS_COUNT = 32, + + // 7.4.3.1: sps_seq_parameter_set_id is in [0, 15]. + EVC_MAX_SPS_COUNT = 16, + + // 7.4.3.2: pps_pic_parameter_set_id is in [0, 63]. + EVC_MAX_PPS_COUNT = 64, + + // 7.4.5: slice header slice_pic_parameter_set_id in [0, 63] + EVC_MAX_SH_COUNT = 64, + + // E.3.2: cpb_cnt_minus1[i] is in [0, 31]. + EVC_MAX_CPB_CNT = 32, + + // A.4.1: in table A.1 the highest level allows a MaxLumaPs of 35 651 584. + EVC_MAX_LUMA_PS = 35651584, + + EVC_MAX_NUM_REF_PICS = 21, + + // A.4.1: pic_width_in_luma_samples and pic_height_in_luma_samples are + // constrained to be not greater than sqrt(MaxLumaPs * 8). Hence height/ + // width are bounded above by sqrt(8 * 35651584) = 16888.2 samples. + EVC_MAX_WIDTH = 16888, + EVC_MAX_HEIGHT = 16888, + + // A.4.1: table A.1 allows at most 22 tile rows for any level. + EVC_MAX_TILE_ROWS = 22, + // A.4.1: table A.1 allows at most 20 tile columns for any level. + EVC_MAX_TILE_COLUMNS = 20, + + // A.4.1: table A.1 allows at most 600 slice segments for any level. + EVC_MAX_SLICE_SEGMENTS = 600, + + // 7.4.7.1: in the worst case (tiles_enabled_flag and + // entropy_coding_sync_enabled_flag are both set), entry points can be + // placed at the beginning of every Ctb row in every tile, giving an + // upper bound of (num_tile_columns_minus1 + 1) * PicHeightInCtbsY - 1. + // Only a stream with very high resolution and perverse parameters could + // get near that, though, so set a lower limit here with the maximum + // possible value for 4K video (at most 135 16x16 Ctb rows). + HEVC_MAX_ENTRY_POINT_OFFSETS = EVC_MAX_TILE_COLUMNS * 135, +}; + +int ff_isom_write_evcc(AVIOContext *pb, const uint8_t *data, + int size, int ps_array_completeness); + +#endif // AVFORMAT_EVC_H \ No newline at end of file diff --git a/libavformat/evcdec.c b/libavformat/evcdec.c new file mode 100644 index 0000000000..58dc671d24 --- /dev/null +++ b/libavformat/evcdec.c @@ -0,0 +1,142 @@ +/* + * RAW EVC video demuxer + * + * Copyright (c) 2021 Dawid Kozinski + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavcodec/get_bits.h" +#include "libavcodec/golomb.h" +#include "libavcodec/internal.h" + +#include "rawdec.h" +#include "avformat.h" + +// The length field that indicates the length in bytes of the following NAL unit is configured to be of 4 bytes +#define EVC_NAL_UNIT_LENGTH_BYTE (4) /* byte */ + +#define EVC_NAL_HEADER_SIZE (2) /* byte */ +#define MAX_SPS_CNT (16) /* defined value in EVC standard */ + +// NALU types +// @see ISO_IEC_23094-1_2020 7.4.2.2 NAL unit header semantics +// +#define EVC_NUT_NONIDR (0) /* Coded slice of a non-IDR picture */ +#define EVC_NUT_IDR (1) /* Coded slice of an IDR picture */ +#define EVC_NUT_SPS (24) /* Sequence parameter set */ +#define EVC_NUT_PPS (25) /* Picture paremeter set */ +#define EVC_NUT_APS (26) /* Adaptation parameter set */ +#define EVC_NUT_FD (27) /* Filler data */ +#define EVC_NUT_SEI (28) /* Supplemental enhancement information */ + +typedef struct EVCParserContext { + int got_sps; + int got_pps; + int got_idr; + int got_nonidr; +} EVCParserContext; + +static int get_nalu_type(const uint8_t *bits, int bits_size) +{ + int unit_type_plus1 = 0; + + if(bits_size >= EVC_NAL_HEADER_SIZE) { + unsigned char *p = (unsigned char *)bits; + // forbidden_zero_bit + if ((p[0] & 0x80) != 0) { + av_log(NULL, AV_LOG_ERROR, "Cannot get bitstream information. Malformed bitstream.\n"); + return -1; + } + + // nal_unit_type + unit_type_plus1 = (p[0] >> 1) & 0x3F; + } + + return unit_type_plus1 - 1; +} + +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size) +{ + uint32_t nalu_len = 0; + + if(bits_size >= EVC_NAL_UNIT_LENGTH_BYTE) { + + int t = 0; + unsigned char *p = (unsigned char *)bits; + + for(int i=0; ibuf; + int bytes_to_read = p->buf_size; + + while(bytes_to_read > EVC_NAL_UNIT_LENGTH_BYTE) { + + nalu_size = read_nal_unit_length(bits, EVC_NAL_UNIT_LENGTH_BYTE); + if(nalu_size == 0) break; + + bits += EVC_NAL_UNIT_LENGTH_BYTE; + bytes_to_read -= EVC_NAL_UNIT_LENGTH_BYTE; + + if(bytes_to_read < nalu_size) break; + + nalu_type = get_nalu_type(bits, bytes_to_read); + + bits += nalu_size; + bytes_to_read -= nalu_size; + + if (nalu_type == EVC_NUT_SPS) + ev->got_sps++; + else if (nalu_type == EVC_NUT_PPS) + ev->got_pps++; + else if (nalu_type == EVC_NUT_IDR ) + ev->got_idr++; + else if (nalu_type == EVC_NUT_NONIDR) + ev->got_nonidr++; + } + + return 0; +} + +static int evc_probe(const AVProbeData *p) +{ + EVCParserContext ev = {0}; + int ret = parse_nal_units(p, &ev); + + if (ret == 0 && ev.got_sps && ev.got_pps && (ev.got_idr || ev.got_nonidr > 3)) + return AVPROBE_SCORE_EXTENSION + 1; // 1 more than .mpg + + return 0; +} + +FF_DEF_RAWVIDEO_DEMUXER(evc, "raw EVC video", evc_probe, "evc", AV_CODEC_ID_EVC)