From patchwork Mon Jan 25 14:15:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 25163 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id CF97044BD19 for ; Mon, 25 Jan 2021 16:17:17 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B837D68A22C; Mon, 25 Jan 2021 16:17:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B7A6D6881E1 for ; Mon, 25 Jan 2021 16:17:16 +0200 (EET) Received: by mail-pg1-f174.google.com with SMTP id i5so9000451pgo.1 for ; Mon, 25 Jan 2021 06:17:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gA4CE3B/Th4VkUILS9EzT1Kbvlcwt7ghLcjFHoYBwgo=; b=QjGHY792NsdzFFkyX/1EUJ7NeqdVurPmPO4moi99XYrAPapxrn95Oqx0YaYJijUH4D X+WrzMlpUEBCKapIFXgSfzfgQC+/DonM30Bwv0GR0d0pGDD49RHtJh6j51hE0sV5QV0W 0hafprMG8Ucfrj5rtoNlLW0MD5HWpCXFYVTGsYK+hy++N34YMvxguyM87fjbetSrmOib RRebLCXI/dkYOYcgKON0ZTpaNButP4tXCX/g93uN8dQhiLdo9fYZzG4rtp1BlfKNm7rj QcqQtKCTRsyU+5gqMqC9j5MPA6WdqPwfFEqvzxo0Rr1iBhRDh0VCOZr5ETb6sSoBwOg0 OtPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gA4CE3B/Th4VkUILS9EzT1Kbvlcwt7ghLcjFHoYBwgo=; b=slgtsS0jl4RQGhwm9BuE4QbTm3L0/G35BleP0Eb+CjfPuvAXN3LAk6to7nABHhDvfB YahYZtH/1HU6dTUgZMJG203BzUaIa1vhQnSt9iYZXwvADllDYsQqu+YnOi39ROZJXeoK /blDFF3+2WdSQJLMQlJMw4JIM8aO2TuMe6qBaY/LuJJgxgTf9VZ3K+5Ul6IKV6Nc4AnD v9m4TDlCPYwOaXXq73M72Gn/KmqtTZVZJcav+DY3A/hljcmkQJD6XUx+HugucC2bXlOd o5mq7iS5T695syKHg3W5tLE/88tLx6N8MZYxtwn/tRi9NafKygRI+MXr3T+8viFg1EXn oa8Q== X-Gm-Message-State: AOAM533zgph8XnG9Y8uRq2Gv8eLMyYwY9UsnZbIxUpls0GomFGtdmN39 n8O1JSXYp0F6uBeTkn4+MWg/c58VCnQ= X-Google-Smtp-Source: ABdhPJxjG9sPsShqT12jpvC6auX4n7vbv7SsmixEyDHKh3jHRgNvi8St21A+o+/9kAAIbw0JpijaVQ== X-Received: by 2002:aa7:85cc:0:b029:1be:181e:32af with SMTP id z12-20020aa785cc0000b02901be181e32afmr709063pfn.50.1611584234774; Mon, 25 Jan 2021 06:17:14 -0800 (PST) Received: from smith.tendawifi.com (23.83.245.51.16clouds.com. [23.83.245.51]) by smtp.gmail.com with ESMTPSA id z6sm5523328pfr.133.2021.01.25.06.17.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 06:17:14 -0800 (PST) From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Mon, 25 Jan 2021 22:15:04 +0800 Message-Id: <20210125141507.11012-7-nuomi2021@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125141507.11012-1-nuomi2021@gmail.com> References: <20210111152357.3965-1-nuomi2021@gmail.com> <20210125141507.11012-1-nuomi2021@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 6/9] avcodec: add vvc parser X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- configure | 1 + libavcodec/Makefile | 1 + libavcodec/parsers.c | 1 + libavcodec/vvc_parser.c | 299 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 302 insertions(+) create mode 100644 libavcodec/vvc_parser.c diff --git a/configure b/configure index d55cd0a79e..74ff58d7b7 100755 --- a/configure +++ b/configure @@ -3167,6 +3167,7 @@ mpegaudio_parser_select="mpegaudioheader" mpegvideo_parser_select="mpegvideo" mpeg4video_parser_select="h263dsp mpegvideo qpeldsp" vc1_parser_select="vc1dsp" +vcc_parser_select="cbs_h266" # bitstream_filters aac_adtstoasc_bsf_select="adts_header" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 0d7a417180..e736c8300a 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1126,6 +1126,7 @@ OBJS-$(CONFIG_VC1_PARSER) += vc1_parser.o vc1.o vc1data.o \ OBJS-$(CONFIG_VP3_PARSER) += vp3_parser.o OBJS-$(CONFIG_VP8_PARSER) += vp8_parser.o OBJS-$(CONFIG_VP9_PARSER) += vp9_parser.o +OBJS-$(CONFIG_VVC_PARSER) += vvc_parser.o OBJS-$(CONFIG_WEBP_PARSER) += webp_parser.o OBJS-$(CONFIG_XMA_PARSER) += xma_parser.o diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index 3156b86b03..9c246165c4 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -71,6 +71,7 @@ extern AVCodecParser ff_vorbis_parser; extern AVCodecParser ff_vp3_parser; extern AVCodecParser ff_vp8_parser; extern AVCodecParser ff_vp9_parser; +extern AVCodecParser ff_vvc_parser; extern AVCodecParser ff_webp_parser; extern AVCodecParser ff_xma_parser; diff --git a/libavcodec/vvc_parser.c b/libavcodec/vvc_parser.c new file mode 100644 index 0000000000..b160ef9426 --- /dev/null +++ b/libavcodec/vvc_parser.c @@ -0,0 +1,299 @@ +/* + * VVC parser + * + * Copyright (C) 2029 Nuo Mi + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "cbs.h" +#include "cbs_h266.h" +#include "internal.h" +#include "parser.h" + +#define START_CODE 0x000001 ///< start_code_prefix_one_3bytes + +#define IS_SLICE(nut) (nut <= VVC_RASL_NUT || (nut >= VVC_IDR_W_RADL && nut <= VVC_GDR_NUT)) + +typedef struct VVCParserContext { + ParseContext pc; + CodedBitstreamContext *cbc; + CodedBitstreamFragment picture_unit; + int parsed_extradata; +} VVCParserContext; + +static const enum AVPixelFormat pix_fmts_8bit[] = { + AV_PIX_FMT_GRAY8, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV444P +}; + +static const enum AVPixelFormat pix_fmts_10bit[] = { + AV_PIX_FMT_GRAY10, AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_YUV422P10, AV_PIX_FMT_YUV444P10 +}; + +static int get_format(const H266RawSPS* sps) +{ + switch (sps->sps_bitdepth_minus8) { + case 0: + return pix_fmts_8bit[sps->sps_chroma_format_idc]; + case 2: + return pix_fmts_10bit[sps->sps_chroma_format_idc]; + } + return AV_PIX_FMT_NONE; +} + +/** + * Find the end of the current frame in the bitstream. + * @return the position of the first byte of the next frame, or END_NOT_FOUND + */ +static int find_frame_end(AVCodecParserContext *s, const uint8_t *buf, + int buf_size) +{ + VVCParserContext *ctx = s->priv_data; + ParseContext *pc = &ctx->pc; + int i; + + for (i = 0; i < buf_size; i++) { + int nut; + + pc->state64 = (pc->state64 << 8) | buf[i]; + + if (((pc->state64 >> 3 * 8) & 0xFFFFFF) != START_CODE) + continue; + + nut = (pc->state64 >> (8 + 3)) & 0x1F; + // Beginning of picture unit + if ((nut >= VVC_OPI_NUT && nut <= VVC_EOB_NUT && nut != VVC_PH_NUT) || + nut == VVC_PREFIX_SEI_NUT || + (nut >= VVC_RSV_NVCL_26 && nut <= VVC_UNSPEC_31)) { + if (pc->frame_start_found) { + pc->frame_start_found = 0; + return i - 5; + } + } else if (nut == VVC_PH_NUT || IS_SLICE(nut)) { + int sh_picture_header_in_slice_header_flag = buf[i] >> 7; + + if (nut == VVC_PH_NUT || sh_picture_header_in_slice_header_flag) { + if (!pc->frame_start_found) { + pc->frame_start_found = 1; + } else { // First slice of next frame found + pc->frame_start_found = 0; + return i - 5; + } + } + } + } + return END_NOT_FOUND; +} + +static int get_pict_type(const CodedBitstreamFragment *pu) +{ + int has_p = 0; + for (int i = 0; i < pu->nb_units; i++) { + CodedBitstreamUnit *unit = &pu->units[i]; + if (IS_SLICE(unit->type)) { + const H266RawSlice *slice = unit->content; + uint8_t type = slice->header.sh_slice_type; + if (type == VVC_SLICE_TYPE_B) { + return AV_PICTURE_TYPE_B; + } + if (type == VVC_SLICE_TYPE_P) { + has_p = 1; + } + } + } + return has_p ? AV_PICTURE_TYPE_P : AV_PICTURE_TYPE_I; +} + +/** + * Parse NAL units of found picture and decode some basic information. + * + * @param s parser context. + * @param avctx codec context. + * @param buf buffer with field/frame data. + * @param buf_size size of the buffer. + */ +static int parse_nal_units(AVCodecParserContext *ctx, const uint8_t *buf, + int buf_size, AVCodecContext *avctx) +{ + VVCParserContext *s = ctx->priv_data; + CodedBitstreamFragment *pu = &s->picture_unit; + CodedBitstreamH266Context *h266 = s->cbc->priv_data; + const H266RawSPS *sps; + const H266RawPPS *pps; + const H266RawPH *ph; + int ret, num = 0, den = 0; + static const uint8_t h266_sub_width_c[] = { + 1, 2, 2, 1 + }; + static const uint8_t h266_sub_height_c[] = { + 1, 2, 1, 1 + }; + + /* set some sane default values */ + ctx->pict_type = AV_PICTURE_TYPE_I; + ctx->key_frame = 0; + ctx->picture_structure = AV_PICTURE_STRUCTURE_FRAME; + + s->cbc->log_ctx = avctx; + + if (avctx->extradata_size && !s->parsed_extradata) { + s->parsed_extradata = 1; + + if ((ret = ff_cbs_read(s->cbc, pu, avctx->extradata, avctx->extradata_size)) < 0) + av_log(avctx, AV_LOG_WARNING, "Failed to parse extradata.\n"); + + ff_cbs_fragment_reset(pu); + } + + if ((ret = ff_cbs_read(s->cbc, pu, buf, buf_size))< 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to parse picture unit.\n"); + goto end; + } + + sps = h266->active_sps; + pps = h266->active_pps; + ph = h266->ph; + if (!ph) { + av_log(avctx, AV_LOG_ERROR, "No picture header.\n"); + goto end; + } + av_assert0(sps); + av_assert0(pps); + + ctx->key_frame = ph->ph_gdr_or_irap_pic_flag; + ctx->coded_width = pps->pps_pic_width_in_luma_samples; + ctx->coded_height = pps->pps_pic_height_in_luma_samples; + ctx->width = pps->pps_pic_width_in_luma_samples - + (pps->pps_conf_win_left_offset + pps->pps_conf_win_right_offset) * + h266_sub_width_c[sps->sps_chroma_format_idc]; + ctx->height = pps->pps_pic_height_in_luma_samples - + (pps->pps_conf_win_top_offset + pps->pps_conf_win_bottom_offset) * + h266_sub_height_c[sps->sps_chroma_format_idc];; + ctx->pict_type = get_pict_type(pu); + ctx->format = get_format(sps); + + avctx->profile = sps->profile_tier_level.general_profile_idc; + avctx->level = sps->profile_tier_level.general_level_idc; + + if (ctx->width != avctx->width || ctx->height != avctx->height) { + ret = ff_set_dimensions(avctx, ctx->width, ctx->height); + if (ret < 0) + goto end; + } + avctx->pix_fmt = ctx->format; + + if(sps->sps_ptl_dpb_hrd_params_present_flag && sps->sps_timing_hrd_params_present_flag) { + num = sps->sps_general_timing_hrd_parameters.num_units_in_tick; + den = sps->sps_general_timing_hrd_parameters.time_scale; + } else { + goto end; + } + if (num != 0 && den != 0) + av_reduce(&avctx->framerate.den, &avctx->framerate.num, + num, den, 1 << 30); +end: + ff_cbs_fragment_reset(pu); + s->cbc->log_ctx = NULL; + return ret; +} + +static int vvc_parser_parse(AVCodecParserContext *s, AVCodecContext *avctx, + const uint8_t **poutbuf, int *poutbuf_size, + const uint8_t *buf, int buf_size) +{ + int next; + VVCParserContext *ctx = s->priv_data; + ParseContext *pc = &ctx->pc; + int is_dummy_buf = !buf_size; + const uint8_t *dummy_buf = buf; + + if (avctx->extradata && !ctx->parsed_extradata) { + av_log(avctx, AV_LOG_INFO, "extra data is not supported yet.\n"); + return AVERROR_PATCHWELCOME; + } + + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) { + next = buf_size; + } else { + next = find_frame_end(s, buf, buf_size); + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { + *poutbuf = NULL; + *poutbuf_size = 0; + return buf_size; + } + } + + is_dummy_buf &= (dummy_buf == buf); + + if (!is_dummy_buf) + parse_nal_units(s, buf, buf_size, avctx); + + *poutbuf = buf; + *poutbuf_size = buf_size; + return next; + +} + +static const CodedBitstreamUnitType decompose_unit_types[] = { + VVC_TRAIL_NUT, + VVC_STSA_NUT, + VVC_RADL_NUT, + VVC_RASL_NUT, + VVC_IDR_W_RADL, + VVC_IDR_N_LP, + VVC_CRA_NUT, + VVC_GDR_NUT, + VVC_VPS_NUT, + VVC_SPS_NUT, + VVC_PPS_NUT, + VVC_PH_NUT, + VVC_AUD_NUT, +}; + +static av_cold int vvc_parser_init(AVCodecParserContext *ctx) +{ + VVCParserContext *s = ctx->priv_data; + int ret; + + ret = ff_cbs_init(&s->cbc, AV_CODEC_ID_VVC, NULL); + if (ret < 0) + return ret; + + s->cbc->decompose_unit_types = decompose_unit_types; + s->cbc->nb_decompose_unit_types = FF_ARRAY_ELEMS(decompose_unit_types); + + return 0; +} + +static void vvc_parser_close(AVCodecParserContext *ctx) +{ + VVCParserContext *s = ctx->priv_data; + + ff_cbs_fragment_free(&s->picture_unit); + ff_cbs_close(&s->cbc); + av_freep(&s->pc.buffer); +} + +AVCodecParser ff_vvc_parser = { + .codec_ids = { AV_CODEC_ID_VVC }, + .priv_data_size = sizeof(VVCParserContext), + .parser_init = vvc_parser_init, + .parser_close = vvc_parser_close, + .parser_parse = vvc_parser_parse, +};