From patchwork Mon Sep 24 00:12:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 10464 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:1286:0:0:0:0:0 with SMTP id 6-v6csp1930230jap; Sun, 23 Sep 2018 17:14:38 -0700 (PDT) X-Google-Smtp-Source: ANB0VdYYmBQTgfveKr+mj0j8ckhEzgEWabLonI9o+gLqFseaCi8y1/e9HJuUGXp29htNznBoauDw X-Received: by 2002:a1c:7f93:: with SMTP id a141-v6mr5594259wmd.45.1537748077944; Sun, 23 Sep 2018 17:14:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537748077; cv=none; d=google.com; s=arc-20160816; b=Jb72btTQnNWYOkOXK0DXvt3Le/BzgIjocmNsekPkUAfrdI1wPL/pz+ulqlsYs0W9zw 5LiP6+RXKHKPAV8RTorK/0LTA7m/MIQT4w37bk9rQXlIoal8YbScp8l2Y8xSzg/qxVGO 92wrXEunuS90071cSpSt4vKlb9OqdfvCaspADqOsRQPyMSgA48I4EMy/RtyIeUmNnyA8 rsfmQN9Yn6SnZQkFsvtaEan3INTQm5+Rh0tLJ0TAnAc+WlyE3KDSTxvwfZsm56jre0Ei 8xpuBCBeAkhqAyjdCN6xsOZyb2uUCoIIVQqsokfAksnPQIoJIGPq9ymmJPobRcI8l6P8 tCkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=cnm/SKyimfTYS3D+IrlbZSNf8ko4YGhIbJK7LPrfec0=; b=smGHH7Wg69xoawJxt6QK8JhDO9/Ynp5OPgwAZWSuo0Cl0LB83P9LGYnPn0mp3Uq+G3 Osa7P4x1pr7m6baO32fR8nIiQVWEAB4gPxD6KGnZBQhYz0LRDdvfuUiBYFVnx22ziRx1 NZzHEt/JFhFbhfxqVeFONcJc0E3NB6r988algtuua4MToOZrgPIVFuORlgK+H/CK1WJV GsqQHo1EgdznkVeK42iwEtMYmlnX++8uhgJQyPMSkogHtRyPx84j7FVqhIlOzoZX3Odf Qk2DvAJwOztw1Rw144MSykh4Wnc6nSIOSjZ4z59AADXEFfYxCAtBvtn03G+wpwE3y4an jMvg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=MA5+zzFh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y31-v6si6257607wrd.211.2018.09.23.17.14.37; Sun, 23 Sep 2018 17:14:37 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=MA5+zzFh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 44DC468A564; Mon, 24 Sep 2018 03:14:20 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f195.google.com (mail-qt1-f195.google.com [209.85.160.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B31C568A24C for ; Mon, 24 Sep 2018 03:14:13 +0300 (EEST) Received: by mail-qt1-f195.google.com with SMTP id l16-v6so7011967qtq.10 for ; Sun, 23 Sep 2018 17:14:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=R/iWcemSq7K3Clb8Fg1OkVHv5DZLEJL89zsuBzASAsA=; b=MA5+zzFh96he88dtw33sscwfNrPOqoXJyBTSR5pm/0y6FhqVQQchPfwLNlcQkLG+Tn 8++Yyof54YdNBqNB3YryzXW0f1HBYAuJ5ZJn3nlMdzigOKdbGS2uIwcGhDEKK9T9AlNs sUFeox49bhflyRv5l5JG4LoTdmj5kJ+uzCZlSLpOz09e4mEhRV4frDecQ4NMTLnyLweX CjZeqZOlUKp+Xx6/2eIHaZnIWOKAiilbk+BeMhsaiDbr3BgX25gfj1QdatLseI2K3K4g ci6UyyH7PlCczMBUu6C6XASfZ9hU8BWF9n9imjW16B4CdmXJ13a25jnIpjlM1GltrAjD Eodg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=R/iWcemSq7K3Clb8Fg1OkVHv5DZLEJL89zsuBzASAsA=; b=TfYj5D4WETtVVqB24lFNOAxbwQNx8YiU3VBh5BT53B/JzQpSqvYcjEvjd/dLmJIXrc CjoW+uCpK1/TztZQawsvW7lXckEze/NWnD7hd68el1T1F4KWQz++paVsmPQm6liRUrD8 7K6ZqRxI8J1fVViAT8h/UFWuiZWZTuI8cC0n0zW0IM1I8cI0644xx04aiTNW7YgNSZc8 UnvEkj69sl7EwL10BG0Po6cDzqb4L1D5484dY3p4968WZc3gmRopgLRTawsCpX/u6HDi O5eJC/JtZL9nG8XG78n/OPldIh02TUq2Tkmvbve9SJb8ri/lKsy7y0Vp2z/TGyKHd92W Dd5A== X-Gm-Message-State: APzg51Aqhh59Ul7nnPiY3JtowjNxFb/iM5VwMY9JTh0ievUvh1mIUSIX YZREUbPaGe1+tqSqe5Bs6md7WAoC X-Received: by 2002:ac8:1c89:: with SMTP id f9-v6mr6131168qtl.265.1537748067672; Sun, 23 Sep 2018 17:14:27 -0700 (PDT) Received: from localhost.localdomain ([179.38.139.56]) by smtp.gmail.com with ESMTPSA id a50-v6sm5249904qtc.93.2018.09.23.17.14.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 23 Sep 2018 17:14:27 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Sep 2018 21:12:30 -0300 Message-Id: <20180924001230.2892-1-jamrial@gmail.com> X-Mailer: git-send-email 2.19.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec: add an AV1 parser X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Simple parser to set keyframes, frame type, structure, width, height, and pixel format, plus stream profile and level. Signed-off-by: James Almer --- Missing Changelog entry and version bump. This depends on "[PATCH v2 2/3] lavc: Add coded bitstream read/write support for AV1" which should be committed in the coming days. The AVCodecParser.split() implementation, added for the sake of completeness, is very naive and much like the h264 and hevc ones can result in useless OBUs being "extracted", but since it's no longer used by libavformat to fill global headers when reading raw containers it shouldn't really matter. It's pretty much used only by the remove_extradata bsf at this point. configure | 1 + libavcodec/Makefile | 1 + libavcodec/av1_parser.c | 218 ++++++++++++++++++++++++++++++++++++++++ libavcodec/parsers.c | 1 + 4 files changed, 221 insertions(+) create mode 100644 libavcodec/av1_parser.c diff --git a/configure b/configure index ca8b599b63..b46c86ec95 100755 --- a/configure +++ b/configure @@ -3020,6 +3020,7 @@ wmv3_crystalhd_decoder_select="crystalhd" # parsers aac_parser_select="adts_header" +av1_parser_select="cbs_av1" h264_parser_select="golomb h264dsp h264parse" hevc_parser_select="hevcparse" mpegaudio_parser_select="mpegaudioheader" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index b2c6995f9a..dc28892e64 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1006,6 +1006,7 @@ OBJS-$(CONFIG_AAC_PARSER) += aac_parser.o aac_ac3_parser.o \ mpeg4audio.o OBJS-$(CONFIG_AC3_PARSER) += ac3tab.o aac_ac3_parser.o OBJS-$(CONFIG_ADX_PARSER) += adx_parser.o adx.o +OBJS-$(CONFIG_AV1_PARSER) += av1_parser.o OBJS-$(CONFIG_AVS2_PARSER) += avs2_parser.o OBJS-$(CONFIG_BMP_PARSER) += bmp_parser.o OBJS-$(CONFIG_CAVSVIDEO_PARSER) += cavs_parser.o diff --git a/libavcodec/av1_parser.c b/libavcodec/av1_parser.c new file mode 100644 index 0000000000..b2e19e2119 --- /dev/null +++ b/libavcodec/av1_parser.c @@ -0,0 +1,218 @@ +/* + * AV1 parser + * + * Copyright (C) 2018 James Almer + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "av1_parse.h" +#include "cbs.h" +#include "cbs_av1.h" +#include "parser.h" + +typedef struct AV1ParseContext { + CodedBitstreamContext *cbc; + CodedBitstreamFragment temporal_unit; + int parsed_extradata; +} AV1ParseContext; + +static int av1_parser_parse(AVCodecParserContext *ctx, + AVCodecContext *avctx, + const uint8_t **out_data, int *out_size, + const uint8_t *data, int size) +{ + AV1ParseContext *s = ctx->priv_data; + CodedBitstreamFragment *td = &s->temporal_unit; + CodedBitstreamAV1Context *av1 = s->cbc->priv_data; + int ret; + + *out_data = data; + *out_size = size; + + ctx->key_frame = -1; + ctx->pict_type = AV_PICTURE_TYPE_NONE; + ctx->picture_structure = AV_PICTURE_STRUCTURE_UNKNOWN; + + if (avctx->extradata_size && !s->parsed_extradata) { + ret = ff_cbs_read(s->cbc, td, avctx->extradata, avctx->extradata_size); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to parse extradata.\n"); + return size; + } + + s->parsed_extradata = 1; + + ff_cbs_fragment_uninit(s->cbc, td); + } + + ret = ff_cbs_read(s->cbc, td, data, size); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to parse temporal unit.\n"); + return size; + } + + if (!av1->sequence_header) { + av_log(avctx, AV_LOG_ERROR, "No sequence header available\n"); + goto end; + } + + for (int i = 0; i < td->nb_units; i++) { + CodedBitstreamUnit *unit = &td->units[i]; + AV1RawOBU *obu = unit->content; + AV1RawSequenceHeader *seq = av1->sequence_header; + AV1RawFrameHeader *frame; + int frame_type, bitdepth, subsampling; + + if (unit->type == AV1_OBU_FRAME) + frame = &obu->obu.frame.header; + else if (unit->type == AV1_OBU_FRAME_HEADER) + frame = &obu->obu.frame_header; + else + continue; + + if (frame->show_existing_frame) { + AV1ReferenceFrameState *ref = &av1->ref[frame->frame_to_show_map_idx]; + + if (!ref->valid) { + av_log(avctx, AV_LOG_ERROR, "Invalid reference frame\n"); + goto end; + } + + ctx->width = ref->frame_width; + ctx->height = ref->frame_height; + frame_type = ref->frame_type; + + ctx->key_frame = 0; + } else if (!frame->show_frame) { + continue; + } else { + ctx->width = av1->frame_width; + ctx->height = av1->frame_height; + frame_type = frame->frame_type; + + ctx->key_frame = frame_type == AV1_FRAME_KEY; + } + + avctx->profile = seq->seq_profile; + avctx->level = seq->seq_level_idx[0]; + + switch (frame_type) { + case AV1_FRAME_KEY: + case AV1_FRAME_INTRA_ONLY: + ctx->pict_type = AV_PICTURE_TYPE_I; + break; + case AV1_FRAME_INTER: + ctx->pict_type = AV_PICTURE_TYPE_P; + break; + case AV1_FRAME_SWITCH: + ctx->pict_type = AV_PICTURE_TYPE_SP; + break; + } + + ctx->picture_structure = AV_PICTURE_STRUCTURE_FRAME; + + subsampling = seq->color_config.subsampling_x << 1 & seq->color_config.subsampling_y; + bitdepth = 8 + seq->color_config.high_bitdepth * 2 + seq->color_config.twelve_bit * 2; + switch (bitdepth) { + case 8: + if (subsampling == 3) ctx->format = seq->color_config.mono_chrome ? AV_PIX_FMT_GRAY8 : + AV_PIX_FMT_YUV420P; + else if (subsampling == 2) ctx->format = AV_PIX_FMT_YUV422P; + else ctx->format = AV_PIX_FMT_YUV444P; + break; + case 10: + if (subsampling == 3) ctx->format = seq->color_config.mono_chrome ? AV_PIX_FMT_GRAY10 : + AV_PIX_FMT_YUV420P10; + else if (subsampling == 2) ctx->format = AV_PIX_FMT_YUV422P10; + else ctx->format = AV_PIX_FMT_YUV444P10; + break; + case 12: + if (subsampling == 3) ctx->format = seq->color_config.mono_chrome ? AV_PIX_FMT_GRAY12 : + AV_PIX_FMT_YUV420P12; + else if (subsampling == 2) ctx->format = AV_PIX_FMT_YUV422P12; + else ctx->format = AV_PIX_FMT_YUV444P12; + break; + } + } + +end: + ff_cbs_fragment_uninit(s->cbc, td); + + return size; +} + +static const CodedBitstreamUnitType decompose_unit_types[] = { + AV1_OBU_TEMPORAL_DELIMITER, + AV1_OBU_SEQUENCE_HEADER, + AV1_OBU_FRAME_HEADER, + AV1_OBU_TILE_GROUP, + AV1_OBU_FRAME, +}; + +static av_cold int av1_parser_init(AVCodecParserContext *ctx) +{ + AV1ParseContext *s = ctx->priv_data; + int ret; + + ret = ff_cbs_init(&s->cbc, AV_CODEC_ID_AV1, NULL); + if (ret < 0) + return ret; + + s->cbc->decompose_unit_types = (CodedBitstreamUnitType *)decompose_unit_types; + s->cbc->nb_decompose_unit_types = FF_ARRAY_ELEMS(decompose_unit_types); + + return 0; +} + +static void av1_parser_close(AVCodecParserContext *ctx) +{ + AV1ParseContext *s = ctx->priv_data; + + ff_cbs_close(&s->cbc); +} + +static int av1_parser_split(AVCodecContext *avctx, + const uint8_t *buf, int buf_size) +{ + AV1OBU obu; + const uint8_t *ptr = buf, *end = buf + buf_size; + + while (ptr < end) { + int len = ff_av1_extract_obu(&obu, ptr, buf_size, avctx); + if (len < 0) + break; + + if (obu.type == AV1_OBU_FRAME_HEADER || + obu.type == AV1_OBU_FRAME) { + return ptr - buf; + } + ptr += len; + buf_size -= len; + } + + return 0; +} + +AVCodecParser ff_av1_parser = { + .codec_ids = { AV_CODEC_ID_AV1 }, + .priv_data_size = sizeof(AV1ParseContext), + .parser_init = av1_parser_init, + .parser_close = av1_parser_close, + .parser_parse = av1_parser_parse, + .split = av1_parser_split, +}; diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index cb86cceecc..f01cad4c84 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -26,6 +26,7 @@ extern AVCodecParser ff_aac_parser; extern AVCodecParser ff_aac_latm_parser; extern AVCodecParser ff_ac3_parser; extern AVCodecParser ff_adx_parser; +extern AVCodecParser ff_av1_parser; extern AVCodecParser ff_avs2_parser; extern AVCodecParser ff_bmp_parser; extern AVCodecParser ff_cavsvideo_parser;