From patchwork Wed Mar 27 00:25:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 12461 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id E043144857A for ; Wed, 27 Mar 2019 02:34:12 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C1BC068A6E6; Wed, 27 Mar 2019 02:34:12 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f194.google.com (mail-qt1-f194.google.com [209.85.160.194]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 623306883B6 for ; Wed, 27 Mar 2019 02:34:06 +0200 (EET) Received: by mail-qt1-f194.google.com with SMTP id k14so16960575qtb.0 for ; Tue, 26 Mar 2019 17:34:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=DIc4Sa55j0SVjvtahlTRtXx/ftexLrqVfPgqiYepymY=; b=sGwa8lgjKvIiTf89FtK89xwi47PwDqnPMPr8CEa25LXrravk1gKXejJpbvghe4rMdh zh9ucCAkz7cLFSRKEtJr+Dt7bq/mdTtkkps0vTqaRQrZnpKm5v3XVmrAjMMzPl90hjsb ZCUppxrXRhCsmmlojtTNrtdIqbZ4OMXAXSJ4sPu1k+44ttKawBrmFFXnPHuUmt8OUqeR iHBlOykBMGZ2VwyxcslZoLO1MPgQzKL6acG2g5GawOaNxbzfec26sg/dbnF0DimZM0na zWE9k7cJ6BcSA263Pp2f8UuRolI/bN+Ul7No7IiLn2Rif20cG6n40/zj3GkCewP1RaCd W5sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=DIc4Sa55j0SVjvtahlTRtXx/ftexLrqVfPgqiYepymY=; b=B+seArnJl6gBv2PMU0yyTOiX1QYMy2SpnslNLmhx+7QcaHZTa8jpWbiFDVcUf49ZJb qyoTGgYKY7p2f0K2f4Tf2woWRv1WOeKirqyEYKJXhYtABWULQJMCZcVYgWeY2N+07u0O w1pKeoQ4cYvIl57ehDiMgx+de17vo2MybWrBINTVlrZ0XVhq8o00exF0wSGbdq31hIEI WhLKL47ENOITiOCMnLGpmPy125zjIaiOL+udHmCefmdh84SX+BINQqPfhr6XoXdslUUJ tu+4P8n5/dW1i3rsW/qG1T580X9Tgnasho89u6bClAfvCEd5qenx8P8a2W1CAEPwh11+ ODHw== X-Gm-Message-State: APjAAAX6c0G8HIpu395g9mdqiQAASXk1FFnNtuu2IKkmfydbuQipUScZ meMy0IxDPWr9vn+btP9TdcuppI8LH1I= X-Google-Smtp-Source: APXvYqy2s4mcza0QSLtkuEzaGDWa7XH2VruNMLGpg1C93iL39LGHjGp9IZtDcw0kKh9k3SUAzL4qvg== X-Received: by 2002:ac8:33ad:: with SMTP id c42mr27975928qtb.318.1553646377413; Tue, 26 Mar 2019 17:26:17 -0700 (PDT) Received: from localhost.localdomain ([191.83.222.129]) by smtp.gmail.com with ESMTPSA id p67sm11449996qkd.39.2019.03.26.17.26.16 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 Mar 2019 17:26:16 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Tue, 26 Mar 2019 21:25:58 -0300 Message-Id: <20190327002558.4536-1-jamrial@gmail.com> X-Mailer: git-send-email 2.21.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3] avcodec: add AV1 frame split bitstream filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This will be needed by the eventual native AV1 decoder. Signed-off-by: James Almer --- Now propagating the Temporal Unit unchanged if splitting can't be performed. configure | 1 + libavcodec/Makefile | 1 + libavcodec/av1_frame_split_bsf.c | 252 +++++++++++++++++++++++++++++++ libavcodec/bitstream_filters.c | 1 + 4 files changed, 255 insertions(+) create mode 100644 libavcodec/av1_frame_split_bsf.c diff --git a/configure b/configure index 331393f8d5..ed33490797 100755 --- a/configure +++ b/configure @@ -3076,6 +3076,7 @@ vc1_parser_select="vc1dsp" # bitstream_filters aac_adtstoasc_bsf_select="adts_header" +av1_frame_split_select="cbs_av1" av1_metadata_bsf_select="cbs_av1" eac3_core_bsf_select="ac3_parser" filter_units_bsf_select="cbs" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 15c43a8a6a..27f326247d 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1065,6 +1065,7 @@ OBJS-$(CONFIG_XMA_PARSER) += xma_parser.o # bitstream filters OBJS-$(CONFIG_AAC_ADTSTOASC_BSF) += aac_adtstoasc_bsf.o mpeg4audio.o OBJS-$(CONFIG_AV1_METADATA_BSF) += av1_metadata_bsf.o +OBJS-$(CONFIG_AV1_FRAME_SPLIT_BSF) += av1_frame_split_bsf.o OBJS-$(CONFIG_CHOMP_BSF) += chomp_bsf.o OBJS-$(CONFIG_DUMP_EXTRADATA_BSF) += dump_extradata_bsf.o OBJS-$(CONFIG_DCA_CORE_BSF) += dca_core_bsf.o diff --git a/libavcodec/av1_frame_split_bsf.c b/libavcodec/av1_frame_split_bsf.c new file mode 100644 index 0000000000..ce1ae81c42 --- /dev/null +++ b/libavcodec/av1_frame_split_bsf.c @@ -0,0 +1,252 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * This bitstream filter splits AV1 Temporal Units into packets containing + * just one frame, plus any leading and trailing OBU that may be present. + * + * Temporal Units already containing only one frame will be passed through + * unchanged, as will those where splitting couldn't be performed. + * + */ + +#include "libavutil/avassert.h" + +#include "avcodec.h" +#include "bsf.h" +#include "cbs.h" +#include "cbs_av1.h" + +typedef struct AV1FSplitContext { + AVPacket *buffer_pkt; + CodedBitstreamContext *cbc; + CodedBitstreamFragment temporal_unit; + + int nb_frames; + int cur_frame; + int cur_frame_idx; + int last_frame_idx; +} AV1FSplitContext; + +static int av1_frame_split_filter(AVBSFContext *ctx, AVPacket *out) +{ + AV1FSplitContext *s = ctx->priv_data; + CodedBitstreamFragment *td = &s->temporal_unit; + int i, ret; + int split = !!s->buffer_pkt->data; + + if (!s->buffer_pkt->data) { + int nb_frames = 0; + + ret = ff_bsf_get_packet_ref(ctx, s->buffer_pkt); + if (ret < 0) + return ret; + + ret = ff_cbs_read_packet(s->cbc, td, s->buffer_pkt); + if (ret < 0) { + av_log(ctx, AV_LOG_WARNING, "Failed to parse temporal unit.\n"); + goto passthrough; + } + + for (i = 0; i < td->nb_units; i++) { + CodedBitstreamUnit *unit = &td->units[i]; + + if (unit->type == AV1_OBU_FRAME || + unit->type == AV1_OBU_FRAME_HEADER) + nb_frames++; + else if (unit->type == AV1_OBU_TILE_LIST) { + av_log(ctx, AV_LOG_VERBOSE, "Large scale tiles are unsupported.\n"); + goto passthrough; + } + } + if (nb_frames > 1) { + s->cur_frame = 0; + s->cur_frame_idx = s->last_frame_idx = 0; + s->nb_frames = nb_frames; + split = 1; + } + } + + if (split) { + AV1RawFrameHeader *frame = NULL; + int cur_frame_type = -1, size = 0; + + for (i = s->cur_frame_idx; i < td->nb_units; i++) { + CodedBitstreamUnit *unit = &td->units[i]; + + size += unit->data_size; + if (unit->type == AV1_OBU_FRAME) { + AV1RawOBU *obu = unit->content; + + if (frame) { + av_log(ctx, AV_LOG_WARNING, "Frame OBU found when Tile data for a " + "previous frame was expected.\n"); + goto passthrough; + } + + frame = &obu->obu.frame.header; + cur_frame_type = obu->header.obu_type; + s->last_frame_idx = s->cur_frame_idx; + s->cur_frame_idx = i + 1; + s->cur_frame++; + + // split here unless it's the last frame, in which case + // include every trailing OBU + if (s->cur_frame < s->nb_frames) + break; + } else if (unit->type == AV1_OBU_FRAME_HEADER) { + AV1RawOBU *obu = unit->content; + + if (frame) { + av_log(ctx, AV_LOG_WARNING, "Frame Header OBU found when Tile data for a " + "previous frame was expected.\n"); + goto passthrough; + } + + frame = &obu->obu.frame_header; + cur_frame_type = obu->header.obu_type; + s->last_frame_idx = s->cur_frame_idx; + s->cur_frame++; + + // split here if show_existing_frame unless it's the last + // frame, in which case include every trailing OBU + if (frame->show_existing_frame && + s->cur_frame < s->nb_frames) { + s->cur_frame_idx = i + 1; + break; + } + } else if (unit->type == AV1_OBU_TILE_GROUP) { + AV1RawOBU *obu = unit->content; + AV1RawTileGroup *group = &obu->obu.tile_group; + + if (!frame || cur_frame_type != AV1_OBU_FRAME_HEADER) { + av_log(ctx, AV_LOG_WARNING, "Unexpected Tile Group OBU found before a " + "Frame Header.\n"); + goto passthrough; + } + + if ((group->tg_end == (frame->tile_cols * frame->tile_rows) - 1) && + // include every trailing OBU with the last frame + s->cur_frame < s->nb_frames) { + s->cur_frame_idx = i + 1; + break; + } + } + } + av_assert0(frame && s->cur_frame <= s->nb_frames); + + ret = av_packet_ref(out, s->buffer_pkt); + if (ret < 0) + goto fail; + + out->data = (uint8_t *)td->units[s->last_frame_idx].data; + out->size = size; + + if (!frame->show_existing_frame && !frame->show_frame) + out->pts = AV_NOPTS_VALUE; + + if (s->cur_frame == s->nb_frames) { + av_packet_unref(s->buffer_pkt); + ff_cbs_fragment_reset(s->cbc, td); + } + + return 0; + } + +passthrough: + av_packet_move_ref(out, s->buffer_pkt); + + ret = 0; +fail: + if (ret < 0) { + av_packet_unref(out); + av_packet_unref(s->buffer_pkt); + } + ff_cbs_fragment_reset(s->cbc, td); + + return ret; +} + +static const CodedBitstreamUnitType decompose_unit_types[] = { + AV1_OBU_TEMPORAL_DELIMITER, + AV1_OBU_SEQUENCE_HEADER, + AV1_OBU_FRAME_HEADER, + AV1_OBU_TILE_GROUP, + AV1_OBU_FRAME, +}; + +static int av1_frame_split_init(AVBSFContext *ctx) +{ + AV1FSplitContext *s = ctx->priv_data; + CodedBitstreamFragment *td = &s->temporal_unit; + int ret; + + s->buffer_pkt = av_packet_alloc(); + if (!s->buffer_pkt) + return AVERROR(ENOMEM); + + ret = ff_cbs_init(&s->cbc, AV_CODEC_ID_AV1, ctx); + if (ret < 0) + return ret; + + s->cbc->decompose_unit_types = (CodedBitstreamUnitType*)decompose_unit_types; + s->cbc->nb_decompose_unit_types = FF_ARRAY_ELEMS(decompose_unit_types); + + if (!ctx->par_in->extradata_size) + return 0; + + ret = ff_cbs_read_extradata(s->cbc, td, ctx->par_in); + if (ret < 0) + av_log(ctx, AV_LOG_WARNING, "Failed to parse extradata.\n"); + + ff_cbs_fragment_reset(s->cbc, td); + + return 0; +} + +static void av1_frame_split_flush(AVBSFContext *ctx) +{ + AV1FSplitContext *s = ctx->priv_data; + + av_packet_unref(s->buffer_pkt); + ff_cbs_fragment_reset(s->cbc, &s->temporal_unit); +} + +static void av1_frame_split_close(AVBSFContext *ctx) +{ + AV1FSplitContext *s = ctx->priv_data; + + av_packet_free(&s->buffer_pkt); + ff_cbs_fragment_free(s->cbc, &s->temporal_unit); + ff_cbs_close(&s->cbc); +} + +static const enum AVCodecID av1_frame_split_codec_ids[] = { + AV_CODEC_ID_AV1, AV_CODEC_ID_NONE, +}; + +const AVBitStreamFilter ff_av1_frame_split_bsf = { + .name = "av1_frame_split", + .priv_data_size = sizeof(AV1FSplitContext), + .init = av1_frame_split_init, + .flush = av1_frame_split_flush, + .close = av1_frame_split_close, + .filter = av1_frame_split_filter, + .codec_ids = av1_frame_split_codec_ids, +}; diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index 2c999d3c1d..463003966a 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -25,6 +25,7 @@ #include "bsf.h" extern const AVBitStreamFilter ff_aac_adtstoasc_bsf; +extern const AVBitStreamFilter ff_av1_frame_split_bsf; extern const AVBitStreamFilter ff_av1_metadata_bsf; extern const AVBitStreamFilter ff_chomp_bsf; extern const AVBitStreamFilter ff_dump_extradata_bsf;