From patchwork Tue Nov 12 15:00:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 16223 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B874244A640 for ; Tue, 12 Nov 2019 17:01:27 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8CA3568A6EB; Tue, 12 Nov 2019 17:01:27 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f194.google.com (mail-qt1-f194.google.com [209.85.160.194]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7C6336898E4 for ; Tue, 12 Nov 2019 17:01:21 +0200 (EET) Received: by mail-qt1-f194.google.com with SMTP id y10so20056216qto.3 for ; Tue, 12 Nov 2019 07:01:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=FZA+/1L3H+Lm6sPtqaotBFnieOjKCZvSNgIePDgWUV0=; b=HtkDBoMKZ5L/dCejgrWdSpiHtJZGhAq56byIyUGKuKPiXo+pG3z3B1PXee9Z4OCcu+ dmeupjs/sdBN5b37QfPk54iPoVsNgksWFgHq9+LCTWpIb8gGb8sHLCDqqn0dNdVYFCiB 6XMBOhrBp3o3p/qOaFpryBMIahJvMLD8Ia5F+9ApYwwSaxnVrCZOtssr7DahHOdLWd/v M6+iHXKfr5YmHVuiLc/GnKh9jLhS2psXOq8k6TOpVxytETjjcwJI2kvtpw1FwnZM2K/d 9mY3QQFIu2JvnHvXiMe33RJQPYHEvDloPvASwQG3a5nfDGPq+9kh1oE0JWVfuzeF0ZfB np3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FZA+/1L3H+Lm6sPtqaotBFnieOjKCZvSNgIePDgWUV0=; b=acdA0SXvcqe6gK6C9JGlI8FxPVdNG/MVBsccL9hcOuzk5ei663EYItYJR1leeOE1zT Ptu4cIMQvfPqd+VpyhnB+jYwYbnvE2GzGPLw4n5aJQThCybmnE9uAI/hY9igid3+2+Jk pWos77uM/VdcfFOvEXsijI+oiKGt11MEKCUdtJdVm7MY8g3v1AR3kgVomUs0fjv2Ioc3 SnLVK+1oPa/uWuUv1Di7ihOR4crqHmvnb8z/bcRfahLI84ps436st4znGU223aG+2i2E eGmB7W5skqEwb8lvxayhYd0reTsdELf++74IMS/xJ7eW93dy8n3w3168ioVNMux5jjOi TzPA== X-Gm-Message-State: APjAAAVt8NGFLCn5ezQmYuBaylZkNDMambKfGpW+yIwmpo8x3W476aUx 6qGyRgbZCsobWJ7PxO7D2dHwKA45 X-Google-Smtp-Source: APXvYqzheQHnvfCtaThMI0cuPcDvQNA/sqImyosFgcUiYHSeJ24mCQykWtznzb0m6oHG5hRFaBm2pw== X-Received: by 2002:ac8:4a93:: with SMTP id l19mr1674185qtq.121.1573570879741; Tue, 12 Nov 2019 07:01:19 -0800 (PST) Received: from localhost.localdomain ([181.23.83.211]) by smtp.gmail.com with ESMTPSA id k26sm3600946qtm.10.2019.11.12.07.01.18 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2019 07:01:19 -0800 (PST) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Tue, 12 Nov 2019 12:00:40 -0300 Message-Id: <20191112150040.2003-1-jamrial@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <2b4092dd-3fc4-7357-e8a5-0a163f9c32fe@gmail.com> References: <2b4092dd-3fc4-7357-e8a5-0a163f9c32fe@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/5 v3] avcodec: add an AV1 frame merge BSF X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: James Almer --- Assorted improvements to avoid creating new references, copying packet properties, etc. configure | 1 + libavcodec/Makefile | 1 + libavcodec/av1_frame_merge_bsf.c | 173 +++++++++++++++++++++++++++++++ libavcodec/bitstream_filters.c | 1 + 4 files changed, 176 insertions(+) create mode 100644 libavcodec/av1_frame_merge_bsf.c diff --git a/configure b/configure index 1de90e93fd..70f60997c1 100755 --- a/configure +++ b/configure @@ -3115,6 +3115,7 @@ vc1_parser_select="vc1dsp" # bitstream_filters aac_adtstoasc_bsf_select="adts_header" +av1_frame_merge_bsf_select="cbs_av1" av1_frame_split_bsf_select="cbs_av1" av1_metadata_bsf_select="cbs_av1" eac3_core_bsf_select="ac3_parser" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index b990c6ba87..006a472a6d 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1075,6 +1075,7 @@ OBJS-$(CONFIG_XMA_PARSER) += xma_parser.o # bitstream filters OBJS-$(CONFIG_AAC_ADTSTOASC_BSF) += aac_adtstoasc_bsf.o mpeg4audio.o OBJS-$(CONFIG_AV1_METADATA_BSF) += av1_metadata_bsf.o +OBJS-$(CONFIG_AV1_FRAME_MERGE_BSF) += av1_frame_merge_bsf.o OBJS-$(CONFIG_AV1_FRAME_SPLIT_BSF) += av1_frame_split_bsf.o OBJS-$(CONFIG_CHOMP_BSF) += chomp_bsf.o OBJS-$(CONFIG_DUMP_EXTRADATA_BSF) += dump_extradata_bsf.o diff --git a/libavcodec/av1_frame_merge_bsf.c b/libavcodec/av1_frame_merge_bsf.c new file mode 100644 index 0000000000..a2cb09b55e --- /dev/null +++ b/libavcodec/av1_frame_merge_bsf.c @@ -0,0 +1,173 @@ +/* + * Copyright (c) 2019 James Almer + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "avcodec.h" +#include "bsf.h" +#include "cbs.h" +#include "cbs_av1.h" + +typedef struct AV1FMergeContext { + CodedBitstreamContext *cbc; + CodedBitstreamFragment *temporal_unit; + CodedBitstreamFragment *frag; + AVPacket *buffer_pkt; + AVPacket *in; +} AV1FMergeContext; + +static int av1_frame_merge_filter(AVBSFContext *bsf, AVPacket *out) +{ + AV1FMergeContext *ctx = bsf->priv_data; + CodedBitstreamFragment *frag = ctx->frag, *tu = ctx->temporal_unit; + int err, i; + + err = ff_bsf_get_packet_ref(bsf, ctx->in); + if (err < 0) { + if (err == AVERROR_EOF && tu->nb_units > 0) + goto eof; + return err; + } + + err = ff_cbs_read_packet(ctx->cbc, frag, ctx->in); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to read packet.\n"); + goto fail; + } + + if (frag->nb_units == 0) { + av_log(bsf, AV_LOG_ERROR, "No OBU in packet.\n"); + err = AVERROR_INVALIDDATA; + goto fail; + } + + if (tu->nb_units == 0) { + if (frag->units[0].type != AV1_OBU_TEMPORAL_DELIMITER) { + av_log(bsf, AV_LOG_ERROR, "Missing Temporal Delimiter.\n"); + err = AVERROR_INVALIDDATA; + goto fail; + } + av_packet_move_ref(ctx->buffer_pkt, ctx->in); + } + + if (tu->nb_units > 0 && frag->units[0].type == AV1_OBU_TEMPORAL_DELIMITER) { +eof: + err = ff_cbs_write_packet(ctx->cbc, ctx->buffer_pkt, tu); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write packet.\n"); + goto fail; + } + av_packet_move_ref(out, ctx->buffer_pkt); + + ff_cbs_fragment_reset(ctx->cbc, tu); + + for (i = 1; i < frag->nb_units; i++) { + if (frag->units[i].type == AV1_OBU_TEMPORAL_DELIMITER) { + av_log(bsf, AV_LOG_ERROR, "Temporal Delimiter in the middle of a packet.\n"); + err = AVERROR_INVALIDDATA; + goto fail; + } + } + // Swap pointers to avoid copying unit content. + ctx->temporal_unit = frag; + ctx->frag = tu; + + av_packet_move_ref(ctx->buffer_pkt, ctx->in); + + return 0; + } + + for (i = 0; i < frag->nb_units; i++) { + if (i && frag->units[i].type == AV1_OBU_TEMPORAL_DELIMITER) { + av_log(bsf, AV_LOG_ERROR, "Temporal Delimiter in the middle of a packet.\n"); + err = AVERROR_INVALIDDATA; + goto fail; + } + err = ff_cbs_insert_unit_content(ctx->cbc, tu, -1, frag->units[i].type, + frag->units[i].content, frag->units[i].content_ref); + if (err < 0) + goto fail; + } + ff_cbs_fragment_reset(ctx->cbc, frag); + av_packet_unref(ctx->in); + + return AVERROR(EAGAIN); + +fail: + ff_cbs_fragment_reset(ctx->cbc, tu); + ff_cbs_fragment_reset(ctx->cbc, frag); + av_packet_unref(ctx->buffer_pkt); + av_packet_unref(ctx->in); + av_packet_unref(out); + + return err; +} + +static int av1_frame_merge_init(AVBSFContext *bsf) +{ + AV1FMergeContext *ctx = bsf->priv_data; + + ctx->temporal_unit = av_mallocz(sizeof(*ctx->temporal_unit)); + ctx->frag = av_mallocz(sizeof(*ctx->frag)); + ctx->buffer_pkt = av_packet_alloc(); + ctx->in = av_packet_alloc(); + if (!ctx->temporal_unit || !ctx->frag || + !ctx->buffer_pkt || !ctx->in) + return AVERROR(ENOMEM); + + return ff_cbs_init(&ctx->cbc, AV_CODEC_ID_AV1, bsf); +} + +static void av1_frame_merge_flush(AVBSFContext *bsf) +{ + AV1FMergeContext *ctx = bsf->priv_data; + + ff_cbs_fragment_reset(ctx->cbc, &ctx->temporal_unit); + ff_cbs_fragment_reset(ctx->cbc, &ctx->frag); + av_packet_unref(ctx->buffer_pkt); + av_packet_unref(ctx->in); +} + +static void av1_frame_merge_close(AVBSFContext *bsf) +{ + AV1FMergeContext *ctx = bsf->priv_data; + + if (ctx->temporal_unit) + ff_cbs_fragment_free(ctx->cbc, ctx->temporal_unit); + if (ctx->frag) + ff_cbs_fragment_free(ctx->cbc, ctx->frag); + av_freep(&ctx->temporal_unit); + av_freep(&ctx->frag); + av_packet_free(&ctx->buffer_pkt); + av_packet_free(&ctx->in); + ff_cbs_close(&ctx->cbc); +} + +static const enum AVCodecID av1_frame_merge_codec_ids[] = { + AV_CODEC_ID_AV1, AV_CODEC_ID_NONE, +}; + +const AVBitStreamFilter ff_av1_frame_merge_bsf = { + .name = "av1_frame_merge", + .priv_data_size = sizeof(AV1FMergeContext), + .init = av1_frame_merge_init, + .flush = av1_frame_merge_flush, + .close = av1_frame_merge_close, + .filter = av1_frame_merge_filter, + .codec_ids = av1_frame_merge_codec_ids, +}; diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index 463003966a..6b5ffe4d70 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -25,6 +25,7 @@ #include "bsf.h" extern const AVBitStreamFilter ff_aac_adtstoasc_bsf; +extern const AVBitStreamFilter ff_av1_frame_merge_bsf; extern const AVBitStreamFilter ff_av1_frame_split_bsf; extern const AVBitStreamFilter ff_av1_metadata_bsf; extern const AVBitStreamFilter ff_chomp_bsf;