From patchwork Tue Nov 12 18:30:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 16224 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 5295544AA0D for ; Tue, 12 Nov 2019 20:30:40 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2C8E568A71F; Tue, 12 Nov 2019 20:30:40 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f180.google.com (mail-qk1-f180.google.com [209.85.222.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4084F68A5C5 for ; Tue, 12 Nov 2019 20:30:34 +0200 (EET) Received: by mail-qk1-f180.google.com with SMTP id 205so15382338qkk.1 for ; Tue, 12 Nov 2019 10:30:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=AGaxEJ1udksSY90ym47QbqtB/ONfczW/HB8RsUqDXSE=; b=bJHqmCA+rs01shV0Au3bffkItL2yLj8AKpRYFm7wa8K92ZU8Lwav4xw9JVRu+31LqY klP10rt9XJBFtC+pvRKgT1LRUNFj7m8Coy9jskuCU2v8ArACNynH7pMD6X0kOe+JMzHM AQj/EFdayKxoSZdlimjQO+gPuHvqpUk41AzkNiDgx+vy6DX3XGu3199nFnc9uHEQmTqL +8kJr5y9MUO3wU01isDTS7HxWTtR7Hk7iBR64l7oeRs46Gwc18zUsgLpxmPz/Ml/E/gW o3lQUjd/Qx8URKdBJjNrAb0kcM0AVmLCu62CutpN0hf3gnJNcSVi5f/brHwcIDpzBpoO I5uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AGaxEJ1udksSY90ym47QbqtB/ONfczW/HB8RsUqDXSE=; b=oFxPcCu7tUrtiuuukp+BELKEroPjW4cta4WDFm6DK8fbUXb4/k5zuprICFnsLIF0Ws YWMHLYMdX3CFEGawpFz9EdsSATszTvESdCeqM1YLz59GwB+Ui6byqaZh2wmyYFxVM0/G vfOcXDUVZ08C1gjUu6xjjphQGI5mX9NM1Rxk/HkjA17GlCa93Y1nfH860sYKgBzJO28D Yj/IZw2hVVXCFNQVPuD4Royh9kH0+lSbgPMVAXTA49D1KD7Qvku15QdytldEjAnctGY1 VfWzuwu8/htMsDb02Gg4XFhqD9/kT5oWzLuLYkxEdB1mB1j/BMm0S/DbQPRfGtY7/6/U Fj+Q== X-Gm-Message-State: APjAAAW2TPdbH0SjYPQTZSUj/GBVPIojD0vASexjw41boHPPvd0VUo8y UNs/o2F8n3x6C+LZYAyNuaFOPuXG X-Google-Smtp-Source: APXvYqyM7KlCNsVWMeuMq5ubVkY+IpTSun8crqMdg9kX0YrVtERWp4udm1PhzIIymTjkmbnHGch/6g== X-Received: by 2002:a37:490c:: with SMTP id w12mr16706953qka.101.1573583432509; Tue, 12 Nov 2019 10:30:32 -0800 (PST) Received: from localhost.localdomain ([181.23.71.177]) by smtp.gmail.com with ESMTPSA id w24sm12113454qta.44.2019.11.12.10.30.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2019 10:30:32 -0800 (PST) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Tue, 12 Nov 2019 15:30:16 -0300 Message-Id: <20191112183016.2710-1-jamrial@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: References: MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/5 v4] avcodec: add an AV1 frame merge bitstream filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: James Almer --- Now using an index to swap fragments. I noticed that the input packet containing props is not necessarily the first in a TU, so adapted the code accordingly. This fixes warnings about missing timestamps when trying to merge the output of av1_frame_split, where the source had the visible frames at the end of a TU rather than at the beginning. Not going to bother with indexes for the packets. av_packet_move_ref() is far from expensive, unlike creating buffer references and expanding the fragment's unit buffer. configure | 1 + libavcodec/Makefile | 1 + libavcodec/av1_frame_merge_bsf.c | 169 +++++++++++++++++++++++++++++++ libavcodec/bitstream_filters.c | 1 + 4 files changed, 172 insertions(+) create mode 100644 libavcodec/av1_frame_merge_bsf.c diff --git a/configure b/configure index 1de90e93fd..70f60997c1 100755 --- a/configure +++ b/configure @@ -3115,6 +3115,7 @@ vc1_parser_select="vc1dsp" # bitstream_filters aac_adtstoasc_bsf_select="adts_header" +av1_frame_merge_bsf_select="cbs_av1" av1_frame_split_bsf_select="cbs_av1" av1_metadata_bsf_select="cbs_av1" eac3_core_bsf_select="ac3_parser" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index b990c6ba87..006a472a6d 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1075,6 +1075,7 @@ OBJS-$(CONFIG_XMA_PARSER) += xma_parser.o # bitstream filters OBJS-$(CONFIG_AAC_ADTSTOASC_BSF) += aac_adtstoasc_bsf.o mpeg4audio.o OBJS-$(CONFIG_AV1_METADATA_BSF) += av1_metadata_bsf.o +OBJS-$(CONFIG_AV1_FRAME_MERGE_BSF) += av1_frame_merge_bsf.o OBJS-$(CONFIG_AV1_FRAME_SPLIT_BSF) += av1_frame_split_bsf.o OBJS-$(CONFIG_CHOMP_BSF) += chomp_bsf.o OBJS-$(CONFIG_DUMP_EXTRADATA_BSF) += dump_extradata_bsf.o diff --git a/libavcodec/av1_frame_merge_bsf.c b/libavcodec/av1_frame_merge_bsf.c new file mode 100644 index 0000000000..96a9e6e863 --- /dev/null +++ b/libavcodec/av1_frame_merge_bsf.c @@ -0,0 +1,169 @@ +/* + * Copyright (c) 2019 James Almer + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "avcodec.h" +#include "bsf.h" +#include "cbs.h" +#include "cbs_av1.h" + +typedef struct AV1FMergeContext { + CodedBitstreamContext *cbc; + CodedBitstreamFragment frag[2]; + AVPacket *pkt, *in; + int idx; +} AV1FMergeContext; + +static int av1_frame_merge_filter(AVBSFContext *bsf, AVPacket *out) +{ + AV1FMergeContext *ctx = bsf->priv_data; + CodedBitstreamFragment *frag = &ctx->frag[ctx->idx], *tu = &ctx->frag[!ctx->idx]; + AVPacket *in = ctx->in, *buffer_pkt = ctx->pkt; + int err, i; + + err = ff_bsf_get_packet_ref(bsf, in); + if (err < 0) { + if (err == AVERROR_EOF && tu->nb_units > 0) + goto eof; + return err; + } + + err = ff_cbs_read_packet(ctx->cbc, frag, in); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to read packet.\n"); + goto fail; + } + + if (frag->nb_units == 0) { + av_log(bsf, AV_LOG_ERROR, "No OBU in packet.\n"); + err = AVERROR_INVALIDDATA; + goto fail; + } + + if (tu->nb_units == 0 && frag->units[0].type != AV1_OBU_TEMPORAL_DELIMITER) { + av_log(bsf, AV_LOG_ERROR, "Missing Temporal Delimiter.\n"); + err = AVERROR_INVALIDDATA; + goto fail; + } + + if (tu->nb_units > 0 && frag->units[0].type == AV1_OBU_TEMPORAL_DELIMITER) { +eof: + err = ff_cbs_write_packet(ctx->cbc, buffer_pkt, tu); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write packet.\n"); + goto fail; + } + av_packet_move_ref(out, buffer_pkt); + + ff_cbs_fragment_reset(ctx->cbc, tu); + + for (i = 1; i < frag->nb_units; i++) { + if (frag->units[i].type == AV1_OBU_TEMPORAL_DELIMITER) { + av_log(bsf, AV_LOG_ERROR, "Temporal Delimiter in the middle of a packet.\n"); + err = AVERROR_INVALIDDATA; + goto fail; + } + } + // Swap fragment index, to avoid copying fragment references. + ctx->idx = !ctx->idx; + + // Buffer the packet if it has a timestamp. + if (in->pts != AV_NOPTS_VALUE) + av_packet_move_ref(buffer_pkt, in); + av_packet_unref(in); + + return 0; + } + + // Buffer packets with timestamps. There should be at most one per TU, be it split or not. + if (!buffer_pkt->data && in->pts != AV_NOPTS_VALUE) + av_packet_move_ref(buffer_pkt, in); + + for (i = 0; i < frag->nb_units; i++) { + if (i && frag->units[i].type == AV1_OBU_TEMPORAL_DELIMITER) { + av_log(bsf, AV_LOG_ERROR, "Temporal Delimiter in the middle of a packet.\n"); + err = AVERROR_INVALIDDATA; + goto fail; + } + err = ff_cbs_insert_unit_content(ctx->cbc, tu, -1, frag->units[i].type, + frag->units[i].content, frag->units[i].content_ref); + if (err < 0) + goto fail; + } + ff_cbs_fragment_reset(ctx->cbc, frag); + av_packet_unref(in); + + return AVERROR(EAGAIN); + +fail: + ff_cbs_fragment_reset(ctx->cbc, tu); + ff_cbs_fragment_reset(ctx->cbc, frag); + av_packet_unref(buffer_pkt); + av_packet_unref(in); + av_packet_unref(out); + + return err; +} + +static int av1_frame_merge_init(AVBSFContext *bsf) +{ + AV1FMergeContext *ctx = bsf->priv_data; + + ctx->in = av_packet_alloc(); + ctx->pkt = av_packet_alloc(); + if (!ctx->in || !ctx->pkt) + return AVERROR(ENOMEM); + + return ff_cbs_init(&ctx->cbc, AV_CODEC_ID_AV1, bsf); +} + +static void av1_frame_merge_flush(AVBSFContext *bsf) +{ + AV1FMergeContext *ctx = bsf->priv_data; + + ff_cbs_fragment_reset(ctx->cbc, &ctx->frag[0]); + ff_cbs_fragment_reset(ctx->cbc, &ctx->frag[1]); + av_packet_unref(ctx->in); + av_packet_unref(ctx->pkt); +} + +static void av1_frame_merge_close(AVBSFContext *bsf) +{ + AV1FMergeContext *ctx = bsf->priv_data; + + ff_cbs_fragment_free(ctx->cbc, &ctx->frag[0]); + ff_cbs_fragment_free(ctx->cbc, &ctx->frag[1]); + av_packet_free(&ctx->in); + av_packet_free(&ctx->pkt); + ff_cbs_close(&ctx->cbc); +} + +static const enum AVCodecID av1_frame_merge_codec_ids[] = { + AV_CODEC_ID_AV1, AV_CODEC_ID_NONE, +}; + +const AVBitStreamFilter ff_av1_frame_merge_bsf = { + .name = "av1_frame_merge", + .priv_data_size = sizeof(AV1FMergeContext), + .init = av1_frame_merge_init, + .flush = av1_frame_merge_flush, + .close = av1_frame_merge_close, + .filter = av1_frame_merge_filter, + .codec_ids = av1_frame_merge_codec_ids, +}; diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index 463003966a..6b5ffe4d70 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -25,6 +25,7 @@ #include "bsf.h" extern const AVBitStreamFilter ff_aac_adtstoasc_bsf; +extern const AVBitStreamFilter ff_av1_frame_merge_bsf; extern const AVBitStreamFilter ff_av1_frame_split_bsf; extern const AVBitStreamFilter ff_av1_metadata_bsf; extern const AVBitStreamFilter ff_chomp_bsf;