From patchwork Mon Jul 9 18:26:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 9656 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:104:0:0:0:0:0 with SMTP id c4-v6csp2797739jad; Mon, 9 Jul 2018 11:29:37 -0700 (PDT) X-Google-Smtp-Source: AAOMgpdHCTteb/PkFF+z4llrIzg2ynz12kfvELJhyWUpUnHfK62Zp7JrM/MhGhyEAm7Ne5A/zqBr X-Received: by 2002:a1c:8c08:: with SMTP id o8-v6mr11859253wmd.60.1531160977798; Mon, 09 Jul 2018 11:29:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1531160977; cv=none; d=google.com; s=arc-20160816; b=NkGP24KJ3gya3IzwzU/6PDdtyvEfmLDfb6e4T/tIUqefF6ylCWgm9HuvSasnXpvqPR A0uL1k586PG++IQlspkfFLAUW4Sqe4yXLZengFQ6BqLemPwfHe0xXp6uBwwnvqoI8mlG j7rsefUmEwOQS2Z8U9uLzTz9ehyHzY/FwniihKPbAgmRTNo9mc8/RE1EeHydRgoq8eSm o7xm7OccjyHaLGP8WMFkiKgm2nE5YmCnnyH2V04nrhvxmY2nPwCgLpyszal3P7Yosnvw sOA5G5RM/XStagwHiJleA9wjGn0mXdlstR45Oojb5bO0eRhcmlZtXvyad1tHlQcUNCtA IK1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=LPsr+PG7FirpsObGjbGTeHAonabBLxlCsmiV/Suxg1s=; b=Q3f563h6nNOlg52uyV2kxvupdZQc3XS3/e+w3+d3YmuJM2wX4A0BWp86i2eAl46l3L YU5ALlcdXm6BlU3drZ0lHgWH2iOAYl3LmWcnuTcBMnKmLPVbsfcxn4wkt829myS1Clhu J70txNpDfQlVtMYArelO2KWW0YQW9l3iOGSxsGTwtUbNaP/M8KOV7+pJU2HL+RW0V1gE 6yoQPnJcutoB+6Ld/aWxdta+mUcKGv6mmTltBXUASL624SyQHgjDjchgwpLnFU+PUxlD ZTgl1lxeR97gQqugI2IZYIVe39qG3rXZQW7Fz8ocTROPb61sKtKk5IrNMqpcnn/Ut9Za h1Sw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=UGlsddMi; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m134-v6si10320170wmg.186.2018.07.09.11.29.36; Mon, 09 Jul 2018 11:29:37 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=UGlsddMi; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7B57268A3A4; Mon, 9 Jul 2018 21:29:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt0-f196.google.com (mail-qt0-f196.google.com [209.85.216.196]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2744068A145 for ; Mon, 9 Jul 2018 21:29:22 +0300 (EEST) Received: by mail-qt0-f196.google.com with SMTP id m13-v6so16204671qth.1 for ; Mon, 09 Jul 2018 11:29:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=r5lZu9p2/3BzX3GiS77dsuBUAMK/gbIhzzOrSr8hBf0=; b=UGlsddMi7otoJZBgkXfMa7CtnBx+7clH6nEcHGClSnfk6Y879VnzdcJpG081eGHw3W L4EA1GyPSi6c7CZeywItKH59JEZRCACIyAmQ0A3v3BhZZJQOe7FsiSEemUoiV0PoC8Jl GXTbzojnDVFiC6b6LJbF5P5slkfRD0JK2l6F8mMoyGhtTNi3Qac1mB8k/zQpsZQjGHYT SMBxG08JGrHOwXYjKtAt+xaywHUgQiNOdbPCmfA5Z7DzlnPhhIdDIQF6f8AmxOKKaj05 ufNaRqf7l1dIRsimwec9sOLKkl8XGexSK4xFFw5vr4v3GMRqQ3jYn3wTIw/dgDuBzRsp mL7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=r5lZu9p2/3BzX3GiS77dsuBUAMK/gbIhzzOrSr8hBf0=; b=DYkzM1yUUSjAc3VI0d9uwoqq1R/EpGOaeXHMzwLHGMrPwYARBezG9zNrvjz5vLa4a6 2y4xEvvrfHuzanS9fhwNng3NLictJOCFnUGlt98s2E4FmimYtq6mSW5SxJ2Q44lKlbJF dZuq/Tbyd+zYqVGHipqfFCKivoadP7qw5gQLkY8xW0uw8A3L2coTsTtohA5H+8/L4TRG 5K2dN1gZT9dsAX0RA16YHw+P24WWuyj6GI5FxhHK3Lj01dobh5rssQMG85/A2uXba/lH IhRrdefxjJJLov5Y9cHmL2T/H4Uh6Qn5dc0BGMdLLgpe6WO2yYnELiLZe4BFFUZ7OkqO 3XaA== X-Gm-Message-State: APt69E0fhfH7xJa7UKMjsF+KHFLuUPPCTw9WlRwKeBjcEO8AJUcZV+mu p4pUexq1tRn7JPEcXhRngb77pA== X-Received: by 2002:a0c:9615:: with SMTP id 21-v6mr18838600qvx.147.1531160967508; Mon, 09 Jul 2018 11:29:27 -0700 (PDT) Received: from localhost.localdomain ([190.188.171.140]) by smtp.gmail.com with ESMTPSA id f3-v6sm13138260qtf.61.2018.07.09.11.29.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Jul 2018 11:29:27 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Mon, 9 Jul 2018 15:26:51 -0300 Message-Id: <20180709182654.9996-3-jamrial@gmail.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180709182654.9996-1-jamrial@gmail.com> References: <20180709182654.9996-1-jamrial@gmail.com> Subject: [FFmpeg-devel] [PATCH 2/5] avcodec/extract_extradata: add support for AV1 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Global header OBUs can be exported straight out of containers like ISOBMFF and Matroska, but for the rest (IVF, AV1 Annex-B encapsulation, and of course raw AV1 bitstreams) this is required. Signed-off-by: James Almer --- libavcodec/Makefile | 2 +- libavcodec/av1.h | 42 ++++++++++++++++ libavcodec/extract_extradata_bsf.c | 80 ++++++++++++++++++++++++++++++ 3 files changed, 123 insertions(+), 1 deletion(-) create mode 100644 libavcodec/av1.h diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 67b2626fc0..2d4bc48dab 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1046,7 +1046,7 @@ OBJS-$(CONFIG_DUMP_EXTRADATA_BSF) += dump_extradata_bsf.o OBJS-$(CONFIG_DCA_CORE_BSF) += dca_core_bsf.o OBJS-$(CONFIG_EAC3_CORE_BSF) += eac3_core_bsf.o OBJS-$(CONFIG_EXTRACT_EXTRADATA_BSF) += extract_extradata_bsf.o \ - h2645_parse.o + av1_parse.o h2645_parse.o OBJS-$(CONFIG_FILTER_UNITS_BSF) += filter_units_bsf.o OBJS-$(CONFIG_H264_METADATA_BSF) += h264_metadata_bsf.o OBJS-$(CONFIG_H264_MP4TOANNEXB_BSF) += h264_mp4toannexb_bsf.o diff --git a/libavcodec/av1.h b/libavcodec/av1.h new file mode 100644 index 0000000000..c989b69974 --- /dev/null +++ b/libavcodec/av1.h @@ -0,0 +1,42 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * AV1 common definitions + */ + +#ifndef AVCODEC_AV1_H +#define AVCODEC_AV1_H + +// OBU types (section 6.2.2). +typedef enum { + // 0 reserved. + AV1_OBU_SEQUENCE_HEADER = 1, + AV1_OBU_TEMPORAL_DELIMITER = 2, + AV1_OBU_FRAME_HEADER = 3, + AV1_OBU_TILE_GROUP = 4, + AV1_OBU_METADATA = 5, + AV1_OBU_FRAME = 6, + AV1_OBU_REDUNDANT_FRAME_HEADER = 7, + AV1_OBU_TILE_LIST = 8, + // 9-14 reserved. + AV1_OBU_PADDING = 15, +} AV1_OBU_Type; + +#endif /* AVCODEC_AV1_H */ diff --git a/libavcodec/extract_extradata_bsf.c b/libavcodec/extract_extradata_bsf.c index 082b3e749b..1184ef2a04 100644 --- a/libavcodec/extract_extradata_bsf.c +++ b/libavcodec/extract_extradata_bsf.c @@ -24,6 +24,8 @@ #include "libavutil/opt.h" #include "avcodec.h" +#include "av1.h" +#include "av1_parse.h" #include "bsf.h" #include "h2645_parse.h" #include "h264.h" @@ -36,6 +38,9 @@ typedef struct ExtractExtradataContext { int (*extract)(AVBSFContext *ctx, AVPacket *pkt, uint8_t **data, int *size); + /* AV1 specifc fields */ + AV1Packet av1_pkt; + /* H264/HEVC specifc fields */ H2645Packet h2645_pkt; @@ -52,6 +57,78 @@ static int val_in_array(const int *arr, int len, int val) return 0; } +static int extract_extradata_av1(AVBSFContext *ctx, AVPacket *pkt, + uint8_t **data, int *size) +{ + static const int extradata_obu_types[] = { + AV1_OBU_SEQUENCE_HEADER, AV1_OBU_METADATA, + }; + ExtractExtradataContext *s = ctx->priv_data; + + int extradata_size = 0, filtered_size = 0; + int nb_extradata_obu_types = FF_ARRAY_ELEMS(extradata_obu_types); + int i, ret = 0; + + ret = ff_av1_packet_split(&s->av1_pkt, pkt->data, pkt->size, ctx); + if (ret < 0) + return ret; + + for (i = 0; i < s->av1_pkt.nb_obus; i++) { + AV1OBU *obu = &s->av1_pkt.obus[i]; + if (val_in_array(extradata_obu_types, nb_extradata_obu_types, obu->type)) { + extradata_size += obu->raw_size; + } else if (s->remove) { + filtered_size += obu->raw_size; + } + } + + if (extradata_size) { + AVBufferRef *filtered_buf; + uint8_t *extradata, *filtered_data; + + if (s->remove) { + filtered_buf = av_buffer_alloc(filtered_size + AV_INPUT_BUFFER_PADDING_SIZE); + if (!filtered_buf) { + return AVERROR(ENOMEM); + } + memset(filtered_buf->data + filtered_size, 0, AV_INPUT_BUFFER_PADDING_SIZE); + + filtered_data = filtered_buf->data; + } + + extradata = av_malloc(extradata_size + AV_INPUT_BUFFER_PADDING_SIZE); + if (!extradata) { + av_buffer_unref(&filtered_buf); + return AVERROR(ENOMEM); + } + memset(extradata + extradata_size, 0, AV_INPUT_BUFFER_PADDING_SIZE); + + *data = extradata; + *size = extradata_size; + + for (i = 0; i < s->av1_pkt.nb_obus; i++) { + AV1OBU *obu = &s->av1_pkt.obus[i]; + if (val_in_array(extradata_obu_types, nb_extradata_obu_types, + obu->type)) { + memcpy(extradata, obu->raw_data, obu->raw_size); + extradata += obu->raw_size; + } else if (s->remove) { + memcpy(filtered_data, obu->raw_data, obu->raw_size); + filtered_data += obu->raw_size; + } + } + + if (s->remove) { + av_buffer_unref(&pkt->buf); + pkt->buf = filtered_buf; + pkt->data = filtered_buf->data; + pkt->size = filtered_size; + } + } + + return 0; +} + static int extract_extradata_h2645(AVBSFContext *ctx, AVPacket *pkt, uint8_t **data, int *size) { @@ -251,6 +328,7 @@ static const struct { int (*extract)(AVBSFContext *ctx, AVPacket *pkt, uint8_t **data, int *size); } extract_tab[] = { + { AV_CODEC_ID_AV1, extract_extradata_av1 }, { AV_CODEC_ID_CAVS, extract_extradata_mpeg4 }, { AV_CODEC_ID_H264, extract_extradata_h2645 }, { AV_CODEC_ID_HEVC, extract_extradata_h2645 }, @@ -311,10 +389,12 @@ fail: static void extract_extradata_close(AVBSFContext *ctx) { ExtractExtradataContext *s = ctx->priv_data; + ff_av1_packet_uninit(&s->av1_pkt); ff_h2645_packet_uninit(&s->h2645_pkt); } static const enum AVCodecID codec_ids[] = { + AV_CODEC_ID_AV1, AV_CODEC_ID_CAVS, AV_CODEC_ID_H264, AV_CODEC_ID_HEVC,