From patchwork Mon Jul 9 18:26:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 9660 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:104:0:0:0:0:0 with SMTP id c4-v6csp2803727jad; Mon, 9 Jul 2018 11:36:35 -0700 (PDT) X-Google-Smtp-Source: AAOMgpdgpGAnWZkWXeK25ljUDAkBwGVje9SUaY/Vo0gsxsqYi7OnlFXCg/KLqRCFFkvevGrtgdjS X-Received: by 2002:a1c:ca0f:: with SMTP id a15-v6mr12518122wmg.102.1531161395502; Mon, 09 Jul 2018 11:36:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1531161395; cv=none; d=google.com; s=arc-20160816; b=oUYuTJwO0eYp6WsVO9Bn7tN9ke2kmuoGM/SihcY2Mf5prD5MdYESkoKZD7TFH/XziL uwQns7ZwIl753JHY/jje/hAozWoHmzjvYQkWR5suYk+aJ4THLzH9vHvoJkToIZYIjlB/ X9CBIeYMoRSCFIzUMyf+YrSyQfl81sJXH+urN36kySsSoA4EAdQJAdbdW0u0y/fhInHS 9XQepxG4X69/zyolLLGP/NSiQcnVRDPRYQhpZKfEI4nwAhgLpQY/JD0wNTaLRq7m3BCz u+t+iHmPYogeAKHm3TUH2XZRvsPli6MgjeuuuNJqGZfwA2P0YCxcVZXEsE8xqrAm4tas 9FPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=d5KYQM+VbVaPSzC02SnK8kC0yfODMut6U3IlwMLH+3g=; b=GnMsPuJlGleln/P162lQe7usBDDDi8qeWhFZCg1OGXcPj7QSfoSNj0drVQoBzCkGJa WOgNqH0KSxVNCc6y18L9Idqr/8CXNDFuUGZFpBkFyarv8QGoxmgdlZl8PrSZG4VgoX1g Z/nOI8qWaIpsgRbnc7dDhFBATNAeGeqBkfv+KkzCypsGcAvBpApvtQSoCCwjkXlT+psS AgYBxPQ1ZGnt5xexgFPpxhpQ8mPnZHrk+zyezsrukFSJd197LAnWrGFfZ8kl2DspI9KV S3oWT9xo0xUtn8Hf3rNJLPP4Klu98HXm5gwBEFyewWfnN5ZuAoKuiEWAZ/GgBltWy4Wq u96Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=fF+P7j37; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j25-v6si9547137wme.67.2018.07.09.11.36.35; Mon, 09 Jul 2018 11:36:35 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=fF+P7j37; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 63B9D68A4F9; Mon, 9 Jul 2018 21:36:27 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk0-f193.google.com (mail-qk0-f193.google.com [209.85.220.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C116C68A4B0 for ; Mon, 9 Jul 2018 21:36:20 +0300 (EEST) Received: by mail-qk0-f193.google.com with SMTP id c131-v6so10219556qkb.0 for ; Mon, 09 Jul 2018 11:36:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=yeYPmMrIvQlci3CRq1c/7mUW+eYy3qt4Py5VUBuvHS0=; b=fF+P7j37KYSkKQ9GgVLVHvN+wVjVDYE6RdsPgzKqVF8V9I3Zz5PVn5oA4ACfjbEFcb GUhPkzdtBrVHmQ7naGgHo97c6X8DKpbViVow07DLEDdZVFariCJ6XnW+8+VVPhMxnxw1 Gn941VxFQ0tBLYeQCjhBWiB1UeD6/4zIoHhwaOjUmSBfVK9x2/qAJhbViaXe7h1s6OyY fpwus1JP6IF1eVxu5ZDtiUkwBx3g+b97w01pntadJO2uT5P3idNPmXL1Y3v28k6HoA/K 83dcwirEKpZCrvQJZcMHzcMGwnbwUAf0mYaE1d/Kk9AV0I3BimrIjgdJN9LjUGBu5Etz L3mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=yeYPmMrIvQlci3CRq1c/7mUW+eYy3qt4Py5VUBuvHS0=; b=I4QxH5y48Ub2lwCBCIGNib10f09xCx3gF9lBypLxfLEPrnCvA8JHh3iFtF4jrIYepz a19BrWEOQP+DLwmcSTvdnOlasyipgRqk7npiRdmoqSmS4Xruqg862D1F6Q8Z/UPhHSzw z7JHti94zltsPz0mlpQ5fRR2C/Wqyux4pqRhWkRT95UydiTybkLz0bHVIICWkbHAkS+p Hgp0qZcWtp63+GYPW9Bvsl6NFkq/FknzXM9ovlFR1kOMKTpHO9s1W3kcXPV0X7giNFe6 AD+Th9BrQiXM4cWhsYhGDloDJukG+IZFjqHouhLiVr/4djWmoqY8NN6meHN9fFJU1Zyr uneg== X-Gm-Message-State: APt69E10LAptUeeXmsTfK+NqraWMqKNnF8kYgC+zAhqcnYnowZhTqDOO rXYh45KwE7Dsbw1JCPGkfb1CGA== X-Received: by 2002:a37:9505:: with SMTP id x5-v6mr18438379qkd.25.1531160970888; Mon, 09 Jul 2018 11:29:30 -0700 (PDT) Received: from localhost.localdomain ([190.188.171.140]) by smtp.gmail.com with ESMTPSA id f3-v6sm13138260qtf.61.2018.07.09.11.29.29 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Jul 2018 11:29:30 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Mon, 9 Jul 2018 15:26:54 -0300 Message-Id: <20180709182654.9996-6-jamrial@gmail.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180709182654.9996-1-jamrial@gmail.com> References: <20180709182654.9996-1-jamrial@gmail.com> Subject: [FFmpeg-devel] [PATCH 5/5] avformat/movenc: add support for AV1 streams X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: James Almer --- ff_av1_filter_obus() could eventually be replaced by an autoinserted filter_units bsf, assuming it doesn't slow down the muxing process too much (CBS is fast reading packets, but not so much assembling and writing packets). ff_isom_write_av1c() however can't be replaced given filter_units doesn't handle extradata (either codecpar or packet side data). libavformat/Makefile | 2 +- libavformat/av1.c | 107 +++++++++++++++++++++++++++++++++++++++++++ libavformat/av1.h | 70 ++++++++++++++++++++++++++++ libavformat/movenc.c | 36 ++++++++++++--- 4 files changed, 207 insertions(+), 8 deletions(-) create mode 100644 libavformat/av1.c create mode 100644 libavformat/av1.h diff --git a/libavformat/Makefile b/libavformat/Makefile index 8fb075f06f..f2f3aabdc2 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -302,7 +302,7 @@ OBJS-$(CONFIG_MM_DEMUXER) += mm.o OBJS-$(CONFIG_MMF_DEMUXER) += mmf.o OBJS-$(CONFIG_MMF_MUXER) += mmf.o rawenc.o OBJS-$(CONFIG_MOV_DEMUXER) += mov.o mov_chan.o mov_esds.o replaygain.o -OBJS-$(CONFIG_MOV_MUXER) += movenc.o avc.o hevc.o vpcc.o \ +OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o vpcc.o \ movenchint.o mov_chan.o rtp.o \ movenccenc.o rawutils.o OBJS-$(CONFIG_MP2_MUXER) += rawenc.o diff --git a/libavformat/av1.c b/libavformat/av1.c new file mode 100644 index 0000000000..3b200176a5 --- /dev/null +++ b/libavformat/av1.c @@ -0,0 +1,107 @@ +/* + * AV1 helper functions for muxers + * Copyright (c) 2018 James Almer + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/mem.h" +#include "libavcodec/av1.h" +#include "libavcodec/av1_parse.h" +#include "av1.h" +#include "avio.h" + +int ff_av1_filter_obus(AVIOContext *pb, const uint8_t *buf, int size) +{ + const uint8_t *end = buf + size; + int64_t obu_size; + int start_pos, type, temporal_id, spatial_id; + + size = 0; + while (buf < end) { + int ret = parse_obu_header(buf, end - buf, &obu_size, &start_pos, + &type, &temporal_id, &spatial_id); + if (ret < 0) + return ret; + + obu_size += start_pos; + if (obu_size > INT_MAX) + return AVERROR_INVALIDDATA; + + switch (type) { + case AV1_OBU_TEMPORAL_DELIMITER: + case AV1_OBU_REDUNDANT_FRAME_HEADER: + case AV1_OBU_PADDING: + break; + default: + avio_write(pb, buf, obu_size); + size += obu_size; + break; + } + buf += obu_size; + } + + return size; +} + +int ff_av1_filter_obus_buf(const uint8_t *buf, uint8_t **out, int *size) +{ + AVIOContext *pb; + int ret; + + ret = avio_open_dyn_buf(&pb); + if (ret < 0) + return ret; + + ret = ff_av1_filter_obus(pb, buf, *size); + *size = avio_close_dyn_buf(pb, out); + + if (ret < 0) + av_freep(out); + + return ret; +} + +int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size) +{ + int64_t obu_size; + int start_pos, type, temporal_id, spatial_id; + + while (size > 0) { + int ret = parse_obu_header(buf, size, &obu_size, &start_pos, + &type, &temporal_id, &spatial_id); + if (ret < 0) + return ret; + + obu_size += start_pos; + if (obu_size > INT_MAX) + return AVERROR_INVALIDDATA; + + switch (type) { + case AV1_OBU_SEQUENCE_HEADER: + case AV1_OBU_METADATA: + avio_write(pb, buf, obu_size); + size -= obu_size; + break; + default: + break; + } + buf += obu_size; + } + + return 0; +} diff --git a/libavformat/av1.h b/libavformat/av1.h new file mode 100644 index 0000000000..733034c12d --- /dev/null +++ b/libavformat/av1.h @@ -0,0 +1,70 @@ +/* + * AV1 helper functions for muxers + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFORMAT_AV1_H +#define AVFORMAT_AV1_H + +#include + +#include "avio.h" + +/** + * Filter out AV1 OBUs not meant to be present in ISOBMFF sample data and write + * the resulting bitstream to the provided AVIOContext. + * + * @param pb pointer to the AVIOContext where the filtered bitstream shall be + * written + * @param buf input data buffer + * @param size size of the input data buffer + * + * @return the amount of bytes written in case of success, a negative AVERROR + * code in case of failure + */ +int ff_av1_filter_obus(AVIOContext *pb, const uint8_t *buf, int size); + +/** + * Filter out AV1 OBUs not meant to be present in ISOBMFF sample data and write + * the resulting bitstream to a newly allocated data buffer. + * + * @param pb pointer to the AVIOContext where the filtered bitstream shall be + * written + * @param buf input data buffer + * @param out pointer to pointer that will hold the allocated data buffer + * @param size size of the input data buffer. The size of the resulting output + data buffer will be written here + * + * @return the amount of bytes written in case of success, a negative AVERROR + * code in case of failure + */ +int ff_av1_filter_obus_buf(const uint8_t *buf, uint8_t **out, int *size); + +/** + * Writes AV1 extradata (Sequence Header and Metadata OBUs) to the provided + * AVIOContext. + * + * @param pb pointer to the AVIOContext where the hvcC shall be written + * @param buf input data buffer + * @param size size of the input data buffer + * + * @return 0 in case of success, a negative AVERROR code in case of failure + */ +int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size); + +#endif /* AVFORMAT_AV1_H */ diff --git a/libavformat/movenc.c b/libavformat/movenc.c index fe0a244a8f..784df6d08d 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -30,6 +30,7 @@ #include "riff.h" #include "avio.h" #include "isom.h" +#include "av1.h" #include "avc.h" #include "libavcodec/ac3_parser_internal.h" #include "libavcodec/dnxhddata.h" @@ -1163,6 +1164,19 @@ static int mov_write_d263_tag(AVIOContext *pb) return 0xf; } +static int mov_write_av1c_tag(AVIOContext *pb, MOVTrack *track) +{ + int64_t pos = avio_tell(pb); + + avio_wb32(pb, 0); + ffio_wfourcc(pb, "av1C"); + avio_w8(pb, 0); /* version */ + avio_wb24(pb, 0); /* flags */ + avio_w8(pb, 0); /* reserved (3), initial_presentation_delay_present (1), initial_presentation_delay_minus_one/reserved (4) */ + ff_isom_write_av1c(pb, track->vos_data, track->vos_len); + return update_size(pb, pos); +} + static int mov_write_avcc_tag(AVIOContext *pb, MOVTrack *track) { int64_t pos = avio_tell(pb); @@ -2009,6 +2023,8 @@ static int mov_write_video_tag(AVIOContext *pb, MOVMuxContext *mov, MOVTrack *tr mov_write_uuid_tag_ipod(pb); } else if (track->par->codec_id == AV_CODEC_ID_VP9) { mov_write_vpcc_tag(mov->fc, pb, track); + } else if (track->par->codec_id == AV_CODEC_ID_AV1) { + mov_write_av1c_tag(pb, track); } else if (track->par->codec_id == AV_CODEC_ID_VC1 && track->vos_len > 0) mov_write_dvc1_tag(pb, track); else if (track->par->codec_id == AV_CODEC_ID_VP6F || @@ -5319,6 +5335,13 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) } else { size = ff_hevc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); } + } else if (par->codec_id == AV_CODEC_ID_AV1) { + if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { + ff_av1_filter_obus_buf(pkt->data, &reformatted_data, &size); + avio_write(pb, reformatted_data, size); + } else { + size = ff_av1_filter_obus(pb, pkt->data, pkt->size); + } #if CONFIG_AC3_PARSER } else if (par->codec_id == AV_CODEC_ID_EAC3) { size = handle_eac3(mov, pkt, trk); @@ -5438,7 +5461,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) av_log(s, AV_LOG_WARNING, "pts has no value\n"); pkt->pts = pkt->dts; } - if (pkt->dts != pkt->pts) + if (pkt->dts != pkt->pts && par->codec_id != AV_CODEC_ID_AV1) trk->flags |= MOV_TRACK_CTTS; trk->cluster[trk->entry].cts = pkt->pts - pkt->dts; trk->cluster[trk->entry].flags = 0; @@ -5512,6 +5535,7 @@ static int mov_write_single_packet(AVFormatContext *s, AVPacket *pkt) if (trk->par->codec_id == AV_CODEC_ID_MP4ALS || trk->par->codec_id == AV_CODEC_ID_AAC || + trk->par->codec_id == AV_CODEC_ID_AV1 || trk->par->codec_id == AV_CODEC_ID_FLAC) { int side_size = 0; uint8_t *side = av_packet_get_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, &side_size); @@ -6209,15 +6233,12 @@ static int mov_init(AVFormatContext *s) pix_fmt == AV_PIX_FMT_MONOWHITE || pix_fmt == AV_PIX_FMT_MONOBLACK; } - if (track->par->codec_id == AV_CODEC_ID_VP9) { + if (track->par->codec_id == AV_CODEC_ID_VP9 || + track->par->codec_id == AV_CODEC_ID_AV1) { if (track->mode != MODE_MP4) { - av_log(s, AV_LOG_ERROR, "VP9 only supported in MP4.\n"); + av_log(s, AV_LOG_ERROR, "%s only supported in MP4.\n", avcodec_get_name(track->par->codec_id)); return AVERROR(EINVAL); } - } else if (track->par->codec_id == AV_CODEC_ID_AV1) { - /* spec is not finished, so forbid for now */ - av_log(s, AV_LOG_ERROR, "AV1 muxing is currently not supported.\n"); - return AVERROR_PATCHWELCOME; } else if (track->par->codec_id == AV_CODEC_ID_VP8) { /* altref frames handling is not defined in the spec as of version v1.0, * so just forbid muxing VP8 streams altogether until a new version does */ @@ -6732,6 +6753,7 @@ const AVCodecTag codec_mp4_tags[] = { { AV_CODEC_ID_DIRAC , MKTAG('d', 'r', 'a', 'c') }, { AV_CODEC_ID_TSCC2 , MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_VP9 , MKTAG('v', 'p', '0', '9') }, + { AV_CODEC_ID_AV1 , MKTAG('a', 'v', '0', '1') }, { AV_CODEC_ID_AAC , MKTAG('m', 'p', '4', 'a') }, { AV_CODEC_ID_MP4ALS , MKTAG('m', 'p', '4', 'a') }, { AV_CODEC_ID_MP3 , MKTAG('m', 'p', '4', 'a') },