From patchwork Wed Oct 19 07:25:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38818 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp216091pzb; Wed, 19 Oct 2022 00:28:06 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4WtRt+NPnT4A+d49UBMl/IhSOmJezH1IsMDXdif7PeH1BcFHaUCNah5w9x4V/eII2Eo+Y3 X-Received: by 2002:a05:6402:51d0:b0:45d:5ba4:c3d2 with SMTP id r16-20020a05640251d000b0045d5ba4c3d2mr6137403edd.132.1666164485809; Wed, 19 Oct 2022 00:28:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666164485; cv=none; d=google.com; s=arc-20160816; b=EJXzsIkVjeiBCCN0td5okpm0u2a5bdKGXXd5bwiyuLQ1Lh60AeRlgGF2+143H4NFjO q27xDOER4HJ/LkMh+5ihKIqDF4q3JRs29zCtvo+ByxYOgmFB2WRflbpSkxPEUV/wT9Sq V4wz6klLeRRQ39MYTybCcFrMNyDLMtlM4uf4sQqOmS2egRisjrzQgTaCf2fE4gS08Ave ElyTTQGh0ZqAxeTFGKai+ye+F8bksEzgOrWYwt6BQHTeI84D/9BuQQpabjP5AH+hp2Pt MutzwWTQO8GUIvZtpQ6qM86rSKImReRDiz4wjviVB9+q2+IN52BnbhdOlGvZTkNmdYWh 7liQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=tYwpqcR29SrhzUGv7h5a3H5AUpFpPLH+mp/jHNYY+fk=; b=CgSoRRGfx3REGAm57GYbIOi6yiDLhg1EGHk2igjJYli83Pnt2eDJciBYtdZgJo3td+ X4OraP52YYvdKBOovMA/wMtw0VPu+8rrOlwWjUj4GyU/2tfW2xZRakgLMTJNhDtuPL8g 5RB7oMjNvtKJC+4TYMb1MbdO1epfHoLAyo7BfLKteIsogCEgPt9jk99fLSJ6hVarjsyy KhBME3NxbOjhawEwoLt6tF49M+q8WuKWj44qckxK5o8m+qSQSX6EMa9YVyfH3FgzR2d1 i4t0A3tQdzj0QaHVxO2ZdffrfA9HNYAvWpE9ugz/zfXdJuh6jDLdhIPQEINKbGaTvI5X 56WA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=NEESUUjY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g6-20020a1709064e4600b0077cb9bc7984si11613795ejw.181.2022.10.19.00.28.05; Wed, 19 Oct 2022 00:28:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=NEESUUjY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1BD0C68BE06; Wed, 19 Oct 2022 10:26:41 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A2AB368BDDE for ; Wed, 19 Oct 2022 10:26:30 +0300 (EEST) Received: by mail-ed1-f52.google.com with SMTP id q19so23892210edd.10 for ; Wed, 19 Oct 2022 00:26:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Vi8tP2cTrmJvwXXjoEzEPj0XCcKgDqCC/RabO8FGEy8=; b=NEESUUjYokl4JyjbQ4t8z6Nk81I4ULN4EcB22k331WXn7s4p06RLq0ZAodfi8d/gd2 nBOaclnU1AmDz7glGv/L3pyeuwMQ7PjxgYSTPzBWfjqbuqZYXuQAMqDe7Ebd/mjLfp87 oRumc/9H1++NZWBdfa+V/1NFwhV/FIV99TLPQXqtKlDgq2R8eBJ7AmSyqcLpHVyegIuJ 5bbf5Y4ujs0f/leFqJhJm1AOq6ed/9xdxv5tZzukyFg9yE1TqrY1FhrtD1Aa5Z3MfQGU Y67zNmQ1g+zDRwsP3Ja6W4Eg3BMPLm3DXZGU8FeCrZ67I6AG1jyLWxvcY/vCUt+313cG 3P5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Vi8tP2cTrmJvwXXjoEzEPj0XCcKgDqCC/RabO8FGEy8=; b=bstOSEFsjpmI2LrjkBz3rIv7ylluS73Uvh5mjdKEDhYL96ZvCPrRblib7FAZ2bh2Gr /fWDWXJVawjrW8jaxgmJxxiaM/J2nnY34lCCjSX/ucr5HjWk1e1ys1o22qEpUIShu37n YFm8fWPskG3fBexy3Gs0OZ0yCbIS2pb5ZP1KiN7anUuxpZ7dUYLwukSdyPbiSE5BvW9/ YQHud+tUb0JyvTfit6dn+vOoPH+uK9fMFD18OEy0fdPfN2N9c9ZoWUlBGTBnRUGDWo+d H/lf+ugK6lq46x24gSpW1kBboMMO6yTPN+752cSrNOV3+yiyBvmEjZtQVxb7oa9vEAVR 17iw== X-Gm-Message-State: ACrzQf2whdGsgeI7mdhSMYwcZw20D8+jAzkWLMkYLANGyZN1njnDj9ny gBiJaN8cW1BHsVDg3Jb9Ds5IVJ2OQyQetA== X-Received: by 2002:a05:6402:28cd:b0:459:19c3:43d0 with SMTP id ef13-20020a05640228cd00b0045919c343d0mr6049870edb.197.1666164390009; Wed, 19 Oct 2022 00:26:30 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id p5-20020a170906498500b0078b83968ad4sm8515150eju.24.2022.10.19.00.26.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 00:26:29 -0700 (PDT) From: thomas.ff@spin-digital.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 19 Oct 2022 09:25:04 +0200 Message-Id: <20221019072508.23460-8-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019072508.23460-1-thomas.ff@spin-digital.com> References: <20221019072508.23460-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 07/11] avformat: add muxer support for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: vngutXKzlNaV From: Thomas Siedel Add muxer for vvcc byte stream format. Add AV_CODEC_ID_VVC to ff_mp4_obj_type Add AV_CODEC_ID_VVC to ISO Media codec (VvcConfigurationBox vvi1, vvc1 defined in ISO/IEC 14496-15:2021) Add VvcConfigurationBox vvcC which extends FullBox type in ISO/IEC 14496-15:2021 Add ff_vvc_muxer to RAW muxers Signed-off-by: Thomas Siedel --- libavformat/Makefile | 7 ++++--- libavformat/allformats.c | 1 + libavformat/isom.c | 1 + libavformat/isom_tags.c | 3 +++ libavformat/mov.c | 6 ++++++ libavformat/movenc.c | 41 +++++++++++++++++++++++++++++++++++++++- libavformat/rawenc.c | 23 ++++++++++++++++++++++ 7 files changed, 78 insertions(+), 4 deletions(-) diff --git a/libavformat/Makefile b/libavformat/Makefile index 00ab4ded89..9ee2526eef 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -336,7 +336,7 @@ OBJS-$(CONFIG_MATROSKA_DEMUXER) += matroskadec.o matroska.o \ oggparsevorbis.o vorbiscomment.o \ qtpalette.o replaygain.o dovi_isom.o OBJS-$(CONFIG_MATROSKA_MUXER) += matroskaenc.o matroska.o \ - av1.o avc.o hevc.o \ + av1.o avc.o hevc.o vvc.o\ flacenc_header.o avlanguage.o \ vorbiscomment.o wv.o dovi_isom.o OBJS-$(CONFIG_MCA_DEMUXER) += mca.o @@ -358,7 +358,7 @@ OBJS-$(CONFIG_MODS_DEMUXER) += mods.o OBJS-$(CONFIG_MOFLEX_DEMUXER) += moflex.o OBJS-$(CONFIG_MOV_DEMUXER) += mov.o mov_chan.o mov_esds.o \ qtpalette.o replaygain.o dovi_isom.o -OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o vpcc.o \ +OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o vvc.o vpcc.o \ movenchint.o mov_chan.o rtp.o \ movenccenc.o movenc_ttml.o rawutils.o \ dovi_isom.o @@ -508,7 +508,7 @@ OBJS-$(CONFIG_RTP_MUXER) += rtp.o \ rtpenc_vp8.o \ rtpenc_vp9.o \ rtpenc_xiph.o \ - avc.o hevc.o + avc.o hevc.o vvc.o OBJS-$(CONFIG_RTSP_DEMUXER) += rtsp.o rtspdec.o httpauth.o \ urldecode.o OBJS-$(CONFIG_RTSP_MUXER) += rtsp.o rtspenc.o httpauth.o \ @@ -596,6 +596,7 @@ OBJS-$(CONFIG_VPK_DEMUXER) += vpk.o OBJS-$(CONFIG_VPLAYER_DEMUXER) += vplayerdec.o subtitles.o OBJS-$(CONFIG_VQF_DEMUXER) += vqf.o OBJS-$(CONFIG_VVC_DEMUXER) += vvcdec.o rawdec.o +OBJS-$(CONFIG_VVC_MUXER) += rawenc.o OBJS-$(CONFIG_W64_DEMUXER) += wavdec.o w64.o pcm.o OBJS-$(CONFIG_W64_MUXER) += wavenc.o w64.o OBJS-$(CONFIG_WAV_DEMUXER) += wavdec.o pcm.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index a4e3822681..006a7dc125 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -475,6 +475,7 @@ extern const AVInputFormat ff_vpk_demuxer; extern const AVInputFormat ff_vplayer_demuxer; extern const AVInputFormat ff_vqf_demuxer; extern const AVInputFormat ff_vvc_demuxer; +extern const AVOutputFormat ff_vvc_muxer; extern const AVInputFormat ff_w64_demuxer; extern const AVOutputFormat ff_w64_muxer; extern const AVInputFormat ff_wav_demuxer; diff --git a/libavformat/isom.c b/libavformat/isom.c index 6d019881e5..9fbccd4437 100644 --- a/libavformat/isom.c +++ b/libavformat/isom.c @@ -36,6 +36,7 @@ const AVCodecTag ff_mp4_obj_type[] = { { AV_CODEC_ID_MPEG4 , 0x20 }, { AV_CODEC_ID_H264 , 0x21 }, { AV_CODEC_ID_HEVC , 0x23 }, + { AV_CODEC_ID_VVC , 0x33 }, { AV_CODEC_ID_AAC , 0x40 }, { AV_CODEC_ID_MP4ALS , 0x40 }, /* 14496-3 ALS */ { AV_CODEC_ID_MPEG2VIDEO , 0x61 }, /* MPEG-2 Main */ diff --git a/libavformat/isom_tags.c b/libavformat/isom_tags.c index e2b80405cc..ec93bdc363 100644 --- a/libavformat/isom_tags.c +++ b/libavformat/isom_tags.c @@ -123,6 +123,9 @@ const AVCodecTag ff_codec_movvideo_tags[] = { { AV_CODEC_ID_HEVC, MKTAG('d', 'v', 'h', 'e') }, /* HEVC-based Dolby Vision derived from hev1 */ /* dvh1 is handled within mov.c */ + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'i', '1') }, /* VVC/H.266 which indicates parameter sets may be in ES */ + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'c', '1') }, /* VVC/H.266 which indicates parameter shall not be in ES */ + { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '1') }, /* AVC-1/H.264 */ { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '2') }, { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '3') }, diff --git a/libavformat/mov.c b/libavformat/mov.c index 1f436e21d6..52719c9d9a 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -1962,6 +1962,11 @@ static int mov_read_glbl(MOVContext *c, AVIOContext *pb, MOVAtom atom) if ((uint64_t)atom.size > (1<<30)) return AVERROR_INVALIDDATA; + if (atom.type == MKTAG('v','v','c','C')) { + avio_rb32(pb); + atom.size -= 4; + } + if (atom.size >= 10) { // Broken files created by legacy versions of libavformat will // wrap a whole fiel atom inside of a glbl atom. @@ -7730,6 +7735,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('s','g','p','d'), mov_read_sgpd }, { MKTAG('s','b','g','p'), mov_read_sbgp }, { MKTAG('h','v','c','C'), mov_read_glbl }, +{ MKTAG('v','v','c','C'), mov_read_glbl }, { MKTAG('u','u','i','d'), mov_read_uuid }, { MKTAG('C','i','n', 0x8e), mov_read_targa_y216 }, { MKTAG('f','r','e','e'), mov_read_free }, diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 754f95912a..3d412d0079 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -59,6 +59,7 @@ #include "libavutil/color_utils.h" #include "libavutil/uuid.h" #include "hevc.h" +#include "vvc.h" #include "rtpenc.h" #include "mov_chan.h" #include "movenc_ttml.h" @@ -1392,6 +1393,23 @@ static int mov_write_hvcc_tag(AVIOContext *pb, MOVTrack *track) return update_size(pb, pos); } +static int mov_write_vvcc_tag(AVIOContext *pb, MOVTrack *track) +{ + int64_t pos = avio_tell(pb); + + avio_wb32(pb, 0); + ffio_wfourcc(pb, "vvcC"); + + avio_w8 (pb, 0); /* version */ + avio_wb24(pb, 0); /* flags */ + + if (track->tag == MKTAG('v','v','i','1')) + ff_isom_write_vvcc(pb, track->vos_data, track->vos_len, 1); + else + ff_isom_write_vvcc(pb, track->vos_data, track->vos_len, 0); + return update_size(pb, pos); +} + /* also used by all avid codecs (dv, imx, meridien) and their variants */ static int mov_write_avid_tag(AVIOContext *pb, MOVTrack *track) { @@ -2286,6 +2304,8 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex avid = 1; } else if (track->par->codec_id == AV_CODEC_ID_HEVC) mov_write_hvcc_tag(pb, track); + else if (track->par->codec_id == AV_CODEC_ID_VVC) + mov_write_vvcc_tag(pb, track); else if (track->par->codec_id == AV_CODEC_ID_H264 && !TAG_IS_AVCI(track->tag)) { mov_write_avcc_tag(pb, track); if (track->mode == MODE_IPOD) @@ -6033,6 +6053,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) if ((par->codec_id == AV_CODEC_ID_DNXHD || par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_HEVC || + par->codec_id == AV_CODEC_ID_VVC || par->codec_id == AV_CODEC_ID_TRUEHD) && !trk->vos_len && !TAG_IS_AVCI(trk->tag)) { /* copy frame to create needed atoms */ @@ -6096,6 +6117,18 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) size = ff_hevc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); } } + } else if (par->codec_id == AV_CODEC_ID_VVC && trk->vos_len > 6 && + (AV_RB24(trk->vos_data) == 1 || AV_RB32(trk->vos_data) == 1)) { + /* extradata is Annex B, assume the bitstream is too and convert it */ + if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { + ret = ff_vvc_annexb2mp4_buf(pkt->data, &reformatted_data, + &size, 0, NULL); + if (ret < 0) + return ret; + avio_write(pb, reformatted_data, size); + } else { + size = ff_vvc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); + } } else if (par->codec_id == AV_CODEC_ID_AV1) { if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { ret = ff_av1_filter_obus_buf(pkt->data, &reformatted_data, @@ -6142,6 +6175,9 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) } else if(par->codec_id == AV_CODEC_ID_HEVC && par->extradata_size > 21) { int nal_size_length = (par->extradata[21] & 0x3) + 1; ret = ff_mov_cenc_avc_write_nal_units(s, &trk->cenc, nal_size_length, pb, pkt->data, size); + } else if(par->codec_id == AV_CODEC_ID_VVC && par->extradata_size > 21) { + int nal_size_length = (par->extradata[21] & 0x3) + 1; + ret = ff_mov_cenc_avc_write_nal_units(s, &trk->cenc, nal_size_length, pb, pkt->data, size); } else { ret = ff_mov_cenc_write_packet(&trk->cenc, pb, pkt->data, size); } @@ -7224,7 +7260,8 @@ static int mov_init(AVFormatContext *s) if (mov->encryption_scheme == MOV_ENC_CENC_AES_CTR) { ret = ff_mov_cenc_init(&track->cenc, mov->encryption_key, - (track->par->codec_id == AV_CODEC_ID_H264 || track->par->codec_id == AV_CODEC_ID_HEVC), + (track->par->codec_id == AV_CODEC_ID_H264 || track->par->codec_id == AV_CODEC_ID_HEVC || + track->par->codec_id == AV_CODEC_ID_VVC), s->flags & AVFMT_FLAG_BITEXACT); if (ret) return ret; @@ -7692,6 +7729,8 @@ static const AVCodecTag codec_mp4_tags[] = { { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '3') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'e', 'v', '1') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'v', 'c', '1') }, + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'c', '1') }, + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'i', '1') }, { AV_CODEC_ID_MPEG2VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MPEG1VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MJPEG, MKTAG('m', 'p', '4', 'v') }, diff --git a/libavformat/rawenc.c b/libavformat/rawenc.c index 267fce252d..7770d63d10 100644 --- a/libavformat/rawenc.c +++ b/libavformat/rawenc.c @@ -401,6 +401,29 @@ const AVOutputFormat ff_hevc_muxer = { }; #endif +#if CONFIG_VVC_MUXER +static int vvc_check_bitstream(AVFormatContext *s, AVStream *st, + const AVPacket *pkt) +{ + if (pkt->size >= 5 && AV_RB32(pkt->data) != 0x0000001 && + AV_RB24(pkt->data) != 0x000001) + return ff_stream_add_bitstream_filter(st, "vvc_mp4toannexb", NULL); + return 1; +} + +const AVOutputFormat ff_vvc_muxer = { + .name = "vvc", + .long_name = NULL_IF_CONFIG_SMALL("raw VVC video"), + .extensions = "vvc,h266,266", + .audio_codec = AV_CODEC_ID_NONE, + .video_codec = AV_CODEC_ID_VVC, + .init = force_one_stream, + .write_packet = ff_raw_write_packet, + .check_bitstream = vvc_check_bitstream, + .flags = AVFMT_NOTIMESTAMPS, +}; +#endif + #if CONFIG_M4V_MUXER const AVOutputFormat ff_m4v_muxer = { .name = "m4v",