From patchwork Tue Mar 21 15:01:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 40760 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d046:b0:cd:afd7:272c with SMTP id hv6csp2661661pzb; Tue, 21 Mar 2023 08:03:33 -0700 (PDT) X-Google-Smtp-Source: AK7set/fl3RhD4PKCZp0uxLYB44Eddw/7r+vDxcQbrupTpTxRZx3wA4SucHow7ycmJ8I6A1DrjEv X-Received: by 2002:a17:907:78d3:b0:930:8885:a629 with SMTP id kv19-20020a17090778d300b009308885a629mr2922099ejc.33.1679411013222; Tue, 21 Mar 2023 08:03:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679411013; cv=none; d=google.com; s=arc-20160816; b=rKFnZawiZQFO/zJk0xSkmm9u6gelIje9YWI3Nnql9T7bYOPe4m6WYcu2X0Mv/CeV8R VykGvSAn59ephmL4tEyBG7C/vlu2O5xqaGQmdWje0v8zPAJSJ5Vo317kq2CA2ANzqzu/ 2KUq7dBkauNY1JYHBSkk7wjzlqM6Sgtq0cFa0peFBQ/odz6gdzC0Bmd9Xzpvg4eA6h2J OrOzqe6BXEmtHtUCAl5iU2tKiCwJcK9C8/SrJdCGvuAaHzpARNI0Dut2b5DFDgmzu5Ca yJ129GOK4ZchaxCV3baVW0XUeC/o34p+cyUbDJBoYDBKnwUoINkZtyFrvP7pGlBrSvr3 VTFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=lTKW25bLaO7KL9ksFveA9rTfwvOEInew8uW5CZjalls=; b=W8486d0rHfNSF3uxu96eyoTAK5T/vkYqh2tgf9L3JJW2x2sHkzDQGmaF8HXtCkgpxJ ADM/AaJEeP5lYrnEjP3RN/yjGP8hv3wD15paCieAP7giwjIa6Vl6bmA9/FJxA21q3sS0 OMeuJta3RP4rOPPDAOd2gS8oymAsL6pgIMEMhda2twc8q9yqsSeahZ4tLTxwF9uPnCfl nAMRWKaeMcNRWrGielKRreMp4SwgjNE6Otnzb6Hd3AEdjIhxNWGykwYcF+AVT4l+1nOy IZXFgOxzWZHfwXi3UCcbuw5R6En6RlPQS28WzaNVZzu9okea7m8oFotaDEAbolhCNB9w ADeg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=LJiYTWbH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id jz24-20020a17090775f800b008d518d4bf7dsi11915689ejc.562.2023.03.21.08.03.32; Tue, 21 Mar 2023 08:03:33 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=LJiYTWbH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6474668C6B9; Tue, 21 Mar 2023 17:01:54 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2310668C602 for ; Tue, 21 Mar 2023 17:01:44 +0200 (EET) Received: by mail-ed1-f52.google.com with SMTP id h8so60853217ede.8 for ; Tue, 21 Mar 2023 08:01:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; t=1679410903; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=0HJq5iB0ZB0I5vphzcCwOYbRZPKL6ncw4bzpvtjDgdU=; b=LJiYTWbHHi7XHQP9s+MG5EEuy4IJRFVQGoWAuz9rKYgnEQMGUsnKyvhOaAZS5mWcBK NAzXWmrs+hIEJUldzyBATBc8pMX1IwFBXnPMYJX9AWAYGdeslG9tes8b7uE95QoMCxNS PwCG/QQlpO3UIKwOw2onQ+F3kUw/O0vRHVI8vQtU0owVDQ4gpCmQiuDLx4Pc5LSpbuJC ZlQpQYuNVwkF89pRaLM1pNzVU8wOu56oewiKarj9wwr7T1K1oURcwu2I1+e7fNu8VdzU Ai+mTk0VANTKeyOjcYmxnsBSiQv+nGl821QV6l3CuabK9xt1UUuBA07KxGpiOyZ/g2Ev EEvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679410903; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0HJq5iB0ZB0I5vphzcCwOYbRZPKL6ncw4bzpvtjDgdU=; b=Ns4+YigcIF1GH98qXsjkE9x/glekYFqLCmmrlH8Tvc72ZCZHMF6bkBL0GSfL/oxwm9 b/HjXowXXcR7pL6v3Mg9eXJH/8jA97zKNHh8SSEtRT0l5vPfjwAVcvWKOgvp/Mwtvyfd rVinLd3gFWCfzCmifEtM+JX1e9VXIIX5JMfN0lFte9r6m476T21cDzM2ZMj7G6o+qd0B uL5ns8LO3Pj1OB3tERJfBwq5WKVJetKY72dJCmxUe+vv1k9X+x997mJ8uGtj709a04j2 kxuJpJuEFYpuj7592zgZcnifhAOiY1CTylTQwCj1omfnzVEx+rWco8cooF3Gx3+rsVKl /Z/g== X-Gm-Message-State: AO0yUKWtSm4qCn6b+gFi9XiAUob5WQBpvM3dHzya4+ZIAUPDlmO3qa+r EpqOwgTzj2DbsWG5fiz5MPHAxdHjAAOrCD/xBOc= X-Received: by 2002:aa7:ce0c:0:b0:501:d3a1:9a3a with SMTP id d12-20020aa7ce0c000000b00501d3a19a3amr3217127edv.25.1679410902921; Tue, 21 Mar 2023 08:01:42 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id x93-20020a50bae6000000b004fbf6b35a56sm6405817ede.76.2023.03.21.08.01.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Mar 2023 08:01:42 -0700 (PDT) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Tue, 21 Mar 2023 16:01:20 +0100 Message-Id: <20230321150124.21999-8-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230321150124.21999-1-thomas.ff@spin-digital.com> References: <20230321150124.21999-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v7 07/11] avformat: add muxer support for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: +P2Cgt6zHQH+ Add muxer for vvcc byte stream format. Add AV_CODEC_ID_VVC to ff_mp4_obj_type. Add AV_CODEC_ID_VVC to ISO Media codec (VvcConfigurationBox vvi1, vvc1 defined in ISO/IEC 14496-15:2021). Add VvcConfigurationBox vvcC which extends FullBox type in ISO/IEC 14496-15:2021. Add ff_vvc_muxer to RAW muxers. --- libavformat/Makefile | 7 ++++--- libavformat/allformats.c | 1 + libavformat/isom.c | 1 + libavformat/isom_tags.c | 3 +++ libavformat/mov.c | 6 ++++++ libavformat/movenc.c | 41 +++++++++++++++++++++++++++++++++++++++- libavformat/rawenc.c | 23 ++++++++++++++++++++++ 7 files changed, 78 insertions(+), 4 deletions(-) diff --git a/libavformat/Makefile b/libavformat/Makefile index 9d8efe1936..15a013f3fc 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -246,6 +246,7 @@ OBJS-$(CONFIG_H263_MUXER) += rawenc.o OBJS-$(CONFIG_H264_DEMUXER) += h264dec.o rawdec.o OBJS-$(CONFIG_H264_MUXER) += rawenc.o OBJS-$(CONFIG_H266_DEMUXER) += h266dec.o rawdec.o +OBJS-$(CONFIG_H266_MUXER) += rawenc.o OBJS-$(CONFIG_HASH_MUXER) += hashenc.o OBJS-$(CONFIG_HCA_DEMUXER) += hca.o OBJS-$(CONFIG_HCOM_DEMUXER) += hcom.o pcm.o @@ -337,7 +338,7 @@ OBJS-$(CONFIG_MATROSKA_DEMUXER) += matroskadec.o matroska.o \ oggparsevorbis.o vorbiscomment.o \ qtpalette.o replaygain.o dovi_isom.o OBJS-$(CONFIG_MATROSKA_MUXER) += matroskaenc.o matroska.o \ - av1.o avc.o hevc.o \ + av1.o avc.o hevc.o h266.o\ flacenc_header.o avlanguage.o \ vorbiscomment.o wv.o dovi_isom.o OBJS-$(CONFIG_MCA_DEMUXER) += mca.o @@ -359,7 +360,7 @@ OBJS-$(CONFIG_MODS_DEMUXER) += mods.o OBJS-$(CONFIG_MOFLEX_DEMUXER) += moflex.o OBJS-$(CONFIG_MOV_DEMUXER) += mov.o mov_chan.o mov_esds.o \ qtpalette.o replaygain.o dovi_isom.o -OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o vpcc.o \ +OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o h266.o vpcc.o \ movenchint.o mov_chan.o rtp.o \ movenccenc.o movenc_ttml.o rawutils.o \ dovi_isom.o @@ -510,7 +511,7 @@ OBJS-$(CONFIG_RTP_MUXER) += rtp.o \ rtpenc_vp8.o \ rtpenc_vp9.o \ rtpenc_xiph.o \ - avc.o hevc.o + avc.o hevc.o h266.o OBJS-$(CONFIG_RTSP_DEMUXER) += rtsp.o rtspdec.o httpauth.o \ urldecode.o OBJS-$(CONFIG_RTSP_MUXER) += rtsp.o rtspenc.o httpauth.o \ diff --git a/libavformat/allformats.c b/libavformat/allformats.c index 855849449d..2ab0061139 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -200,6 +200,7 @@ extern const FFOutputFormat ff_h263_muxer; extern const AVInputFormat ff_h264_demuxer; extern const FFOutputFormat ff_h264_muxer; extern const AVInputFormat ff_h266_demuxer; +extern const FFOutputFormat ff_h266_muxer; extern const FFOutputFormat ff_hash_muxer; extern const AVInputFormat ff_hca_demuxer; extern const AVInputFormat ff_hcom_demuxer; diff --git a/libavformat/isom.c b/libavformat/isom.c index 6d019881e5..9fbccd4437 100644 --- a/libavformat/isom.c +++ b/libavformat/isom.c @@ -36,6 +36,7 @@ const AVCodecTag ff_mp4_obj_type[] = { { AV_CODEC_ID_MPEG4 , 0x20 }, { AV_CODEC_ID_H264 , 0x21 }, { AV_CODEC_ID_HEVC , 0x23 }, + { AV_CODEC_ID_VVC , 0x33 }, { AV_CODEC_ID_AAC , 0x40 }, { AV_CODEC_ID_MP4ALS , 0x40 }, /* 14496-3 ALS */ { AV_CODEC_ID_MPEG2VIDEO , 0x61 }, /* MPEG-2 Main */ diff --git a/libavformat/isom_tags.c b/libavformat/isom_tags.c index 86c7272525..5da9d03b90 100644 --- a/libavformat/isom_tags.c +++ b/libavformat/isom_tags.c @@ -123,6 +123,9 @@ const AVCodecTag ff_codec_movvideo_tags[] = { { AV_CODEC_ID_HEVC, MKTAG('d', 'v', 'h', 'e') }, /* HEVC-based Dolby Vision derived from hev1 */ /* dvh1 is handled within mov.c */ + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'i', '1') }, /* VVC/H.266 which indicates parameter sets may be in ES */ + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'c', '1') }, /* VVC/H.266 which indicates parameter shall not be in ES */ + { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '1') }, /* AVC-1/H.264 */ { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '2') }, { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '3') }, diff --git a/libavformat/mov.c b/libavformat/mov.c index 057fd872b1..5567593771 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -2107,6 +2107,11 @@ static int mov_read_glbl(MOVContext *c, AVIOContext *pb, MOVAtom atom) if ((uint64_t)atom.size > (1<<30)) return AVERROR_INVALIDDATA; + if (atom.type == MKTAG('v','v','c','C')) { + avio_rb32(pb); + atom.size -= 4; + } + if (atom.size >= 10) { // Broken files created by legacy versions of libavformat will // wrap a whole fiel atom inside of a glbl atom. @@ -7905,6 +7910,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('s','g','p','d'), mov_read_sgpd }, { MKTAG('s','b','g','p'), mov_read_sbgp }, { MKTAG('h','v','c','C'), mov_read_glbl }, +{ MKTAG('v','v','c','C'), mov_read_glbl }, { MKTAG('u','u','i','d'), mov_read_uuid }, { MKTAG('C','i','n', 0x8e), mov_read_targa_y216 }, { MKTAG('f','r','e','e'), mov_read_free }, diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 68e7f8222b..4844d54990 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -58,6 +58,7 @@ #include "libavutil/timecode.h" #include "libavutil/dovi_meta.h" #include "libavutil/uuid.h" +#include "h266.h" #include "hevc.h" #include "rtpenc.h" #include "mov_chan.h" @@ -1455,6 +1456,23 @@ static int mov_write_hvcc_tag(AVIOContext *pb, MOVTrack *track) return update_size(pb, pos); } +static int mov_write_vvcc_tag(AVIOContext *pb, MOVTrack *track) +{ + int64_t pos = avio_tell(pb); + + avio_wb32(pb, 0); + ffio_wfourcc(pb, "vvcC"); + + avio_w8 (pb, 0); /* version */ + avio_wb24(pb, 0); /* flags */ + + if (track->tag == MKTAG('v','v','c','1')) + ff_isom_write_vvcc(pb, track->vos_data, track->vos_len, 1); + else + ff_isom_write_vvcc(pb, track->vos_data, track->vos_len, 0); + return update_size(pb, pos); +} + /* also used by all avid codecs (dv, imx, meridien) and their variants */ static int mov_write_avid_tag(AVIOContext *pb, MOVTrack *track) { @@ -2348,6 +2366,8 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex avid = 1; } else if (track->par->codec_id == AV_CODEC_ID_HEVC) mov_write_hvcc_tag(pb, track); + else if (track->par->codec_id == AV_CODEC_ID_VVC) + mov_write_vvcc_tag(pb, track); else if (track->par->codec_id == AV_CODEC_ID_H264 && !TAG_IS_AVCI(track->tag)) { mov_write_avcc_tag(pb, track); if (track->mode == MODE_IPOD) @@ -6122,6 +6142,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) if ((par->codec_id == AV_CODEC_ID_DNXHD || par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_HEVC || + par->codec_id == AV_CODEC_ID_VVC || par->codec_id == AV_CODEC_ID_VP9 || par->codec_id == AV_CODEC_ID_TRUEHD) && !trk->vos_len && !TAG_IS_AVCI(trk->tag)) { @@ -6186,6 +6207,18 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) size = ff_hevc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); } } + } else if (par->codec_id == AV_CODEC_ID_VVC && trk->vos_len > 6 && + (AV_RB24(trk->vos_data) == 1 || AV_RB32(trk->vos_data) == 1)) { + /* extradata is Annex B, assume the bitstream is too and convert it */ + if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { + ret = ff_h266_annexb2mp4_buf(pkt->data, &reformatted_data, + &size, 0, NULL); + if (ret < 0) + return ret; + avio_write(pb, reformatted_data, size); + } else { + size = ff_h266_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); + } } else if (par->codec_id == AV_CODEC_ID_AV1) { if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { ret = ff_av1_filter_obus_buf(pkt->data, &reformatted_data, @@ -6232,6 +6265,9 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) } else if(par->codec_id == AV_CODEC_ID_HEVC && par->extradata_size > 21) { int nal_size_length = (par->extradata[21] & 0x3) + 1; ret = ff_mov_cenc_avc_write_nal_units(s, &trk->cenc, nal_size_length, pb, pkt->data, size); + } else if(par->codec_id == AV_CODEC_ID_VVC && par->extradata_size > 21) { + int nal_size_length = (par->extradata[21] & 0x3) + 1; + ret = ff_mov_cenc_avc_write_nal_units(s, &trk->cenc, nal_size_length, pb, pkt->data, size); } else { ret = ff_mov_cenc_write_packet(&trk->cenc, pb, pkt->data, size); } @@ -7314,7 +7350,8 @@ static int mov_init(AVFormatContext *s) if (mov->encryption_scheme == MOV_ENC_CENC_AES_CTR) { ret = ff_mov_cenc_init(&track->cenc, mov->encryption_key, - (track->par->codec_id == AV_CODEC_ID_H264 || track->par->codec_id == AV_CODEC_ID_HEVC), + (track->par->codec_id == AV_CODEC_ID_H264 || track->par->codec_id == AV_CODEC_ID_HEVC || + track->par->codec_id == AV_CODEC_ID_VVC), s->flags & AVFMT_FLAG_BITEXACT); if (ret) return ret; @@ -7782,6 +7819,8 @@ static const AVCodecTag codec_mp4_tags[] = { { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '3') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'e', 'v', '1') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'v', 'c', '1') }, + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'c', '1') }, + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'i', '1') }, { AV_CODEC_ID_MPEG2VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MPEG1VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MJPEG, MKTAG('m', 'p', '4', 'v') }, diff --git a/libavformat/rawenc.c b/libavformat/rawenc.c index 78fadda967..c5176d9a5e 100644 --- a/libavformat/rawenc.c +++ b/libavformat/rawenc.c @@ -378,6 +378,29 @@ const FFOutputFormat ff_h264_muxer = { }; #endif +#if CONFIG_H266_MUXER +static int h266_check_bitstream(AVFormatContext *s, AVStream *st, + const AVPacket *pkt) +{ + if (pkt->size >= 5 && AV_RB32(pkt->data) != 0x0000001 && + AV_RB24(pkt->data) != 0x000001) + return ff_stream_add_bitstream_filter(st, "h266_mp4toannexb", NULL); + return 1; +} + +const FFOutputFormat ff_h266_muxer = { + .p.name = "vvc", + .p.long_name = NULL_IF_CONFIG_SMALL("raw H.266/VVC video"), + .p.extensions = "vvc,h266,266", + .p.audio_codec = AV_CODEC_ID_NONE, + .p.video_codec = AV_CODEC_ID_VVC, + .init = force_one_stream, + .write_packet = ff_raw_write_packet, + .check_bitstream = h266_check_bitstream, + .p.flags = AVFMT_NOTIMESTAMPS, +}; +#endif + #if CONFIG_HEVC_MUXER static int hevc_check_bitstream(AVFormatContext *s, AVStream *st, const AVPacket *pkt)