From patchwork Mon Oct 24 14:06:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 38964 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2249585pzb; Mon, 24 Oct 2022 07:09:06 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5ETfR69BkysL1CF2jQRRu/jBghmRigQawLYWRK6HiuU9TbbTKmpV9NWVgLPovzm+oF4dqd X-Received: by 2002:a17:906:8a4a:b0:78d:5ff6:7507 with SMTP id gx10-20020a1709068a4a00b0078d5ff67507mr27580893ejc.194.1666620545919; Mon, 24 Oct 2022 07:09:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666620545; cv=none; d=google.com; s=arc-20160816; b=V70ASMTs7DAO32Bn71k7+7rzMxz54xTOtZxjkLiBEcZ+inqrDPo00ULXyNOtpqOooA 51rNvWtgo3NccJqIYQY2BKcEvqysMfPJdm0PPG0gMu2WEKJcZe7S8VvOwfgJaWuCA3AY v6scUPO1kFP4T1TfXxyzAyXo0OZRL+50sXITeT18WnWdLRFssnBTkVxGsduH5RY1PNoC i2zs3ZK/5k4D9+j2M/kaJuqNK3O/ZMX+KeqVTiNA0U26q5iEDJIh5vms3XwDAVtWfJiy xHJbOyrZdYBGDScRPhvKkMvCApcqxCsoTImlW1HFFu7OUtgxYAfax2g6CWqjsn0Ch9B3 LtiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=I+dezETIAYo2iKi9FmD5XWY/sL20nGM0KElKeEQh228=; b=u+SneJqkNXuxnUU7dC9FsdnmO45Yb26ZX0Tc4hpRzaRlhz6veLVOn4i84uESQphocj jCTTlYlTPIJhdpzofNEam7MZZ27GYfIGfvAEE5nI/CcD57or28ad1nZS/qqvPRUQcqTm 5G/lQAA0yVNVb1jcC0NvXCd1n/EyEye0DHPmkYt7kqj1zmzFDDc3eR6NzcZbidkmvfV6 PBUmmFRNKFsazUIHzpRVp7pm2qKsRsXAfmQGw13LQR0GSSnKfkjJW/5kUt4hjfEWV9sC 0IizpJmQHAFJLOwDGrozDaD1Y810ny7Z2dRTNJRVUc+evUfIhsQrZlikkLWXXkjrnNll ZkZw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=Ky3PpbXW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sh40-20020a1709076ea800b0078d288c1047si27864143ejc.841.2022.10.24.07.09.05; Mon, 24 Oct 2022 07:09:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=Ky3PpbXW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7B1CA68BBFE; Mon, 24 Oct 2022 17:07:51 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DCD1B68BC17 for ; Mon, 24 Oct 2022 17:07:41 +0300 (EEST) Received: by mail-ed1-f45.google.com with SMTP id b12so31767697edd.6 for ; Mon, 24 Oct 2022 07:07:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QX78N43I566M9tXtnMI1jkReABMi3UXxhaW8slDxvAc=; b=Ky3PpbXW2H2iT5tuI17CIxhkX50QaNMhHL+xIEKnJJTZh39oYvO6MtbSOD2j8o2L4x BdaONEUiMLIxAu/uqyj+WW8b5fmxHWFbQWFmcOd7WPsrI/Hq820+XYqKoAw5gtwKOqoU dQANTGxzn1MiVhUJ/CkypbZQqx+tqlazNCf0Vm+zq4j+6UxACmQt8Dtv7A3/POZ+SW8E 9sTEyr1HNdLLroW4147Tl6bZVOGyWAuj3V91+sfvtA7UAA7ntitUWj2XHn4SCzolCgFK m16A29FpB8PSwdJX/Bn0lTO9M0Uup5w/KsJWGK/cJBdqMvZCO+AmnUcqZKDBggJVcpMq EFsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QX78N43I566M9tXtnMI1jkReABMi3UXxhaW8slDxvAc=; b=maJlKQK1ogK2cFtYrVW616CTENOib69/0dVDARs/EjyhKTQnAa3VrPPGXELUg/ngT3 KmU1jAzryzi9KiFS0fdNXvu6BL36N5zB1DruUTcDyJM9clgbbttz6LR+ou+zlOT9/iMy OC0gSGjuRfG6Y0nS0fJs7YAH0SfEvJUt5ykq2aiLcJdKCqoSiqqYzEKMUy498JUD4OJE NMe7M43JRS0nNXQLYIlcBVVfflDRUygtpXKRT4CHld/IxUGJcPWWO2yip02Wgx8XbjzJ jIc8mm7j8UhS+zKI5rEgrhHiXnDH+qOhevYbxZDrYkjwNMWoIXqARLdDnV5/RpZiLWb8 NswA== X-Gm-Message-State: ACrzQf0Wd7FB8hUw9ZJ8KcuSuBq20Z4wIAHXVrm0EVmPwzo4yBb0y9MM Cd4RbB++St9jSiBFkFwxtLz1am3lvN8rsQ== X-Received: by 2002:a05:6402:448c:b0:457:52eb:b57e with SMTP id er12-20020a056402448c00b0045752ebb57emr31449383edb.178.1666620460916; Mon, 24 Oct 2022 07:07:40 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id s19-20020a170906455300b0079800b81709sm6056436ejq.219.2022.10.24.07.07.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 07:07:40 -0700 (PDT) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 16:06:41 +0200 Message-Id: <20221024140645.4945-7-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024140645.4945-1-thomas.ff@spin-digital.com> References: <20221024140645.4945-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 06/10] avformat: add muxer support for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: cR4bYcys2oZC Add muxer for vvcc byte stream format. Add AV_CODEC_ID_VVC to ff_mp4_obj_type Add AV_CODEC_ID_VVC to ISO Media codec (VvcConfigurationBox vvi1, vvc1 defined in ISO/IEC 14496-15:2021) Add VvcConfigurationBox vvcC which extends FullBox type in ISO/IEC 14496-15:2021 Add ff_vvc_muxer to RAW muxers Signed-off-by: Thomas Siedel --- libavformat/Makefile | 7 ++++--- libavformat/allformats.c | 1 + libavformat/isom.c | 1 + libavformat/isom_tags.c | 3 +++ libavformat/mov.c | 6 ++++++ libavformat/movenc.c | 41 +++++++++++++++++++++++++++++++++++++++- libavformat/rawenc.c | 23 ++++++++++++++++++++++ 7 files changed, 78 insertions(+), 4 deletions(-) diff --git a/libavformat/Makefile b/libavformat/Makefile index 00ab4ded89..9ee2526eef 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -336,7 +336,7 @@ OBJS-$(CONFIG_MATROSKA_DEMUXER) += matroskadec.o matroska.o \ oggparsevorbis.o vorbiscomment.o \ qtpalette.o replaygain.o dovi_isom.o OBJS-$(CONFIG_MATROSKA_MUXER) += matroskaenc.o matroska.o \ - av1.o avc.o hevc.o \ + av1.o avc.o hevc.o vvc.o\ flacenc_header.o avlanguage.o \ vorbiscomment.o wv.o dovi_isom.o OBJS-$(CONFIG_MCA_DEMUXER) += mca.o @@ -358,7 +358,7 @@ OBJS-$(CONFIG_MODS_DEMUXER) += mods.o OBJS-$(CONFIG_MOFLEX_DEMUXER) += moflex.o OBJS-$(CONFIG_MOV_DEMUXER) += mov.o mov_chan.o mov_esds.o \ qtpalette.o replaygain.o dovi_isom.o -OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o vpcc.o \ +OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o vvc.o vpcc.o \ movenchint.o mov_chan.o rtp.o \ movenccenc.o movenc_ttml.o rawutils.o \ dovi_isom.o @@ -508,7 +508,7 @@ OBJS-$(CONFIG_RTP_MUXER) += rtp.o \ rtpenc_vp8.o \ rtpenc_vp9.o \ rtpenc_xiph.o \ - avc.o hevc.o + avc.o hevc.o vvc.o OBJS-$(CONFIG_RTSP_DEMUXER) += rtsp.o rtspdec.o httpauth.o \ urldecode.o OBJS-$(CONFIG_RTSP_MUXER) += rtsp.o rtspenc.o httpauth.o \ @@ -596,6 +596,7 @@ OBJS-$(CONFIG_VPK_DEMUXER) += vpk.o OBJS-$(CONFIG_VPLAYER_DEMUXER) += vplayerdec.o subtitles.o OBJS-$(CONFIG_VQF_DEMUXER) += vqf.o OBJS-$(CONFIG_VVC_DEMUXER) += vvcdec.o rawdec.o +OBJS-$(CONFIG_VVC_MUXER) += rawenc.o OBJS-$(CONFIG_W64_DEMUXER) += wavdec.o w64.o pcm.o OBJS-$(CONFIG_W64_MUXER) += wavenc.o w64.o OBJS-$(CONFIG_WAV_DEMUXER) += wavdec.o pcm.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index a4e3822681..006a7dc125 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -475,6 +475,7 @@ extern const AVInputFormat ff_vpk_demuxer; extern const AVInputFormat ff_vplayer_demuxer; extern const AVInputFormat ff_vqf_demuxer; extern const AVInputFormat ff_vvc_demuxer; +extern const AVOutputFormat ff_vvc_muxer; extern const AVInputFormat ff_w64_demuxer; extern const AVOutputFormat ff_w64_muxer; extern const AVInputFormat ff_wav_demuxer; diff --git a/libavformat/isom.c b/libavformat/isom.c index 6d019881e5..9fbccd4437 100644 --- a/libavformat/isom.c +++ b/libavformat/isom.c @@ -36,6 +36,7 @@ const AVCodecTag ff_mp4_obj_type[] = { { AV_CODEC_ID_MPEG4 , 0x20 }, { AV_CODEC_ID_H264 , 0x21 }, { AV_CODEC_ID_HEVC , 0x23 }, + { AV_CODEC_ID_VVC , 0x33 }, { AV_CODEC_ID_AAC , 0x40 }, { AV_CODEC_ID_MP4ALS , 0x40 }, /* 14496-3 ALS */ { AV_CODEC_ID_MPEG2VIDEO , 0x61 }, /* MPEG-2 Main */ diff --git a/libavformat/isom_tags.c b/libavformat/isom_tags.c index e2b80405cc..ec93bdc363 100644 --- a/libavformat/isom_tags.c +++ b/libavformat/isom_tags.c @@ -123,6 +123,9 @@ const AVCodecTag ff_codec_movvideo_tags[] = { { AV_CODEC_ID_HEVC, MKTAG('d', 'v', 'h', 'e') }, /* HEVC-based Dolby Vision derived from hev1 */ /* dvh1 is handled within mov.c */ + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'i', '1') }, /* VVC/H.266 which indicates parameter sets may be in ES */ + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'c', '1') }, /* VVC/H.266 which indicates parameter shall not be in ES */ + { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '1') }, /* AVC-1/H.264 */ { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '2') }, { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '3') }, diff --git a/libavformat/mov.c b/libavformat/mov.c index 1f436e21d6..52719c9d9a 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -1962,6 +1962,11 @@ static int mov_read_glbl(MOVContext *c, AVIOContext *pb, MOVAtom atom) if ((uint64_t)atom.size > (1<<30)) return AVERROR_INVALIDDATA; + if (atom.type == MKTAG('v','v','c','C')) { + avio_rb32(pb); + atom.size -= 4; + } + if (atom.size >= 10) { // Broken files created by legacy versions of libavformat will // wrap a whole fiel atom inside of a glbl atom. @@ -7730,6 +7735,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('s','g','p','d'), mov_read_sgpd }, { MKTAG('s','b','g','p'), mov_read_sbgp }, { MKTAG('h','v','c','C'), mov_read_glbl }, +{ MKTAG('v','v','c','C'), mov_read_glbl }, { MKTAG('u','u','i','d'), mov_read_uuid }, { MKTAG('C','i','n', 0x8e), mov_read_targa_y216 }, { MKTAG('f','r','e','e'), mov_read_free }, diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 754f95912a..3d412d0079 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -59,6 +59,7 @@ #include "libavutil/color_utils.h" #include "libavutil/uuid.h" #include "hevc.h" +#include "vvc.h" #include "rtpenc.h" #include "mov_chan.h" #include "movenc_ttml.h" @@ -1392,6 +1393,23 @@ static int mov_write_hvcc_tag(AVIOContext *pb, MOVTrack *track) return update_size(pb, pos); } +static int mov_write_vvcc_tag(AVIOContext *pb, MOVTrack *track) +{ + int64_t pos = avio_tell(pb); + + avio_wb32(pb, 0); + ffio_wfourcc(pb, "vvcC"); + + avio_w8 (pb, 0); /* version */ + avio_wb24(pb, 0); /* flags */ + + if (track->tag == MKTAG('v','v','i','1')) + ff_isom_write_vvcc(pb, track->vos_data, track->vos_len, 1); + else + ff_isom_write_vvcc(pb, track->vos_data, track->vos_len, 0); + return update_size(pb, pos); +} + /* also used by all avid codecs (dv, imx, meridien) and their variants */ static int mov_write_avid_tag(AVIOContext *pb, MOVTrack *track) { @@ -2286,6 +2304,8 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex avid = 1; } else if (track->par->codec_id == AV_CODEC_ID_HEVC) mov_write_hvcc_tag(pb, track); + else if (track->par->codec_id == AV_CODEC_ID_VVC) + mov_write_vvcc_tag(pb, track); else if (track->par->codec_id == AV_CODEC_ID_H264 && !TAG_IS_AVCI(track->tag)) { mov_write_avcc_tag(pb, track); if (track->mode == MODE_IPOD) @@ -6033,6 +6053,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) if ((par->codec_id == AV_CODEC_ID_DNXHD || par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_HEVC || + par->codec_id == AV_CODEC_ID_VVC || par->codec_id == AV_CODEC_ID_TRUEHD) && !trk->vos_len && !TAG_IS_AVCI(trk->tag)) { /* copy frame to create needed atoms */ @@ -6096,6 +6117,18 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) size = ff_hevc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); } } + } else if (par->codec_id == AV_CODEC_ID_VVC && trk->vos_len > 6 && + (AV_RB24(trk->vos_data) == 1 || AV_RB32(trk->vos_data) == 1)) { + /* extradata is Annex B, assume the bitstream is too and convert it */ + if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { + ret = ff_vvc_annexb2mp4_buf(pkt->data, &reformatted_data, + &size, 0, NULL); + if (ret < 0) + return ret; + avio_write(pb, reformatted_data, size); + } else { + size = ff_vvc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); + } } else if (par->codec_id == AV_CODEC_ID_AV1) { if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { ret = ff_av1_filter_obus_buf(pkt->data, &reformatted_data, @@ -6142,6 +6175,9 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) } else if(par->codec_id == AV_CODEC_ID_HEVC && par->extradata_size > 21) { int nal_size_length = (par->extradata[21] & 0x3) + 1; ret = ff_mov_cenc_avc_write_nal_units(s, &trk->cenc, nal_size_length, pb, pkt->data, size); + } else if(par->codec_id == AV_CODEC_ID_VVC && par->extradata_size > 21) { + int nal_size_length = (par->extradata[21] & 0x3) + 1; + ret = ff_mov_cenc_avc_write_nal_units(s, &trk->cenc, nal_size_length, pb, pkt->data, size); } else { ret = ff_mov_cenc_write_packet(&trk->cenc, pb, pkt->data, size); } @@ -7224,7 +7260,8 @@ static int mov_init(AVFormatContext *s) if (mov->encryption_scheme == MOV_ENC_CENC_AES_CTR) { ret = ff_mov_cenc_init(&track->cenc, mov->encryption_key, - (track->par->codec_id == AV_CODEC_ID_H264 || track->par->codec_id == AV_CODEC_ID_HEVC), + (track->par->codec_id == AV_CODEC_ID_H264 || track->par->codec_id == AV_CODEC_ID_HEVC || + track->par->codec_id == AV_CODEC_ID_VVC), s->flags & AVFMT_FLAG_BITEXACT); if (ret) return ret; @@ -7692,6 +7729,8 @@ static const AVCodecTag codec_mp4_tags[] = { { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '3') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'e', 'v', '1') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'v', 'c', '1') }, + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'c', '1') }, + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'i', '1') }, { AV_CODEC_ID_MPEG2VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MPEG1VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MJPEG, MKTAG('m', 'p', '4', 'v') }, diff --git a/libavformat/rawenc.c b/libavformat/rawenc.c index 267fce252d..7770d63d10 100644 --- a/libavformat/rawenc.c +++ b/libavformat/rawenc.c @@ -401,6 +401,29 @@ const AVOutputFormat ff_hevc_muxer = { }; #endif +#if CONFIG_VVC_MUXER +static int vvc_check_bitstream(AVFormatContext *s, AVStream *st, + const AVPacket *pkt) +{ + if (pkt->size >= 5 && AV_RB32(pkt->data) != 0x0000001 && + AV_RB24(pkt->data) != 0x000001) + return ff_stream_add_bitstream_filter(st, "vvc_mp4toannexb", NULL); + return 1; +} + +const AVOutputFormat ff_vvc_muxer = { + .name = "vvc", + .long_name = NULL_IF_CONFIG_SMALL("raw VVC video"), + .extensions = "vvc,h266,266", + .audio_codec = AV_CODEC_ID_NONE, + .video_codec = AV_CODEC_ID_VVC, + .init = force_one_stream, + .write_packet = ff_raw_write_packet, + .check_bitstream = vvc_check_bitstream, + .flags = AVFMT_NOTIMESTAMPS, +}; +#endif + #if CONFIG_M4V_MUXER const AVOutputFormat ff_m4v_muxer = { .name = "m4v",