From patchwork Tue Jan 3 13:40:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 39857 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:bc95:b0:ad:ade2:bfd2 with SMTP id fx21csp8076701pzb; Tue, 3 Jan 2023 05:42:57 -0800 (PST) X-Google-Smtp-Source: AMrXdXskuBabYAxtsoLqM17pbsCIMRGPg9v+vUfaiUoauTkwniVcQGPFZuF9tuUjE0MnjeYZ29/D X-Received: by 2002:a17:907:8b09:b0:7c1:6e82:35fc with SMTP id sz9-20020a1709078b0900b007c16e8235fcmr51128467ejc.40.1672753377194; Tue, 03 Jan 2023 05:42:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672753377; cv=none; d=google.com; s=arc-20160816; b=ngcf8CIiZl0dA1+NSEdsT07Z5/40epFsVJYzFadj/tB2cQhWABt2my3n6YcwMRIgnc Jy4Bw+nSs11kglf34EMwVVLci5T23IqOLxnGue7FKYvdPV5PbiXGgWxGhnhP7uuxdH/+ lUyPFx/WZTasH+VkDazt07kPFNfQtKLkGuhQHKXK/NheUXMrJHFmhH6Qh8tDs+GUf7AO COpAp7IKSiSujw/C6MLqXH/taiz+QSLrlhHiaL9EMp3uAKmXMEo8G7C67+4V9K+9uYTF MY0YXScNcZDu0G0KDy+bD62ysm4ro4uoo/Nzt91ZqCCKyjQQBD5Fe2WSgGEb/eUPpMRN Yv2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=Dd7o72lv+nbFqqMwjSv/ObQC/X69yfo0C5MpDVMhETc=; b=gytjHMoiTlnTBBjyT8Q+1e4/Nw17osFq0/XvDWTp1MSe92WWUxV4jn3NRdFGqkYpYC HW93W/Ue7H5Dk7aSRTEwozm5vbyaJHSUMj7zIGdSJFdfMmq1uxyZX66yJ1mtBhbZoy/W Hp0yDKNn0HMMAxMUQR+B7RVnD0ebmQegm/PBnDkeCVbdIPFVBtPBe1ly7eNsTWexSeGW nRgVtu3hfjaKU8NJfvWXdS7Bh7LijNbPWuOIZ20Kkjf9DaZhtX1opNdzhZQ/FxnsRWj9 L5BGCCskm/cleyxmfTlFtyTrQW1jeWeTfqYjn+kr1UvUmReJv3zgQB2Sk01wDq7pHny4 O69Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=pdGwyWVy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hq28-20020a1709073f1c00b007d5291a692bsi26937632ejc.260.2023.01.03.05.42.56; Tue, 03 Jan 2023 05:42:57 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=pdGwyWVy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8A06F68BDA1; Tue, 3 Jan 2023 15:41:46 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E93B168BD7E for ; Tue, 3 Jan 2023 15:41:37 +0200 (EET) Received: by mail-ej1-f45.google.com with SMTP id kw15so73546188ejc.10 for ; Tue, 03 Jan 2023 05:41:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=ZgyRFlus8OKwMFVBGSvxHXNujqiRYHIgJrm2p2nA9AI=; b=pdGwyWVyJbuFZxGyRgZQYx6fwejdig4oD43sRbYW9QBI2T2x79itdF+5CK5s0/oDP7 5IG7X52s/NvFX4Bn10SKoBdf5mYbLl46B4PNvJMSXM0kUpYvWdZ71Bv7Wgnyx2qAi3HC 6cw7KLlQxjUmxxw8jWqdj8s7Pu+nhtHmxFzfsNEjn9x2Xurmq8NF48CS6b/sYa6aJ5fZ Gxjd29SQegin7gsyh7H62J8L+40MByFB9iOtaGRjkZWKHkmerpsbWufV+qCf1MC/k4Kz q9RtkFMGYC7m6Y0WhbfwBBlx1hreuMnw+d6VUCOZS/6OQBaDk1veLtpb54hGqFp/VOaU FLVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZgyRFlus8OKwMFVBGSvxHXNujqiRYHIgJrm2p2nA9AI=; b=krSSkvMomuE1Su1H4IoXr7hx4a9EYjygZ/Da9NLDPbdv6f45h8tPEVAAW9K4HKfD+L 7ep7Nyj7W45ndDQPEu4Pro+Bu1vEJMKnpCBtOy1P37q+4ok2E0LshwbwHgKMLZYYYlWb 381RwTOrvrRrqXqk2b6RO5zkfpBXqrtO6mraVKFTNKohWsqbSozICPg7N5xOWKIDIDja u8EsAUaNb8HwizHZ9tPAjDmG4HpfQJU9K9aSwedf6+MbtU+4O9t1ZMggdjTj0w14kzJM 9w+PKslJhS5UZpW76cYHBXXd5jiYh+asyj80uY2stXHCZaRNG+q+nly5UrJt9vZsGv0h LXNQ== X-Gm-Message-State: AFqh2kqN2833szzkNS10+r0xcHHGHUDXmysPmxheElyKoKkdZkurmmp8 n4lhjqtysKBb1Tu6skg3E64uS2gN/iYEaEM2 X-Received: by 2002:a17:907:6746:b0:840:758a:9155 with SMTP id qm6-20020a170907674600b00840758a9155mr17750623ejc.12.1672753296810; Tue, 03 Jan 2023 05:41:36 -0800 (PST) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id q26-20020a17090676da00b008302732f569sm14151095ejn.78.2023.01.03.05.41.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Jan 2023 05:41:36 -0800 (PST) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Tue, 3 Jan 2023 14:40:36 +0100 Message-Id: <20230103134040.41140-7-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230103134040.41140-1-thomas.ff@spin-digital.com> References: <20230103134040.41140-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v5 06/10] avformat: add muxer support for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: OoQPlUac21zh Add muxer for vvcc byte stream format. Add AV_CODEC_ID_VVC to ff_mp4_obj_type. Add AV_CODEC_ID_VVC to ISO Media codec (VvcConfigurationBox vvi1, vvc1 defined in ISO/IEC 14496-15:2021). Add VvcConfigurationBox vvcC which extends FullBox type in ISO/IEC 14496-15:2021. Add ff_vvc_muxer to RAW muxers. --- libavformat/Makefile | 7 ++++--- libavformat/allformats.c | 1 + libavformat/isom.c | 1 + libavformat/isom_tags.c | 3 +++ libavformat/mov.c | 6 ++++++ libavformat/movenc.c | 41 +++++++++++++++++++++++++++++++++++++++- libavformat/rawenc.c | 23 ++++++++++++++++++++++ 7 files changed, 78 insertions(+), 4 deletions(-) diff --git a/libavformat/Makefile b/libavformat/Makefile index 00ab4ded89..9ee2526eef 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -336,7 +336,7 @@ OBJS-$(CONFIG_MATROSKA_DEMUXER) += matroskadec.o matroska.o \ oggparsevorbis.o vorbiscomment.o \ qtpalette.o replaygain.o dovi_isom.o OBJS-$(CONFIG_MATROSKA_MUXER) += matroskaenc.o matroska.o \ - av1.o avc.o hevc.o \ + av1.o avc.o hevc.o vvc.o\ flacenc_header.o avlanguage.o \ vorbiscomment.o wv.o dovi_isom.o OBJS-$(CONFIG_MCA_DEMUXER) += mca.o @@ -358,7 +358,7 @@ OBJS-$(CONFIG_MODS_DEMUXER) += mods.o OBJS-$(CONFIG_MOFLEX_DEMUXER) += moflex.o OBJS-$(CONFIG_MOV_DEMUXER) += mov.o mov_chan.o mov_esds.o \ qtpalette.o replaygain.o dovi_isom.o -OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o vpcc.o \ +OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o vvc.o vpcc.o \ movenchint.o mov_chan.o rtp.o \ movenccenc.o movenc_ttml.o rawutils.o \ dovi_isom.o @@ -508,7 +508,7 @@ OBJS-$(CONFIG_RTP_MUXER) += rtp.o \ rtpenc_vp8.o \ rtpenc_vp9.o \ rtpenc_xiph.o \ - avc.o hevc.o + avc.o hevc.o vvc.o OBJS-$(CONFIG_RTSP_DEMUXER) += rtsp.o rtspdec.o httpauth.o \ urldecode.o OBJS-$(CONFIG_RTSP_MUXER) += rtsp.o rtspenc.o httpauth.o \ @@ -596,6 +596,7 @@ OBJS-$(CONFIG_VPK_DEMUXER) += vpk.o OBJS-$(CONFIG_VPLAYER_DEMUXER) += vplayerdec.o subtitles.o OBJS-$(CONFIG_VQF_DEMUXER) += vqf.o OBJS-$(CONFIG_VVC_DEMUXER) += vvcdec.o rawdec.o +OBJS-$(CONFIG_VVC_MUXER) += rawenc.o OBJS-$(CONFIG_W64_DEMUXER) += wavdec.o w64.o pcm.o OBJS-$(CONFIG_W64_MUXER) += wavenc.o w64.o OBJS-$(CONFIG_WAV_DEMUXER) += wavdec.o pcm.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index 5fce5b4774..9ca9e9e9eb 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -478,6 +478,7 @@ extern const AVInputFormat ff_vpk_demuxer; extern const AVInputFormat ff_vplayer_demuxer; extern const AVInputFormat ff_vqf_demuxer; extern const AVInputFormat ff_vvc_demuxer; +extern const AVOutputFormat ff_vvc_muxer; extern const AVInputFormat ff_w64_demuxer; extern const AVOutputFormat ff_w64_muxer; extern const AVInputFormat ff_wav_demuxer; diff --git a/libavformat/isom.c b/libavformat/isom.c index 6d019881e5..9fbccd4437 100644 --- a/libavformat/isom.c +++ b/libavformat/isom.c @@ -36,6 +36,7 @@ const AVCodecTag ff_mp4_obj_type[] = { { AV_CODEC_ID_MPEG4 , 0x20 }, { AV_CODEC_ID_H264 , 0x21 }, { AV_CODEC_ID_HEVC , 0x23 }, + { AV_CODEC_ID_VVC , 0x33 }, { AV_CODEC_ID_AAC , 0x40 }, { AV_CODEC_ID_MP4ALS , 0x40 }, /* 14496-3 ALS */ { AV_CODEC_ID_MPEG2VIDEO , 0x61 }, /* MPEG-2 Main */ diff --git a/libavformat/isom_tags.c b/libavformat/isom_tags.c index e2b80405cc..ec93bdc363 100644 --- a/libavformat/isom_tags.c +++ b/libavformat/isom_tags.c @@ -123,6 +123,9 @@ const AVCodecTag ff_codec_movvideo_tags[] = { { AV_CODEC_ID_HEVC, MKTAG('d', 'v', 'h', 'e') }, /* HEVC-based Dolby Vision derived from hev1 */ /* dvh1 is handled within mov.c */ + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'i', '1') }, /* VVC/H.266 which indicates parameter sets may be in ES */ + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'c', '1') }, /* VVC/H.266 which indicates parameter shall not be in ES */ + { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '1') }, /* AVC-1/H.264 */ { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '2') }, { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '3') }, diff --git a/libavformat/mov.c b/libavformat/mov.c index 29bd3103e3..7e8dea1049 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -1967,6 +1967,11 @@ static int mov_read_glbl(MOVContext *c, AVIOContext *pb, MOVAtom atom) if ((uint64_t)atom.size > (1<<30)) return AVERROR_INVALIDDATA; + if (atom.type == MKTAG('v','v','c','C')) { + avio_rb32(pb); + atom.size -= 4; + } + if (atom.size >= 10) { // Broken files created by legacy versions of libavformat will // wrap a whole fiel atom inside of a glbl atom. @@ -7735,6 +7740,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('s','g','p','d'), mov_read_sgpd }, { MKTAG('s','b','g','p'), mov_read_sbgp }, { MKTAG('h','v','c','C'), mov_read_glbl }, +{ MKTAG('v','v','c','C'), mov_read_glbl }, { MKTAG('u','u','i','d'), mov_read_uuid }, { MKTAG('C','i','n', 0x8e), mov_read_targa_y216 }, { MKTAG('f','r','e','e'), mov_read_free }, diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 7d49892283..14fc313584 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -59,6 +59,7 @@ #include "libavutil/color_utils.h" #include "libavutil/uuid.h" #include "hevc.h" +#include "vvc.h" #include "rtpenc.h" #include "mov_chan.h" #include "movenc_ttml.h" @@ -1393,6 +1394,23 @@ static int mov_write_hvcc_tag(AVIOContext *pb, MOVTrack *track) return update_size(pb, pos); } +static int mov_write_vvcc_tag(AVIOContext *pb, MOVTrack *track) +{ + int64_t pos = avio_tell(pb); + + avio_wb32(pb, 0); + ffio_wfourcc(pb, "vvcC"); + + avio_w8 (pb, 0); /* version */ + avio_wb24(pb, 0); /* flags */ + + if (track->tag == MKTAG('v','v','i','1')) + ff_isom_write_vvcc(pb, track->vos_data, track->vos_len, 1); + else + ff_isom_write_vvcc(pb, track->vos_data, track->vos_len, 0); + return update_size(pb, pos); +} + /* also used by all avid codecs (dv, imx, meridien) and their variants */ static int mov_write_avid_tag(AVIOContext *pb, MOVTrack *track) { @@ -2287,6 +2305,8 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex avid = 1; } else if (track->par->codec_id == AV_CODEC_ID_HEVC) mov_write_hvcc_tag(pb, track); + else if (track->par->codec_id == AV_CODEC_ID_VVC) + mov_write_vvcc_tag(pb, track); else if (track->par->codec_id == AV_CODEC_ID_H264 && !TAG_IS_AVCI(track->tag)) { mov_write_avcc_tag(pb, track); if (track->mode == MODE_IPOD) @@ -6035,6 +6055,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) if ((par->codec_id == AV_CODEC_ID_DNXHD || par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_HEVC || + par->codec_id == AV_CODEC_ID_VVC || par->codec_id == AV_CODEC_ID_VP9 || par->codec_id == AV_CODEC_ID_TRUEHD) && !trk->vos_len && !TAG_IS_AVCI(trk->tag)) { @@ -6099,6 +6120,18 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) size = ff_hevc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); } } + } else if (par->codec_id == AV_CODEC_ID_VVC && trk->vos_len > 6 && + (AV_RB24(trk->vos_data) == 1 || AV_RB32(trk->vos_data) == 1)) { + /* extradata is Annex B, assume the bitstream is too and convert it */ + if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { + ret = ff_vvc_annexb2mp4_buf(pkt->data, &reformatted_data, + &size, 0, NULL); + if (ret < 0) + return ret; + avio_write(pb, reformatted_data, size); + } else { + size = ff_vvc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); + } } else if (par->codec_id == AV_CODEC_ID_AV1) { if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { ret = ff_av1_filter_obus_buf(pkt->data, &reformatted_data, @@ -6145,6 +6178,9 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) } else if(par->codec_id == AV_CODEC_ID_HEVC && par->extradata_size > 21) { int nal_size_length = (par->extradata[21] & 0x3) + 1; ret = ff_mov_cenc_avc_write_nal_units(s, &trk->cenc, nal_size_length, pb, pkt->data, size); + } else if(par->codec_id == AV_CODEC_ID_VVC && par->extradata_size > 21) { + int nal_size_length = (par->extradata[21] & 0x3) + 1; + ret = ff_mov_cenc_avc_write_nal_units(s, &trk->cenc, nal_size_length, pb, pkt->data, size); } else { ret = ff_mov_cenc_write_packet(&trk->cenc, pb, pkt->data, size); } @@ -7227,7 +7263,8 @@ static int mov_init(AVFormatContext *s) if (mov->encryption_scheme == MOV_ENC_CENC_AES_CTR) { ret = ff_mov_cenc_init(&track->cenc, mov->encryption_key, - (track->par->codec_id == AV_CODEC_ID_H264 || track->par->codec_id == AV_CODEC_ID_HEVC), + (track->par->codec_id == AV_CODEC_ID_H264 || track->par->codec_id == AV_CODEC_ID_HEVC || + track->par->codec_id == AV_CODEC_ID_VVC), s->flags & AVFMT_FLAG_BITEXACT); if (ret) return ret; @@ -7695,6 +7732,8 @@ static const AVCodecTag codec_mp4_tags[] = { { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '3') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'e', 'v', '1') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'v', 'c', '1') }, + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'c', '1') }, + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'i', '1') }, { AV_CODEC_ID_MPEG2VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MPEG1VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MJPEG, MKTAG('m', 'p', '4', 'v') }, diff --git a/libavformat/rawenc.c b/libavformat/rawenc.c index 267fce252d..7770d63d10 100644 --- a/libavformat/rawenc.c +++ b/libavformat/rawenc.c @@ -401,6 +401,29 @@ const AVOutputFormat ff_hevc_muxer = { }; #endif +#if CONFIG_VVC_MUXER +static int vvc_check_bitstream(AVFormatContext *s, AVStream *st, + const AVPacket *pkt) +{ + if (pkt->size >= 5 && AV_RB32(pkt->data) != 0x0000001 && + AV_RB24(pkt->data) != 0x000001) + return ff_stream_add_bitstream_filter(st, "vvc_mp4toannexb", NULL); + return 1; +} + +const AVOutputFormat ff_vvc_muxer = { + .name = "vvc", + .long_name = NULL_IF_CONFIG_SMALL("raw VVC video"), + .extensions = "vvc,h266,266", + .audio_codec = AV_CODEC_ID_NONE, + .video_codec = AV_CODEC_ID_VVC, + .init = force_one_stream, + .write_packet = ff_raw_write_packet, + .check_bitstream = vvc_check_bitstream, + .flags = AVFMT_NOTIMESTAMPS, +}; +#endif + #if CONFIG_M4V_MUXER const AVOutputFormat ff_m4v_muxer = { .name = "m4v",