From patchwork Fri Feb 10 17:41:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 40355 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5494:b0:bf:7b3a:fd32 with SMTP id i20csp1566000pzk; Fri, 10 Feb 2023 09:42:58 -0800 (PST) X-Google-Smtp-Source: AK7set9hshM+JL+3KsGF/mlGhLcSAp7N0lxFgfIxgwdmVrbTu+EJcmL9PaLjrXYVTo5I+znHh6Y/ X-Received: by 2002:a17:906:1511:b0:88d:d304:3432 with SMTP id b17-20020a170906151100b0088dd3043432mr16295087ejd.60.1676050978557; Fri, 10 Feb 2023 09:42:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676050978; cv=none; d=google.com; s=arc-20160816; b=oXg+otmTV30RbKKLt6m6vNBSl7nVlzR2vewiP68jf82fV0qn/qL/1UD7jx8lm7yIlb HeCOO5CxYIpsWrDg8CxX7BHnt3vPKGJpx7611hYBlfDN3qha611ASCT6OWLxz4PzJjB+ qZs0k71dBeYMNH4uT91C4auzk+58gZkBZAuLn2F2+BBLKsT5ZBJsUk3ZvSt03lrtm5DC hA0BkPP17Fw0gUq2766U2B3twgF7eDdG2/fHLOF5Wb0erLViS1bxBVVg0Vz6PIX00IWr ODbWI5wjeieRp4+pwNbDMQR/X1dZ1jXk96I6PnBh8GCm/g+FrBhg+2ypequnDYEeg1vA K4/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=CID5GGxc0197JA68pkEW/DQSO3hANHRjtBxQzHgisHw=; b=FXl4pJjC5/fBWE8Dlijat6kt8iC5c1lLeT1Ueo5+vXQyAOxmBcxa6XLAxo6Fdv9Pzm 0coYZv5tjgEMRScCyevbRUl7uXN4dY9eGhS8NC2+qh1xxIxgGWuosPx8hLxv4W+Td7eG OSszk72WzGQjv6Gb/oiB0EzOsIG+Q1FpzR57WsNHz+8891oSLbK60GDYUm/lgqRHGHeX OXw9qQ68W3RN1aVi0k8s7I+WutkTYBMNbAfLCwxuIZvCMZV1diryMXBY9h+Ah/F/W9Jd 8rI+tM2EIpxyq/BPoEgd3jIGpsy+IQxrSJA20zrkSHn+GjhUF4BT9HsdXFqLuHw7/+ko MoCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=h70bf6dM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id up25-20020a170907cc9900b008aa4586c520si5732710ejc.753.2023.02.10.09.42.58; Fri, 10 Feb 2023 09:42:58 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=h70bf6dM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E0D4568BE86; Fri, 10 Feb 2023 19:41:53 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3125468BDC9 for ; Fri, 10 Feb 2023 19:41:44 +0200 (EET) Received: by mail-wr1-f45.google.com with SMTP id y1so5813886wru.2 for ; Fri, 10 Feb 2023 09:41:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=mU0H6cHHVeCTR7h1qj9QEuRUXAdlMAj4uLMzh3VnDF0=; b=h70bf6dMSMqzcR3EmRNOj/KW6EyXAjJ3OwIsA2C/2JbdTCQ5A/sbXE6efUuT/MT5YR lszgqHaRqBNUusFOU86mQIsN+kMQua+1SoaAmOPujhvUbIEKxVhYAD9OQd+8R5oLsNlS BGqTs9x4p5j0aUJzzeXaCvPl1TjMD84oZYrt0Nl4tp5NG/AEdcmdo9Gzz1BVUXrIM9Tl zvcxhQngPw3a96tYRJFOhQzG82IRvbuoJpGa9MOm6bMD3nY1Ol8wHt+IrSbe2Z9siFUP J1j37BuWOl1qlxuNm5xwcG2OLbfkQ46q5Z+Q6YjCaDiQn6jJfHXTzBapJd9A2KQa4kjj PmAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mU0H6cHHVeCTR7h1qj9QEuRUXAdlMAj4uLMzh3VnDF0=; b=UMzmulkRy3AlZYac9OdISMsrjjV4G4LXxpMKcknZzV1p+W0kNY+5dno8JKsn2EuW6G lN76gWt/HRx424NAKZZSCqkvJl7nI6CQEFj0aq55xRuSmW5QWdgeqZKStBk1BCRI6Kbi 7OFAZMqAS1THKCnE5g/Ug8htVJedT7UCekdE0UvTIeoVUF0qikhwWoxKMvzSpdPedgfL BZlfA5Kxems8aTwdEmREG3lHogjp6U1OYzQdlBLsWZHNh4RABztX+BtvFgrkwhMLOgDR ZX+rJa4mW3Cxw2jMMlax6fzs/trNif26POSByMwfKB4DH3GwHneJebWPJRJHKjHbHTti t6kg== X-Gm-Message-State: AO0yUKXToJhJel4fOv1Up3Sk72aWuGDL7PV8cFjM3GkdQs40SSXrZAxX huzqcF9tFnYPIeFEL1QZRB+imxnMt037VIzg X-Received: by 2002:a05:6000:1c7:b0:2c3:fdad:345 with SMTP id t7-20020a05600001c700b002c3fdad0345mr10421148wrx.70.1676050903163; Fri, 10 Feb 2023 09:41:43 -0800 (PST) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id w13-20020a5d608d000000b002c54b6382c8sm1589245wrt.82.2023.02.10.09.41.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Feb 2023 09:41:42 -0800 (PST) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Fri, 10 Feb 2023 18:41:02 +0100 Message-Id: <20230210174106.44514-8-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230210174106.44514-1-thomas.ff@spin-digital.com> References: <20230210174106.44514-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v6 07/11] avformat: add muxer support for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: MkKjrSt83Ri0 Add muxer for vvcc byte stream format. Add AV_CODEC_ID_VVC to ff_mp4_obj_type. Add AV_CODEC_ID_VVC to ISO Media codec (VvcConfigurationBox vvi1, vvc1 defined in ISO/IEC 14496-15:2021). Add VvcConfigurationBox vvcC which extends FullBox type in ISO/IEC 14496-15:2021. Add ff_vvc_muxer to RAW muxers. --- libavformat/Makefile | 7 ++++--- libavformat/allformats.c | 1 + libavformat/isom.c | 1 + libavformat/isom_tags.c | 3 +++ libavformat/mov.c | 6 ++++++ libavformat/movenc.c | 41 +++++++++++++++++++++++++++++++++++++++- libavformat/rawenc.c | 23 ++++++++++++++++++++++ 7 files changed, 78 insertions(+), 4 deletions(-) diff --git a/libavformat/Makefile b/libavformat/Makefile index 722dbf7ae8..e7b6f1dcb2 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -246,6 +246,7 @@ OBJS-$(CONFIG_H263_MUXER) += rawenc.o OBJS-$(CONFIG_H264_DEMUXER) += h264dec.o rawdec.o OBJS-$(CONFIG_H264_MUXER) += rawenc.o OBJS-$(CONFIG_H266_DEMUXER) += h266dec.o rawdec.o +OBJS-$(CONFIG_H266_MUXER) += rawenc.o OBJS-$(CONFIG_HASH_MUXER) += hashenc.o OBJS-$(CONFIG_HCA_DEMUXER) += hca.o OBJS-$(CONFIG_HCOM_DEMUXER) += hcom.o pcm.o @@ -337,7 +338,7 @@ OBJS-$(CONFIG_MATROSKA_DEMUXER) += matroskadec.o matroska.o \ oggparsevorbis.o vorbiscomment.o \ qtpalette.o replaygain.o dovi_isom.o OBJS-$(CONFIG_MATROSKA_MUXER) += matroskaenc.o matroska.o \ - av1.o avc.o hevc.o \ + av1.o avc.o hevc.o h266.o\ flacenc_header.o avlanguage.o \ vorbiscomment.o wv.o dovi_isom.o OBJS-$(CONFIG_MCA_DEMUXER) += mca.o @@ -359,7 +360,7 @@ OBJS-$(CONFIG_MODS_DEMUXER) += mods.o OBJS-$(CONFIG_MOFLEX_DEMUXER) += moflex.o OBJS-$(CONFIG_MOV_DEMUXER) += mov.o mov_chan.o mov_esds.o \ qtpalette.o replaygain.o dovi_isom.o -OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o vpcc.o \ +OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o h266.o vpcc.o \ movenchint.o mov_chan.o rtp.o \ movenccenc.o movenc_ttml.o rawutils.o \ dovi_isom.o @@ -509,7 +510,7 @@ OBJS-$(CONFIG_RTP_MUXER) += rtp.o \ rtpenc_vp8.o \ rtpenc_vp9.o \ rtpenc_xiph.o \ - avc.o hevc.o + avc.o hevc.o h266.o OBJS-$(CONFIG_RTSP_DEMUXER) += rtsp.o rtspdec.o httpauth.o \ urldecode.o OBJS-$(CONFIG_RTSP_MUXER) += rtsp.o rtspenc.o httpauth.o \ diff --git a/libavformat/allformats.c b/libavformat/allformats.c index 437616bf0f..2dd76dd07a 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -200,6 +200,7 @@ extern const FFOutputFormat ff_h263_muxer; extern const AVInputFormat ff_h264_demuxer; extern const FFOutputFormat ff_h264_muxer; extern const AVInputFormat ff_h266_demuxer; +extern const FFOutputFormat ff_h266_muxer; extern const FFOutputFormat ff_hash_muxer; extern const AVInputFormat ff_hca_demuxer; extern const AVInputFormat ff_hcom_demuxer; diff --git a/libavformat/isom.c b/libavformat/isom.c index 6d019881e5..9fbccd4437 100644 --- a/libavformat/isom.c +++ b/libavformat/isom.c @@ -36,6 +36,7 @@ const AVCodecTag ff_mp4_obj_type[] = { { AV_CODEC_ID_MPEG4 , 0x20 }, { AV_CODEC_ID_H264 , 0x21 }, { AV_CODEC_ID_HEVC , 0x23 }, + { AV_CODEC_ID_VVC , 0x33 }, { AV_CODEC_ID_AAC , 0x40 }, { AV_CODEC_ID_MP4ALS , 0x40 }, /* 14496-3 ALS */ { AV_CODEC_ID_MPEG2VIDEO , 0x61 }, /* MPEG-2 Main */ diff --git a/libavformat/isom_tags.c b/libavformat/isom_tags.c index e2b80405cc..ec93bdc363 100644 --- a/libavformat/isom_tags.c +++ b/libavformat/isom_tags.c @@ -123,6 +123,9 @@ const AVCodecTag ff_codec_movvideo_tags[] = { { AV_CODEC_ID_HEVC, MKTAG('d', 'v', 'h', 'e') }, /* HEVC-based Dolby Vision derived from hev1 */ /* dvh1 is handled within mov.c */ + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'i', '1') }, /* VVC/H.266 which indicates parameter sets may be in ES */ + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'c', '1') }, /* VVC/H.266 which indicates parameter shall not be in ES */ + { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '1') }, /* AVC-1/H.264 */ { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '2') }, { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '3') }, diff --git a/libavformat/mov.c b/libavformat/mov.c index 6ab43b00c6..a1632bedb8 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -1967,6 +1967,11 @@ static int mov_read_glbl(MOVContext *c, AVIOContext *pb, MOVAtom atom) if ((uint64_t)atom.size > (1<<30)) return AVERROR_INVALIDDATA; + if (atom.type == MKTAG('v','v','c','C')) { + avio_rb32(pb); + atom.size -= 4; + } + if (atom.size >= 10) { // Broken files created by legacy versions of libavformat will // wrap a whole fiel atom inside of a glbl atom. @@ -7757,6 +7762,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('s','g','p','d'), mov_read_sgpd }, { MKTAG('s','b','g','p'), mov_read_sbgp }, { MKTAG('h','v','c','C'), mov_read_glbl }, +{ MKTAG('v','v','c','C'), mov_read_glbl }, { MKTAG('u','u','i','d'), mov_read_uuid }, { MKTAG('C','i','n', 0x8e), mov_read_targa_y216 }, { MKTAG('f','r','e','e'), mov_read_free }, diff --git a/libavformat/movenc.c b/libavformat/movenc.c index aca8b9d585..39260669a9 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -58,6 +58,7 @@ #include "libavutil/timecode.h" #include "libavutil/dovi_meta.h" #include "libavutil/uuid.h" +#include "h266.h" #include "hevc.h" #include "rtpenc.h" #include "mov_chan.h" @@ -1393,6 +1394,23 @@ static int mov_write_hvcc_tag(AVIOContext *pb, MOVTrack *track) return update_size(pb, pos); } +static int mov_write_vvcc_tag(AVIOContext *pb, MOVTrack *track) +{ + int64_t pos = avio_tell(pb); + + avio_wb32(pb, 0); + ffio_wfourcc(pb, "vvcC"); + + avio_w8 (pb, 0); /* version */ + avio_wb24(pb, 0); /* flags */ + + if (track->tag == MKTAG('v','v','i','1')) + ff_isom_write_vvcc(pb, track->vos_data, track->vos_len, 1); + else + ff_isom_write_vvcc(pb, track->vos_data, track->vos_len, 0); + return update_size(pb, pos); +} + /* also used by all avid codecs (dv, imx, meridien) and their variants */ static int mov_write_avid_tag(AVIOContext *pb, MOVTrack *track) { @@ -2286,6 +2304,8 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex avid = 1; } else if (track->par->codec_id == AV_CODEC_ID_HEVC) mov_write_hvcc_tag(pb, track); + else if (track->par->codec_id == AV_CODEC_ID_VVC) + mov_write_vvcc_tag(pb, track); else if (track->par->codec_id == AV_CODEC_ID_H264 && !TAG_IS_AVCI(track->tag)) { mov_write_avcc_tag(pb, track); if (track->mode == MODE_IPOD) @@ -6060,6 +6080,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) if ((par->codec_id == AV_CODEC_ID_DNXHD || par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_HEVC || + par->codec_id == AV_CODEC_ID_VVC || par->codec_id == AV_CODEC_ID_VP9 || par->codec_id == AV_CODEC_ID_TRUEHD) && !trk->vos_len && !TAG_IS_AVCI(trk->tag)) { @@ -6124,6 +6145,18 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) size = ff_hevc_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); } } + } else if (par->codec_id == AV_CODEC_ID_VVC && trk->vos_len > 6 && + (AV_RB24(trk->vos_data) == 1 || AV_RB32(trk->vos_data) == 1)) { + /* extradata is Annex B, assume the bitstream is too and convert it */ + if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { + ret = ff_h266_annexb2mp4_buf(pkt->data, &reformatted_data, + &size, 0, NULL); + if (ret < 0) + return ret; + avio_write(pb, reformatted_data, size); + } else { + size = ff_h266_annexb2mp4(pb, pkt->data, pkt->size, 0, NULL); + } } else if (par->codec_id == AV_CODEC_ID_AV1) { if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { ret = ff_av1_filter_obus_buf(pkt->data, &reformatted_data, @@ -6170,6 +6203,9 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) } else if(par->codec_id == AV_CODEC_ID_HEVC && par->extradata_size > 21) { int nal_size_length = (par->extradata[21] & 0x3) + 1; ret = ff_mov_cenc_avc_write_nal_units(s, &trk->cenc, nal_size_length, pb, pkt->data, size); + } else if(par->codec_id == AV_CODEC_ID_VVC && par->extradata_size > 21) { + int nal_size_length = (par->extradata[21] & 0x3) + 1; + ret = ff_mov_cenc_avc_write_nal_units(s, &trk->cenc, nal_size_length, pb, pkt->data, size); } else { ret = ff_mov_cenc_write_packet(&trk->cenc, pb, pkt->data, size); } @@ -7252,7 +7288,8 @@ static int mov_init(AVFormatContext *s) if (mov->encryption_scheme == MOV_ENC_CENC_AES_CTR) { ret = ff_mov_cenc_init(&track->cenc, mov->encryption_key, - (track->par->codec_id == AV_CODEC_ID_H264 || track->par->codec_id == AV_CODEC_ID_HEVC), + (track->par->codec_id == AV_CODEC_ID_H264 || track->par->codec_id == AV_CODEC_ID_HEVC || + track->par->codec_id == AV_CODEC_ID_VVC), s->flags & AVFMT_FLAG_BITEXACT); if (ret) return ret; @@ -7720,6 +7757,8 @@ static const AVCodecTag codec_mp4_tags[] = { { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '3') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'e', 'v', '1') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'v', 'c', '1') }, + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'c', '1') }, + { AV_CODEC_ID_VVC, MKTAG('v', 'v', 'i', '1') }, { AV_CODEC_ID_MPEG2VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MPEG1VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MJPEG, MKTAG('m', 'p', '4', 'v') }, diff --git a/libavformat/rawenc.c b/libavformat/rawenc.c index 78fadda967..c5176d9a5e 100644 --- a/libavformat/rawenc.c +++ b/libavformat/rawenc.c @@ -378,6 +378,29 @@ const FFOutputFormat ff_h264_muxer = { }; #endif +#if CONFIG_H266_MUXER +static int h266_check_bitstream(AVFormatContext *s, AVStream *st, + const AVPacket *pkt) +{ + if (pkt->size >= 5 && AV_RB32(pkt->data) != 0x0000001 && + AV_RB24(pkt->data) != 0x000001) + return ff_stream_add_bitstream_filter(st, "h266_mp4toannexb", NULL); + return 1; +} + +const FFOutputFormat ff_h266_muxer = { + .p.name = "vvc", + .p.long_name = NULL_IF_CONFIG_SMALL("raw H.266/VVC video"), + .p.extensions = "vvc,h266,266", + .p.audio_codec = AV_CODEC_ID_NONE, + .p.video_codec = AV_CODEC_ID_VVC, + .init = force_one_stream, + .write_packet = ff_raw_write_packet, + .check_bitstream = h266_check_bitstream, + .p.flags = AVFMT_NOTIMESTAMPS, +}; +#endif + #if CONFIG_HEVC_MUXER static int hevc_check_bitstream(AVFormatContext *s, AVStream *st, const AVPacket *pkt)