From patchwork Thu Nov 3 12:21:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Siedel X-Patchwork-Id: 39140 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp500573pzb; Thu, 3 Nov 2022 05:24:14 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7poDwQSENiiapDqZiicTW/3z00+FVbPFbutb9UI3szMPPiXUuxj7UqOev/fmHT7OPADq/C X-Received: by 2002:a17:906:9c84:b0:7ad:90d9:a6cf with SMTP id fj4-20020a1709069c8400b007ad90d9a6cfmr28193401ejc.595.1667478253901; Thu, 03 Nov 2022 05:24:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667478253; cv=none; d=google.com; s=arc-20160816; b=U3n3B7YH5rL7eGG7hbCQ+pxtTOB6QKYlbPJ2LAi6SqVtpJy7M+wn2rlIl6f/cURRPr wPS4Ziv/EQvOQ0C2eR22OGqneRlR0tSxfuzN15+MWmnzLUcqMtjaIM5E7htY3S/DvoGR zZEVrFScrnRTBnvG3AREkVLKCcnGdpBg26tlL6QTXNvCawfcj+Hj/5EeH0lO7xDZ+Wl0 CEe2sD3xO2EWi1oCyHioh1P5OPLVfqDx2nfwvZlXx/llkkkKoO3BBzO2crhNuM3hZ9Wj BwlXaJ8NlRmNY+iZQn8y5tid8ovqW9m9wimSU8xSraLAQnCGUsT2djI7rft6UrFIsQn3 0o/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=Ml7We848sIoh7vyUHHRIIJ+g4TYcuR8J1Oaqkl382lw=; b=Ry3CN/6QhBjDZYyywMJZ/I62DdgEQWv5ZGabpuU6mX1zSTrreK3kt3ntPIpmHKS3sY C97rTfWDBItMkdM6Q1J0LNstsDHEsbgllGEYLrERgXlEu8gc5esAq9RN2O4q7xTYzay+ intpP2ec6t4ywtb1Nfgy9/TpbwksOeZJWgIcldXZKwXm9V5O5tny1gNXH8G8X5HwnSiW FoT+8pVzbQbtqCBdnbHWqrf7e4YxaxBUlvwcTMYxEX8eFgTm7pmBNMwY8oSv0aaoe/Ln FyUQbZbPm4RhHzvvNyZz+N3m24b0KyjUBoPJkVhz5kxdf+48LXAOUzGAqKtxNzTpO6Jm atQA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=7fBrbC2p; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id oz30-20020a1709077d9e00b0078d0a31f987si1187194ejc.755.2022.11.03.05.24.12; Thu, 03 Nov 2022 05:24:13 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@spin-digital-com.20210112.gappssmtp.com header.s=20210112 header.b=7fBrbC2p; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0245368BF75; Thu, 3 Nov 2022 14:22:49 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f50.google.com (mail-ed1-f50.google.com [209.85.208.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9925F68BF72 for ; Thu, 3 Nov 2022 14:22:40 +0200 (EET) Received: by mail-ed1-f50.google.com with SMTP id r14so2705026edc.7 for ; Thu, 03 Nov 2022 05:22:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=St8zVojTt6hF9wYILFoo4TW8YX35pKHeGo2YGJtAgEw=; b=7fBrbC2pA6aHjcGkhmDXJYqnv7AxtrM+2qgwGMq71E+5wN9wd1RIM53cr6J3v2DMDV ogueGbgo0jm4J2ChxXjgBsGOOwlzxB0UGqIpVPww2GkaPcCfXIZa3qPl2ENj9y1SJh+z rIdcNOOcKc0Rsur7OanmMplZ/wJ677F9Twc7AmqvT0ZR2l4lTkcL+v7gYW1ykyIo8y8x 5er0FCcAzKDpBT/25Yy21RULvoEwuEs9j9Dm2Ltb3jgfqidab6Wn8lRc3CaAHO0mxwOz PyI84hdkTn1m1mRzSJv67udRpgjFrsWgndansnCHUb1UxxXprsRw6WXmOJq0asi2KxNV 3qCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=St8zVojTt6hF9wYILFoo4TW8YX35pKHeGo2YGJtAgEw=; b=siYbUlamn3BZaOOMRfuOoc1TWhCyubtGmLR70Bqz+pMJ2PvO/l2/GCySz+2vkNPjbM auXkz+txiEdq1Gxntj10ZS1mWIMSSQfjAoZ+EQ56Snj/pyThKcziPH74bD+qd2U4UByP k+Vo6sL58EowAp3kZBk7yaC2SZbj9k3B+3Y4OuMRRTsF1R2rwbGoGwPJlG6GFcEIER5E OD+V+AfuFuipS1OmIXPSCAQz6NnpZQtmkMmw9Og2sF79IEF1v+1gdxUla/QnyQzNHJ6+ P61e2/mQpTLu38iYY+FZm+Vg+MYYIgPRNdGmxdsCuiRy25aHecO4J2b0OOPwKjREJpuE XZ8Q== X-Gm-Message-State: ACrzQf2TfcxH/0xc6gEybNYOemBZ1ovdtShrbdr6b85aq1D62+EDPFot jIh0oRZWRNp4d9aLYM4lVSGF3JdMsiZVsw== X-Received: by 2002:a05:6402:2681:b0:461:cbda:fe4b with SMTP id w1-20020a056402268100b00461cbdafe4bmr30203925edd.124.1667478159724; Thu, 03 Nov 2022 05:22:39 -0700 (PDT) Received: from thomas-win.localdomain ([213.138.44.237]) by smtp.gmail.com with ESMTPSA id c7-20020a0564021f8700b0045c010d0584sm457117edc.47.2022.11.03.05.22.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Nov 2022 05:22:39 -0700 (PDT) From: Thomas Siedel To: ffmpeg-devel@ffmpeg.org Date: Thu, 3 Nov 2022 13:21:57 +0100 Message-Id: <20221103122158.27169-10-thomas.ff@spin-digital.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221103122158.27169-1-thomas.ff@spin-digital.com> References: <20221103122158.27169-1-thomas.ff@spin-digital.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 09/10] avformat: add ts stream types for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Thomas Siedel Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: X6jIJColYW84 Add transport stream stream type 0x33 for vvc. Add STREAM_TYPE_VIDEO_VVC to MPEG-1/2 and MPEG-2 transport stream. Add basic transport stream support for TS mux/demux. Signed-off-by: Thomas Siedel --- configure | 2 +- libavformat/mpeg.c | 3 ++ libavformat/mpeg.h | 1 + libavformat/mpegts.c | 2 ++ libavformat/mpegts.h | 1 + libavformat/mpegtsenc.c | 65 +++++++++++++++++++++++++++++++++++++++++ 6 files changed, 73 insertions(+), 1 deletion(-) diff --git a/configure b/configure index 8e3d11bca5..c5a11a6ed3 100755 --- a/configure +++ b/configure @@ -3485,7 +3485,7 @@ mp3_demuxer_select="mpegaudio_parser" mp3_muxer_select="mpegaudioheader" mp4_muxer_select="mov_muxer" mpegts_demuxer_select="iso_media" -mpegts_muxer_select="ac3_parser adts_muxer latm_muxer h264_mp4toannexb_bsf hevc_mp4toannexb_bsf" +mpegts_muxer_select="ac3_parser adts_muxer latm_muxer h264_mp4toannexb_bsf hevc_mp4toannexb_bsf vvc_mp4toannexb_bsf" mpegtsraw_demuxer_select="mpegts_demuxer" mxf_muxer_select="pcm_rechunk_bsf" mxf_d10_muxer_select="mxf_muxer" diff --git a/libavformat/mpeg.c b/libavformat/mpeg.c index 864b08d8f8..970926df6b 100644 --- a/libavformat/mpeg.c +++ b/libavformat/mpeg.c @@ -546,6 +546,9 @@ redo: } else if (es_type == STREAM_TYPE_VIDEO_HEVC) { codec_id = AV_CODEC_ID_HEVC; type = AVMEDIA_TYPE_VIDEO; + } else if (es_type == STREAM_TYPE_VIDEO_VVC) { + codec_id = AV_CODEC_ID_VVC; + type = AVMEDIA_TYPE_VIDEO; } else if (es_type == STREAM_TYPE_AUDIO_AC3) { codec_id = AV_CODEC_ID_AC3; type = AVMEDIA_TYPE_AUDIO; diff --git a/libavformat/mpeg.h b/libavformat/mpeg.h index b635295776..20592eb184 100644 --- a/libavformat/mpeg.h +++ b/libavformat/mpeg.h @@ -56,6 +56,7 @@ #define STREAM_TYPE_VIDEO_MPEG4 0x10 #define STREAM_TYPE_VIDEO_H264 0x1b #define STREAM_TYPE_VIDEO_HEVC 0x24 +#define STREAM_TYPE_VIDEO_VVC 0x33 #define STREAM_TYPE_VIDEO_CAVS 0x42 #define STREAM_TYPE_AUDIO_AC3 0x81 diff --git a/libavformat/mpegts.c b/libavformat/mpegts.c index d97702fcd7..61a800c85f 100644 --- a/libavformat/mpegts.c +++ b/libavformat/mpegts.c @@ -811,6 +811,7 @@ static const StreamType ISO_types[] = { { 0x20, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_H264 }, { 0x21, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_JPEG2000 }, { 0x24, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_HEVC }, + { 0x33, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_VVC }, { 0x42, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_CAVS }, { 0xd1, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_DIRAC }, { 0xd2, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_AVS2 }, @@ -865,6 +866,7 @@ static const StreamType REGD_types[] = { { MKTAG('D', 'T', 'S', '3'), AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_DTS }, { MKTAG('E', 'A', 'C', '3'), AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_EAC3 }, { MKTAG('H', 'E', 'V', 'C'), AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_HEVC }, + { MKTAG('V', 'V', 'C', ' '), AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_VVC }, { MKTAG('K', 'L', 'V', 'A'), AVMEDIA_TYPE_DATA, AV_CODEC_ID_SMPTE_KLV }, { MKTAG('I', 'D', '3', ' '), AVMEDIA_TYPE_DATA, AV_CODEC_ID_TIMED_ID3 }, { MKTAG('V', 'C', '-', '1'), AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_VC1 }, diff --git a/libavformat/mpegts.h b/libavformat/mpegts.h index a48f14e768..14ae312c50 100644 --- a/libavformat/mpegts.h +++ b/libavformat/mpegts.h @@ -128,6 +128,7 @@ #define STREAM_TYPE_METADATA 0x15 #define STREAM_TYPE_VIDEO_H264 0x1b #define STREAM_TYPE_VIDEO_HEVC 0x24 +#define STREAM_TYPE_VIDEO_VVC 0x33 #define STREAM_TYPE_VIDEO_CAVS 0x42 #define STREAM_TYPE_VIDEO_AVS2 0xd2 #define STREAM_TYPE_VIDEO_AVS3 0xd4 diff --git a/libavformat/mpegtsenc.c b/libavformat/mpegtsenc.c index 48d39e6a7d..85546f6a49 100644 --- a/libavformat/mpegtsenc.c +++ b/libavformat/mpegtsenc.c @@ -366,6 +366,9 @@ static int get_dvb_stream_type(AVFormatContext *s, AVStream *st) case AV_CODEC_ID_HEVC: stream_type = STREAM_TYPE_VIDEO_HEVC; break; + case AV_CODEC_ID_VVC: + stream_type = STREAM_TYPE_VIDEO_VVC; + break; case AV_CODEC_ID_CAVS: stream_type = STREAM_TYPE_VIDEO_CAVS; break; @@ -462,6 +465,11 @@ static int get_m2ts_stream_type(AVFormatContext *s, AVStream *st) case AV_CODEC_ID_HEVC: stream_type = STREAM_TYPE_VIDEO_HEVC; break; + case AV_CODEC_ID_VVC: + av_log(s, AV_LOG_ERROR, + "MPEGTS VVC %s.\n", avcodec_get_name(st->codecpar->codec_id)); + stream_type = STREAM_TYPE_VIDEO_VVC; + break; case AV_CODEC_ID_PCM_BLURAY: stream_type = 0x80; break; @@ -1783,6 +1791,21 @@ static int check_hevc_startcode(AVFormatContext *s, const AVStream *st, const AV return 0; } +static int check_vvc_startcode(AVFormatContext *s, const AVStream *st, const AVPacket *pkt) +{ + if (pkt->size < 5 || AV_RB32(pkt->data) != 0x0000001 && AV_RB24(pkt->data) != 0x000001) { + if (!st->nb_frames) { + av_log(s, AV_LOG_ERROR, "VVC bitstream malformed, no startcode found\n"); + return AVERROR_PATCHWELCOME; + } + av_log(s, AV_LOG_WARNING, "VVC bitstream error, startcode missing, size %d", pkt->size); + if (pkt->size) + av_log(s, AV_LOG_WARNING, " data %08"PRIX32, AV_RB32(pkt->data)); + av_log(s, AV_LOG_WARNING, "\n"); + } + return 0; +} + /* Based on GStreamer's gst-plugins-base/ext/ogg/gstoggstream.c * Released under the LGPL v2.1+, written by * Vincent Penquerc'h @@ -1975,6 +1998,42 @@ static int mpegts_write_packet_internal(AVFormatContext *s, AVPacket *pkt) buf = data; size = pkt->size + 7 + extradd; } + } else if (st->codecpar->codec_id == AV_CODEC_ID_VVC) { + const uint8_t *p = buf, *buf_end = p + size; + uint32_t state = -1; + uint32_t naltype = -1; + int extradd = (pkt->flags & AV_PKT_FLAG_KEY) ? st->codecpar->extradata_size : 0; + int ret = check_vvc_startcode(s, st, pkt); + if (ret < 0) + return ret; + + if (extradd && AV_RB24(st->codecpar->extradata) > 1) + extradd = 0; + + do { + p = avpriv_find_start_code(p, buf_end, &state); + // state contains byte behind start code, p points 2 bytes behind start code + naltype = ((AV_RB8(p)>>3) & 0x1F); + av_log(s, AV_LOG_TRACE, "nal %"PRId32"\n", naltype ); + if (naltype == 14 ) // VPS + extradd = 0; + } while (p < buf_end && naltype != 20 && naltype >= 12); + + if (naltype >= 12) + extradd = 0; + if (naltype != 20) { // AUD NAL + data = av_malloc(pkt->size + 7 + extradd); + if (!data) + return AVERROR(ENOMEM); + memcpy(data + 7, st->codecpar->extradata, extradd); + memcpy(data + 7 + extradd, pkt->data, pkt->size); + AV_WB32(data, 0x00000001); + data[4] = 20; + data[5] = 1; + data[6] = 0x50; // any slice type (0x4) + rbsp stop one bit + buf = data; + size = pkt->size + 7 + extradd; + } } else if (st->codecpar->codec_id == AV_CODEC_ID_OPUS) { if (pkt->size < 2) { av_log(s, AV_LOG_ERROR, "Opus packet too short\n"); @@ -2231,6 +2290,12 @@ static int mpegts_check_bitstream(AVFormatContext *s, AVStream *st, (st->codecpar->extradata_size > 0 && st->codecpar->extradata[0] == 1))) ret = ff_stream_add_bitstream_filter(st, "hevc_mp4toannexb", NULL); + } else if (st->codecpar->codec_id == AV_CODEC_ID_VVC) { + if (pkt->size >= 5 && AV_RB32(pkt->data) != 0x0000001 && + (AV_RB24(pkt->data) != 0x000001 || + (st->codecpar->extradata_size > 0 && + st->codecpar->extradata[0] == 1))) + ret = ff_stream_add_bitstream_filter(st, "vvc_mp4toannexb", NULL); } return ret;