From patchwork Sun Dec 17 16:24:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Diego Felix de Souza via ffmpeg-devel X-Patchwork-Id: 45192 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1225:b0:181:818d:5e7f with SMTP id v37csp6886285pzf; Sun, 17 Dec 2023 08:24:48 -0800 (PST) X-Google-Smtp-Source: AGHT+IHHxjIO4qyaC5iZ8iEnsYDx4NIBAj7ka7IG0xR6OaxGw+zISDifUzGvJ+7ehlBj2NEMU52J X-Received: by 2002:a05:600c:3147:b0:40c:3fa7:bba8 with SMTP id h7-20020a05600c314700b0040c3fa7bba8mr6570714wmo.155.1702830287919; Sun, 17 Dec 2023 08:24:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702830287; cv=none; d=google.com; s=arc-20160816; b=g1zo7AQlXwy6vjz9TVcnKxw1zb9zWO6gCgy9e0hKaFY0mmNQQzIUK4+SW9qiv5/HYk VYrq5SJRnv68QHx7kg6P15lVEF23nR8+nY2ofmBuQVhDJN49Qe5tclT/GS75uJa6tJzv dYcrxgGbpa5YSMpD/vqpM0kWE9zjuZ3YbgEN9bMbPajX4eCPOHeLss2j+6//rYTGD1C8 fQIRYlpqaKGhQqQT6e2pIM3Pu2iZusyzAr3K8B1iVk8+6us0kU0EZnW3a3jlqBAipeF5 lclaVPEV+c0Ot73i4YT97h0La2Ahu7+XVzDvuXPLVuUU1MQ0C7hQdd1iBTLVojfrwQ4L RHXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:message-id:delivered-to; bh=01r2xhnVwwAxLVuVsLAtqaUkfQGLxgLnDismF/VPVro=; fh=frUT2hx60kzuIVmkNUv7bM3Y/kw3lxlKoLH+4fdDqZk=; b=fTFdgtqxOerVRavHnhFMoF3AyehwtJELso3gTxamJ60l5Epgcm6LuGqgOekMnzmfRd fDSpfNvpc2Oq8S9vc4qApWl53IuyEH9jXGXu49zyriDR7m6JazLn6/bL5VX532sZ5nsV PQkbp/foHTMN7lPwvJCNw0AFRwOk1vujd20BuHBTY+rjmt1tDWojfkIBQ/ueyCsSFdOt VGpv/AC4a3mW9PYV68zmcQjlaC46gXKRlDbzE4xALLQFfs7EbjHTtOsgIAZpfnCJx3In +XTdmgot7QhqdW/0GEhBiiUJ2A9ENbs+Owa8f6cD64yeVEUorNyUKn/rJIFm0CwKtVFP YXbQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b23-20020a170906491700b00a1db7b9706fsi8874001ejq.188.2023.12.17.08.24.47; Sun, 17 Dec 2023 08:24:47 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 985CB68D0EE; Sun, 17 Dec 2023 18:24:44 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out203-205-251-73.mail.qq.com (out203-205-251-73.mail.qq.com [203.205.251.73]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6086F68CFF7 for ; Sun, 17 Dec 2023 18:24:37 +0200 (EET) Received: from localhost.localdomain ([103.102.203.203]) by newxmesmtplogicsvrszb9-1.qq.com (NewEsmtp) with SMTP id 61EB6842; Mon, 18 Dec 2023 00:24:30 +0800 X-QQ-mid: xmsmtpt1702830270t9gyi81eu Message-ID: X-QQ-XMAILINFO: MIXpHopat2IaqYBfVcoO3y0vInPXOec/Y/DqzFRTYJ+SpFB9sCguIEHN8pWvkf N7yUAGeBJEoa1FxIEwqwDXD7sCwGeUUoy5eR0ocm3nAG2pknEcvpcwY9p0S5RISGVccTUUPN2Tbz oaCy45i39Uykclj6lwWmDDh+/eNNbOuT5sE3Hrcie7wbYRpmXTamtRtYIWzfQBxVcd6V7f+vgptP jJzK/neMaV1dP0O9IJL+PW7xV6BABblJbZyC45u37p37B67++Yk83egxMTVZz561bT62MdV6kysv QxbyBSy4cqBGGNDr7bhNKHGRJmnXXM4ePSarVRTzMAg4INEuo+D7xdi0/AlucZ3hbSL90jLffEeR VlHp/sVM3Lsh4+da71nNMNG+DopoT6WrmLP2ikb5tZUZ+L4LnBNTHzyEzceidziEqgurwGI/eNZm 597n0Rh2qojlsQSdAZL5QLwtq94xIUWw0OKyUtXzjHxuaXFZ0jqhy8attIC+ljal4gTF+Q5Nchwd RyPpChqI8IZz28bNM2XOdEjiA7ft6r/2Umx1QhPzSODmujU4J+/sFzuj3Gz455MO1HTpHJodYDt+ N/Aho1WGC51azwR25sGCnCj3hav7wHpYFzH35LpnT0Fu0YVoGHqVA+9KPVACSHlLpUhXVrpwIBrz 8RhKpxaKTkkXQA1WgxAW8mk++07kTkF2sAo1lVYdoVSNbXkwuEgJKLIcz9Zk59PDQPdfaZzf1Sot 9Hc5ykrKsDzJuEiRYuc2w+LC9/EcCd7x7bE6+ZST7NamWPsP3aTpAFWbeufhrH6aQr/hsPndeUdB zzLi/LMz5sTdLbu37gcxscan4jD2vnN0TU9JD1Z1cvD5xekF7nDri4NE0ENick6cbkvtI9wlt0Br F8KiYkUwVhWhCrrnJz37iVb9K8UUHarvLPrKHwfCjwFN9OhphsVHkC4vQEbRXhHNd5MiFPhv2ht8 M0b9prKsVqSukE19dwFYyQkgYR72Qo0QRoVa/6xVXHiEu2Mz6qHea9pqYmC+QUwMJh/+uXNgMjEo /p2Tnv/Eg77SWIYXvVjlUC97Qk1X0OU7maMuUdoEUXUtlYrQ0i5DHBZBT68wkxwC2/yn8OO4/TKe I65AlX X-QQ-XMRINFO: OD9hHCdaPRBwq3WW+NvGbIU= To: ffmpeg-devel@ffmpeg.org Date: Mon, 18 Dec 2023 00:24:25 +0800 X-OQ-MSGID: <20231217162425.93044-1-411294962@qq.com> X-Mailer: git-send-email 2.42.1 In-Reply-To: <29ed7620-f60f-4f31-bc22-3f0903456c07@gmail.com> References: <29ed7620-f60f-4f31-bc22-3f0903456c07@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/2] avformat/flvenc: support enhanced flv PacketTypeMetadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: zhupengfei via ffmpeg-devel From: Diego Felix de Souza via ffmpeg-devel Reply-To: FFmpeg development discussions and patches Cc: 411294962@qq.com Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 96DCqIkvix1m From: Zhu Pengfei <411294962@qq.com> Signed-off-by: Zhu Pengfei <411294962@qq.com> --- libavformat/flvenc.c | 155 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 155 insertions(+) diff --git a/libavformat/flvenc.c b/libavformat/flvenc.c index f6d10f331c..7268394b93 100644 --- a/libavformat/flvenc.c +++ b/libavformat/flvenc.c @@ -24,6 +24,7 @@ #include "libavutil/intfloat.h" #include "libavutil/avassert.h" #include "libavutil/mathematics.h" +#include "libavutil/mastering_display_metadata.h" #include "libavcodec/codec_desc.h" #include "libavcodec/mpeg4audio.h" #include "avio.h" @@ -124,6 +125,7 @@ typedef struct FLVContext { int flags; int64_t last_ts[FLV_STREAM_TYPE_NB]; + int write_metadata_pkt; } FLVContext; static int get_audio_flags(AVFormatContext *s, AVCodecParameters *par) @@ -478,6 +480,158 @@ static void write_metadata(AVFormatContext *s, unsigned int ts) avio_wb32(pb, flv->metadata_totalsize + 11); } +static void flv_write_metadata_packet(AVFormatContext *s, AVCodecParameters *par, unsigned int ts) +{ + AVIOContext *pb = s->pb; + FLVContext *flv = s->priv_data; + AVContentLightMetadata *lightMetadata = NULL; + AVMasteringDisplayMetadata *displayMetadata = NULL; + const int flags_size = 5; + int64_t metadata_size_pos = 0; + int64_t total_size = 0; + const AVPacketSideData *side_data = NULL; + + if (par->codec_id == AV_CODEC_ID_HEVC || par->codec_id == AV_CODEC_ID_AV1 || + par->codec_id == AV_CODEC_ID_VP9) { + if (flv->write_metadata_pkt) return; + + side_data = av_packet_side_data_get(par->coded_side_data, par->nb_coded_side_data, + AV_PKT_DATA_CONTENT_LIGHT_LEVEL); + if (side_data) + lightMetadata = (AVContentLightMetadata *)side_data->data; + + side_data = av_packet_side_data_get(par->coded_side_data, par->nb_coded_side_data, + AV_PKT_DATA_MASTERING_DISPLAY_METADATA); + if (side_data) + displayMetadata = (AVMasteringDisplayMetadata *)side_data->data; + + if (!lightMetadata && !displayMetadata) return; + + /* + * Reference Enhancing FLV + * https://github.com/veovera/enhanced-rtmp/blob/main/enhanced-rtmp.pdf + * */ + avio_w8(pb, FLV_TAG_TYPE_VIDEO); //write video tag type + metadata_size_pos = avio_tell(pb); + avio_wb24(pb, 0 + flags_size); + put_timestamp(pb, ts); //ts = pkt->dts, gen + avio_wb24(pb, flv->reserved); + + if (par->codec_id == AV_CODEC_ID_HEVC) { + avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeMetadata| FLV_FRAME_VIDEO_INFO_CMD); // ExVideoTagHeader mode with PacketTypeMetadata + avio_write(pb, "hvc1", 4); + } else if (par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9) { + avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeMetadata| FLV_FRAME_VIDEO_INFO_CMD); + avio_write(pb, par->codec_id == AV_CODEC_ID_AV1 ? "av01" : "vp09", 4); + } + + avio_w8(pb, AMF_DATA_TYPE_STRING); + put_amf_string(pb, "colorInfo"); + + avio_w8(pb, AMF_DATA_TYPE_OBJECT); + + put_amf_string(pb, "colorConfig"); // colorConfig + + /* mixed array (hash) with size and string/type/data tuples */ + avio_w8(pb, AMF_DATA_TYPE_MIXEDARRAY); + + avio_wb32(pb, 0); // write array count + + if (par->color_trc != AVCOL_TRC_UNSPECIFIED && + par->color_trc < AVCOL_TRC_NB) { + put_amf_string(pb, "transferCharacteristics"); // color_trc + put_amf_double(pb, par->color_trc); + } + + if (par->color_space != AVCOL_SPC_UNSPECIFIED && + par->color_space < AVCOL_SPC_NB) { + put_amf_string(pb, "matrixCoefficients"); // colorspace + put_amf_double(pb, par->color_space); + } + + if (par->color_primaries != AVCOL_PRI_UNSPECIFIED && + par->color_primaries < AVCOL_PRI_NB) { + put_amf_string(pb, "colorPrimaries"); // color_primaries + put_amf_double(pb, par->color_primaries); + } + + put_amf_string(pb, ""); + avio_w8(pb, AMF_END_OF_OBJECT); // array end of object + + if (lightMetadata) { + put_amf_string(pb, "hdrCll"); + + /* mixed array (hash) with size and string/type/data tuples */ + avio_w8(pb, AMF_DATA_TYPE_MIXEDARRAY); + + avio_wb32(pb, 0); // write array count + + put_amf_string(pb, "maxFall"); + put_amf_double(pb, lightMetadata->MaxFALL); + + put_amf_string(pb, "maxCLL"); + put_amf_double(pb, lightMetadata->MaxCLL); + + // array end of object + put_amf_string(pb, ""); + avio_w8(pb, AMF_END_OF_OBJECT); + } + + if (displayMetadata && (displayMetadata->has_primaries || displayMetadata->has_luminance)) { + put_amf_string(pb, "hdrMdcv"); + + /* mixed array (hash) with size and string/type/data tuples */ + avio_w8(pb, AMF_DATA_TYPE_MIXEDARRAY); + avio_wb32(pb, 0); // write array count + + if (displayMetadata->has_primaries) { + put_amf_string(pb, "redX"); + put_amf_double(pb, av_q2d(displayMetadata->display_primaries[0][0])); + + put_amf_string(pb, "redY"); + put_amf_double(pb, av_q2d(displayMetadata->display_primaries[0][1])); + + put_amf_string(pb, "greenX"); + put_amf_double(pb, av_q2d(displayMetadata->display_primaries[1][0])); + + put_amf_string(pb, "greenY"); + put_amf_double(pb, av_q2d(displayMetadata->display_primaries[1][1])); + + put_amf_string(pb, "blueX"); + put_amf_double(pb, av_q2d(displayMetadata->display_primaries[2][0])); + + put_amf_string(pb, "blueY"); + put_amf_double(pb, av_q2d(displayMetadata->display_primaries[2][1])); + + put_amf_string(pb, "whitePointX"); + put_amf_double(pb, av_q2d(displayMetadata->white_point[0])); + + put_amf_string(pb, "whitePointY"); + put_amf_double(pb, av_q2d(displayMetadata->white_point[1])); + } + if (displayMetadata->has_luminance) { + put_amf_string(pb, "maxLuminance"); + put_amf_double(pb, av_q2d(displayMetadata->max_luminance)); + + put_amf_string(pb, "minLuminance"); + put_amf_double(pb, av_q2d(displayMetadata->min_luminance)); + } + put_amf_string(pb, ""); + avio_w8(pb, AMF_END_OF_OBJECT); + } + avio_w8(pb, AMF_DATA_TYPE_OBJECT_END); + + total_size = avio_tell(pb) - metadata_size_pos - 10; + avio_seek(pb, metadata_size_pos, SEEK_SET); + avio_wb24(pb, total_size); + avio_tell(pb); + avio_skip(pb, total_size + 10 - 3); + avio_tell(pb); + avio_wb32(pb, total_size + 11); // previous tag size + flv->write_metadata_pkt = 1; + } +} + static int unsupported_codec(AVFormatContext *s, const char* type, int codec_id) { @@ -878,6 +1032,7 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt) memcpy(par->extradata, side, side_size); flv_write_codec_header(s, par, pkt->dts); } + flv_write_metadata_packet(s, par, pkt->dts); } if (flv->delay == AV_NOPTS_VALUE)