From patchwork Mon Aug 5 14:01:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mondain X-Patchwork-Id: 50899 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:1f5a:b0:489:2eb3:e4c4 with SMTP id jm26csp1421166vqb; Mon, 5 Aug 2024 07:02:17 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWCaUC0Nx8ol2L/dOtdsahzz9TprLzsPAdMSK+9ypdwmnYhMIapZJopsU93v6cuuSfSdZuGCAsfNYVzWgMOLysJNTUbPT7h+rzMmQ== X-Google-Smtp-Source: AGHT+IHC5ZjfTLFAFPLmQPfAEuFeX9oC/dB3lRbzw5fnxyIeci4uGP8ERsbUzGPWRD3hAMROSUaH X-Received: by 2002:aa7:c50e:0:b0:5a2:87d3:6ee6 with SMTP id 4fb4d7f45d1cf-5b7f57f41f4mr8160093a12.32.1722866537673; Mon, 05 Aug 2024 07:02:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1722866537; cv=none; d=google.com; s=arc-20160816; b=lKickJFyqpfBJVW1lmt19dX1uRSbmpb3gVrPz6/e2h1t1/NN1ow63PMXP3SEpDTo3B lRldiCAKhNa9D3Ere5Fp4EF7IfmpDpGbW7clTsIIF2jhN1RcQwwWv2d0i008KNhNRDor e0pUtEvSIUCxqlsRjN6C/09xCK8EjzluodB3k0GfOA2+L8l4b/rBbHBbhLMMheB/6hWC umJr438zmhjLuFy4Ymsi91043Nw8SMFxq9qIf8jMzjmng7mwNRWXba0uwqFdtk6m7syk DlaiSHjJHgIGHgM4bN9gpTWzKDv0vX7RdJEuApLqJ3NPx/eI2OMIxaCsQsX7huv/nrEU aljw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:to:message-id:date:from:mime-version :dkim-signature:delivered-to; bh=JGbF69wUT0KY8LR3nMjr5AH3qGrmPoj6PgVa/QjN0j8=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=q3Vr4AVLRC7KroOD/EpYLT2BBYxPyYvUW4tPvoLxksQIAfN4WFdFoV/vIVtz7ILRvo eJt5c9jaT1/pErmqVXywwvqP8dHzYZqWPPQhmrVdHLWrpHuZtYyk5MxnvO4APtmzrELv /ZWA2LeC1buAI1pGKYheTQCKEwzumwvfKAooxyQesQtxL9Zolx0K4AM57tiWKDVOZVtO eFy/E6Zrd/OOylOMNIReFcUIWk0cmSwCNrZa2BkkWLARRv+fKQ5tKRRE2bPatOXnFkoO GisK3n+eBEYF9bGwMVBAQRXVP840l4bSdElPlMrCrtRBggv6rZcE7RAclLJqFSr+28Qr 1yRQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=VTUyEUzn; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5b83c038b44si4279541a12.465.2024.08.05.07.02.16; Mon, 05 Aug 2024 07:02:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=VTUyEUzn; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0EDB168D766; Mon, 5 Aug 2024 17:02:13 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-il1-f179.google.com (mail-il1-f179.google.com [209.85.166.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8074468D766 for ; Mon, 5 Aug 2024 17:02:06 +0300 (EEST) Received: by mail-il1-f179.google.com with SMTP id e9e14a558f8ab-397ba4f7387so55100715ab.0 for ; Mon, 05 Aug 2024 07:02:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722866525; x=1723471325; darn=ffmpeg.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=7OS2k+regiwceDTDLaOi4jGJG0gLvhZ/n4I5MNfUn3Y=; b=VTUyEUzngcYrmF/gwO/vah9pIShcQOeS8gOBT5B1biE5yypdgG9EfFVJ2yRa7XNCaB /QnXerLOEdTQKz3hkcjle2s/TybRKFwcSQWYPrs6ou4fqmJHYJb4VZW0ZBvde6qlK24o vHmCJ1WSWctA8hfxmt3lkL0E5We8n+VaXVQMPEhZFDuJHLM5woLr0trL8F4SxF88DCbE ssGKKS080zKbNFLgDjF7Jee1cbauXW6gU8EaWeet3IrQC2GNeS214Vt4KzaB4R2gQBkd 2Euwb6DSk2kgGOMqt6R+tSA2dKz20/cKOZx/LaJwiGWq3N7swENdy5FtItsH/kuF/Dpl GoIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722866525; x=1723471325; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=7OS2k+regiwceDTDLaOi4jGJG0gLvhZ/n4I5MNfUn3Y=; b=RwqnF9GfJwhUV+t19ETtHvHABWslk3VNVmPV6ryhbc0S3bm5Bb8k3Lzu17AwPBk9eE wuVEpMJ/sz6GUNMt9MzxEy7puvsw4AFwSeBrkTfFF+EpKKXEZmJmZ2EVWL/wLLWMVJAN /rJwAi/FLwbY0CFsf02/Z1b7VGc/Tney1WMn3+njiK49yKsBLOj3JfJiScVcwNEUuVXc 2BoxkAbbaWYc8iExQujzebivzboJuQesU5QcLkdXGX40nWNQzo49509cub3mgRihz4bC kH6ygIrCHWU6LBfLbuoNqv0mTyW1sOKO8sZKZttVDmv/x+khcOamerXCA/X6NqAU3KbM 49kQ== X-Gm-Message-State: AOJu0YyCb3HxPVlT1vIrvQM+XRcWElJEcSJIkwBBlCwDz0bYxrUDdHSU 4g8bwbNJLtva5sDqgWILQQ4le+KALBnF2wFyKtuJU8ybTnNtnF6feovHjZ4pdXuGG86ilOvKo8z eHorOIJi809z/zdSMA7UqndoryXzxWFwv X-Received: by 2002:a92:6610:0:b0:399:2c60:9951 with SMTP id e9e14a558f8ab-39b1fbf5c86mr126760015ab.20.1722866524261; Mon, 05 Aug 2024 07:02:04 -0700 (PDT) MIME-Version: 1.0 From: Mondain Date: Mon, 5 Aug 2024 07:01:37 -0700 Message-ID: To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH] Add enhanced rtmp codec vp8 to flv format X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ZM5cqEn9qvnI Fixes: Reading and writing of VP8 media in FLV files, to match existing enhanced codecs Signed-off-by: Paul Gregoire --- libavformat/flvdec.c | 12 +++++++++-- libavformat/flvenc.c | 51 +++++++++++++++++++++++++++++--------------- 2 files changed, 44 insertions(+), 19 deletions(-) side_data = av_packet_side_data_get(par->coded_side_data, par->nb_coded_side_data, AV_PKT_DATA_CONTENT_LIGHT_LEVEL); @@ -519,12 +520,18 @@ static void flv_write_metadata_packet(AVFormatContext *s, AVCodecParameters *par put_timestamp(pb, ts); //ts = pkt->dts, gen avio_wb24(pb, flv->reserved); - if (par->codec_id == AV_CODEC_ID_HEVC) { - avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeMetadata| FLV_FRAME_VIDEO_INFO_CMD); // ExVideoTagHeader mode with PacketTypeMetadata + if (par->codec_id == AV_CODEC_ID_VP8) { + avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeMetadata | FLV_FRAME_VIDEO_INFO_CMD); // ExVideoTagHeader mode with PacketTypeMetadata + avio_write(pb, "vp08", 4); + } else if (par->codec_id == AV_CODEC_ID_VP9) { + avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeMetadata | FLV_FRAME_VIDEO_INFO_CMD); + avio_write(pb, "vp09", 4); + } else if (par->codec_id == AV_CODEC_ID_HEVC) { + avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeMetadata | FLV_FRAME_VIDEO_INFO_CMD); avio_write(pb, "hvc1", 4); - } else if (par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9) { - avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeMetadata| FLV_FRAME_VIDEO_INFO_CMD); - avio_write(pb, par->codec_id == AV_CODEC_ID_AV1 ? "av01" : "vp09", 4); + } else if (par->codec_id == AV_CODEC_ID_AV1) { + avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeMetadata | FLV_FRAME_VIDEO_INFO_CMD); + avio_write(pb, "av01", 4); } avio_w8(pb, AMF_DATA_TYPE_STRING); @@ -639,7 +646,8 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i if (par->codec_id == AV_CODEC_ID_AAC || par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_MPEG4 || par->codec_id == AV_CODEC_ID_HEVC - || par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9) { + || par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9 + || par->codec_id == AV_CODEC_ID_VP8) { int64_t pos; avio_w8(pb, par->codec_type == AVMEDIA_TYPE_VIDEO ? @@ -682,23 +690,28 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i } avio_write(pb, par->extradata, par->extradata_size); } else { - if (par->codec_id == AV_CODEC_ID_HEVC) { + if (par->codec_id == AV_CODEC_ID_VP8) { avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeSequenceStart | FLV_FRAME_KEY); // ExVideoTagHeader mode with PacketTypeSequenceStart + avio_write(pb, "vp08", 4); + } else if (par->codec_id == AV_CODEC_ID_VP9) { + avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeSequenceStart | FLV_FRAME_KEY); + avio_write(pb, "vp09", 4); + } else if (par->codec_id == AV_CODEC_ID_HEVC) { + avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeSequenceStart | FLV_FRAME_KEY); avio_write(pb, "hvc1", 4); - } else if (par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9) { + } else if (par->codec_id == AV_CODEC_ID_AV1) { avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeSequenceStart | FLV_FRAME_KEY); - avio_write(pb, par->codec_id == AV_CODEC_ID_AV1 ? "av01" : "vp09", 4); + avio_write(pb, "av01", 4); } else { avio_w8(pb, par->codec_tag | FLV_FRAME_KEY); // flags avio_w8(pb, 0); // AVC sequence header avio_wb24(pb, 0); // composition time } - if (par->codec_id == AV_CODEC_ID_HEVC) ff_isom_write_hvcc(pb, par->extradata, par->extradata_size, 0); else if (par->codec_id == AV_CODEC_ID_AV1) ff_isom_write_av1c(pb, par->extradata, par->extradata_size, 1); - else if (par->codec_id == AV_CODEC_ID_VP9) + else if (par->codec_id == AV_CODEC_ID_VP9 || par->codec_id == AV_CODEC_ID_VP8) ff_isom_write_vpcc(s, pb, par->extradata, par->extradata_size, par); else ff_isom_write_avcc(pb, par->extradata, par->extradata_size); @@ -1001,7 +1014,7 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt) flags_size = 2; else if (par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_MPEG4 || par->codec_id == AV_CODEC_ID_HEVC || par->codec_id == AV_CODEC_ID_AV1 || - par->codec_id == AV_CODEC_ID_VP9) + par->codec_id == AV_CODEC_ID_VP9 || par->codec_id == AV_CODEC_ID_VP8) flags_size = 5; else flags_size = 1; @@ -1011,7 +1024,8 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt) if (par->codec_id == AV_CODEC_ID_AAC || par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_MPEG4 || par->codec_id == AV_CODEC_ID_HEVC - || par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9) { + || par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9 + || par->codec_id == AV_CODEC_ID_VP8) { size_t side_size; uint8_t *side = av_packet_get_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, &side_size); if (side && side_size > 0 && (side_size != par->extradata_size || memcmp(side, par->extradata, side_size))) { @@ -1034,7 +1048,7 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt) } if (par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_MPEG4 || par->codec_id == AV_CODEC_ID_HEVC || par->codec_id == AV_CODEC_ID_AV1 || - par->codec_id == AV_CODEC_ID_VP9) { + par->codec_id == AV_CODEC_ID_VP9 || par->codec_id == AV_CODEC_ID_VP8) { if (pkt->pts == AV_NOPTS_VALUE) { av_log(s, AV_LOG_ERROR, "Packet is missing PTS\n"); return AVERROR(EINVAL); @@ -1150,9 +1164,12 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt) avio_write(pb, "hvc1", 4); if (pkttype == PacketTypeCodedFrames) avio_wb24(pb, pkt->pts - pkt->dts); - } else if (par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9) { + } else if (par->codec_id == AV_CODEC_ID_AV1) { + avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeCodedFrames | frametype); + avio_write(pb, "av01", 4); + } else if (par->codec_id == AV_CODEC_ID_VP9 || par->codec_id == AV_CODEC_ID_VP8) { avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeCodedFrames | frametype); - avio_write(pb, par->codec_id == AV_CODEC_ID_AV1 ? "av01" : "vp09", 4); + avio_write(pb, par->codec_id == AV_CODEC_ID_VP8 ? "vp08" : "vp09", 4); } else { avio_w8(pb, flags); } diff --git a/libavformat/flvdec.c b/libavformat/flvdec.c index 22a9b9e4a7..2383682a63 100644 --- a/libavformat/flvdec.c +++ b/libavformat/flvdec.c @@ -340,6 +340,8 @@ static int flv_same_video_codec(AVCodecParameters *vpar, uint32_t flv_codecid) return vpar->codec_id == AV_CODEC_ID_AV1; case MKBETAG('v', 'p', '0', '9'): return vpar->codec_id == AV_CODEC_ID_VP9; + case MKBETAG('v', 'p', '0', '8'): + return vpar->codec_id == AV_CODEC_ID_VP8; case FLV_CODECID_H263: return vpar->codec_id == AV_CODEC_ID_FLV1; case FLV_CODECID_SCREEN: @@ -378,6 +380,10 @@ static int flv_set_video_codec(AVFormatContext *s, AVStream *vstream, par->codec_id = AV_CODEC_ID_VP9; vstreami->need_parsing = AVSTREAM_PARSE_HEADERS; break; + case MKBETAG('v', 'p', '0', '8'): + par->codec_id = AV_CODEC_ID_VP8; + vstreami->need_parsing = AVSTREAM_PARSE_HEADERS; + break; case FLV_CODECID_H263: par->codec_id = AV_CODEC_ID_FLV1; break; @@ -1443,7 +1449,8 @@ retry_duration: st->codecpar->codec_id == AV_CODEC_ID_MPEG4 || st->codecpar->codec_id == AV_CODEC_ID_HEVC || st->codecpar->codec_id == AV_CODEC_ID_AV1 || - st->codecpar->codec_id == AV_CODEC_ID_VP9) { + st->codecpar->codec_id == AV_CODEC_ID_VP9 || + st->codecpar->codec_id == AV_CODEC_ID_VP8) { int type = 0; if (enhanced_flv && stream_type == FLV_STREAM_TYPE_VIDEO) { type = flags & 0x0F; @@ -1481,7 +1488,8 @@ retry_duration: } if (type == 0 && (!st->codecpar->extradata || st->codecpar->codec_id == AV_CODEC_ID_AAC || st->codecpar->codec_id == AV_CODEC_ID_H264 || st->codecpar->codec_id == AV_CODEC_ID_HEVC || - st->codecpar->codec_id == AV_CODEC_ID_AV1 || st->codecpar->codec_id == AV_CODEC_ID_VP9)) { + st->codecpar->codec_id == AV_CODEC_ID_AV1 || st->codecpar->codec_id == AV_CODEC_ID_VP9 || + st->codecpar->codec_id == AV_CODEC_ID_VP8)) { AVDictionaryEntry *t; if (st->codecpar->extradata) { diff --git a/libavformat/flvenc.c b/libavformat/flvenc.c index f34df61c0e..32678e3136 100644 --- a/libavformat/flvenc.c +++ b/libavformat/flvenc.c @@ -54,6 +54,7 @@ static const AVCodecTag flv_video_codec_ids[] = { { AV_CODEC_ID_H264, FLV_CODECID_H264 }, { AV_CODEC_ID_HEVC, MKBETAG('h', 'v', 'c', '1') }, { AV_CODEC_ID_AV1, MKBETAG('a', 'v', '0', '1') }, + { AV_CODEC_ID_VP8, MKBETAG('v', 'p', '0', '8') }, { AV_CODEC_ID_VP9, MKBETAG('v', 'p', '0', '9') }, { AV_CODEC_ID_NONE, 0 } }; @@ -497,7 +498,7 @@ static void flv_write_metadata_packet(AVFormatContext *s, AVCodecParameters *par if (flv->metadata_pkt_written) return; if (par->codec_id == AV_CODEC_ID_HEVC || par->codec_id == AV_CODEC_ID_AV1 || - par->codec_id == AV_CODEC_ID_VP9) { + par->codec_id == AV_CODEC_ID_VP8 || par->codec_id == AV_CODEC_ID_VP9) { int flags_size = 5;