From patchwork Tue May 21 09:02:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timo Rothenpieler X-Patchwork-Id: 49088 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:3a48:b0:1af:fc2d:ff5a with SMTP id zu8csp5096953pzb; Tue, 21 May 2024 02:04:25 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCU/8Fp7gSrbRqyOR1HSxjlcCSUmJBwTiKUfyHhNzw0vfiknQSLCXjw1S7f/93cBpWQsgmqnF/LMdzVUj67JLiFjv31gD0b93Q+jzA== X-Google-Smtp-Source: AGHT+IGdZwBdCKF+K4hUJHx+jqUIxlg11a+DHEZL3e4cXNiNzh77FVLY3UPSg2Rt6jeEwSV+ar9M X-Received: by 2002:a19:5e55:0:b0:522:2990:7134 with SMTP id 2adb3069b0e04-52407ce92edmr2284728e87.33.1716282265290; Tue, 21 May 2024 02:04:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716282265; cv=none; d=google.com; s=arc-20160816; b=SOreZfaxfHrH9gzYmlQb8S7GhER9kpFnIRUmFPo0E2yAMEQ0bcNB3R+ZhCDm8unT82 rn6ax00K8OpbEgkZqeTNq5P4Piwypx1G57E3AMRS37ZrGpCYKxtUbWU1CTXRDmnLqBwS 5aKavlV0mPSqOf1DKbLPs/+3Tg0QpTtJAi9pWRFOvttaBK7LdhdgxnbN1OCLSykpAyxI zWj4WxUIcTmJ5spmN3SEdAc3fw6QEUYyM8pKd5A2F9GDCVFl5Ui5jGHoAm8adha//eB+ zV0bV5RwZQIwopsUpYHfR9DQamtJp4/Z4tpJQsgHulqPSP7eFe0t7O+S5frO6zF0IOwd NpHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=l3Odh+JNDmsQYmHeSkVFsIRlQ6l6l6M82ZsDh0edm1U=; fh=LnlYe9qYwgML7nWWXqAumr7YCmPjjpEPjQf6GasgJC0=; b=X8hR1NV+rGvnls6qN7c6xSbmNPkGJyo8/wsQtzPGlQyAD1c1fR6KxH/wm619a3yHfM EIk6y5rH+jEQq/2fcgsAn3Mnv9ALE/aE4UkCgfbisG/i1yFxQ1Hm7Qs5r4+b89Cp1O3d iZScGqygs9b0CnfYGeQtuBx+rn1dLwHTo90alOAqR9vSb4bou8YR9fbTi7LXfhjGZQOY XkkPbK59TgaPLFshtxcYuVqwxfmoMr9vKInURg1GCMQHsNh8zD/b1IkjWU9fAWvkn5pb Jtx5po7qzCmv3NfGSIgXCrdn3HEjTKzpQuq9nndtKa6cS0ypsalOPYeRIWA8vsqNNm8Y dUCA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=qJ4hXYdu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-521f38d416fsi8921193e87.246.2024.05.21.02.04.24; Tue, 21 May 2024 02:04:25 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=qJ4hXYdu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9C62F68D373; Tue, 21 May 2024 12:03:46 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from btbn.de (btbn.de [144.76.60.213]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4CBB068D2D4 for ; Tue, 21 May 2024 12:03:38 +0300 (EEST) Received: from [authenticated] by btbn.de (Postfix) with ESMTPSA id CBC9627FFD841; Tue, 21 May 2024 11:03:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rothenpieler.org; s=mail; t=1716282217; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QkVKMHFA1KKxEuPDOv/lSbYjcdELBSSDQQNtIZmMVFQ=; b=qJ4hXYduvLMob0p4SFL9nF06XeFW8gTrVsJjLZ5E1twroYY5q6VPt3ndur85keMVU+6BHT NB4lFtkTcnPrI/9xZ7mK7NNChdBEKO63bpCGV77xX9YHybk/QuPHg563dVJDjxJ9tdeIrf NQmW/s27l95NHZK2FWXlKrzfvOoG6B07Q6reTUyKIgUoeYuEM2GfS/bcz6rvPVHSAZ3nu9 hdZtmJlhxRB/kkCmz4gxljSDt6joeEXcXzLGdK8Jsc4BmzuuMtVT4zMlWtXAJskr2UIQSu hLUEGDST71uJNCbeYMOUl74h9dDNGNJHXcxcOeqmveUo7V7HSfkSIkxFE1Y5Fw== From: Timo Rothenpieler To: ffmpeg-devel@ffmpeg.org Date: Tue, 21 May 2024 11:02:12 +0200 Message-ID: <20240521090316.782-4-timo@rothenpieler.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240521090316.782-1-timo@rothenpieler.org> References: <20240521090316.782-1-timo@rothenpieler.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 03/13] avformat/flvenc: add enhanced audio codecs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Timo Rothenpieler Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Oy4ozGoVvEda --- libavformat/flv.h | 6 ++ libavformat/flvenc.c | 143 +++++++++++++++++++++++++++++++------------ 2 files changed, 111 insertions(+), 38 deletions(-) diff --git a/libavformat/flv.h b/libavformat/flv.h index 653c2bc82c..d030d576f3 100644 --- a/libavformat/flv.h +++ b/libavformat/flv.h @@ -103,6 +103,7 @@ enum { FLV_CODECID_NELLYMOSER = 6 << FLV_AUDIO_CODECID_OFFSET, FLV_CODECID_PCM_ALAW = 7 << FLV_AUDIO_CODECID_OFFSET, FLV_CODECID_PCM_MULAW = 8 << FLV_AUDIO_CODECID_OFFSET, + FLV_CODECID_EX_HEADER = 9 << FLV_AUDIO_CODECID_OFFSET, FLV_CODECID_AAC = 10<< FLV_AUDIO_CODECID_OFFSET, FLV_CODECID_SPEEX = 11<< FLV_AUDIO_CODECID_OFFSET, }; @@ -128,6 +129,11 @@ enum { PacketTypeMultitrack = 6, }; +enum { + AudioPacketTypeSequenceStart = 0, + AudioPacketTypeCodedFrames = 1, +}; + enum { MultitrackTypeOneTrack = 0x00, MultitrackTypeManyTracks = 0x10, diff --git a/libavformat/flvenc.c b/libavformat/flvenc.c index 7ddc59e369..4e79b06499 100644 --- a/libavformat/flvenc.c +++ b/libavformat/flvenc.c @@ -68,6 +68,10 @@ static const AVCodecTag flv_audio_codec_ids[] = { { AV_CODEC_ID_PCM_MULAW, FLV_CODECID_PCM_MULAW >> FLV_AUDIO_CODECID_OFFSET }, { AV_CODEC_ID_PCM_ALAW, FLV_CODECID_PCM_ALAW >> FLV_AUDIO_CODECID_OFFSET }, { AV_CODEC_ID_SPEEX, FLV_CODECID_SPEEX >> FLV_AUDIO_CODECID_OFFSET }, + { AV_CODEC_ID_OPUS, MKBETAG('O', 'p', 'u', 's') }, + { AV_CODEC_ID_FLAC, MKBETAG('f', 'L', 'a', 'C') }, + { AV_CODEC_ID_AC3, MKBETAG('a', 'c', '-', '3') }, + { AV_CODEC_ID_EAC3, MKBETAG('e', 'c', '-', '3') }, { AV_CODEC_ID_NONE, 0 } }; @@ -138,6 +142,9 @@ static int get_audio_flags(AVFormatContext *s, AVCodecParameters *par) if (par->codec_id == AV_CODEC_ID_AAC) // specs force these parameters return FLV_CODECID_AAC | FLV_SAMPLERATE_44100HZ | FLV_SAMPLESSIZE_16BIT | FLV_STEREO; + if (par->codec_id == AV_CODEC_ID_OPUS || par->codec_id == AV_CODEC_ID_FLAC + || par->codec_id == AV_CODEC_ID_AC3 || par->codec_id == AV_CODEC_ID_EAC3) + return FLV_CODECID_EX_HEADER; // only needed for codec support check else if (par->codec_id == AV_CODEC_ID_SPEEX) { if (par->sample_rate != 16000) { av_log(s, AV_LOG_ERROR, @@ -634,6 +641,42 @@ static int unsupported_codec(AVFormatContext *s, return AVERROR(ENOSYS); } +static void flv_write_aac_header(AVFormatContext* s, AVCodecParameters* par) +{ + AVIOContext *pb = s->pb; + FLVContext *flv = s->priv_data; + + if (!par->extradata_size && (flv->flags & FLV_AAC_SEQ_HEADER_DETECT)) { + PutBitContext pbc; + int samplerate_index; + int channels = par->ch_layout.nb_channels + - (par->ch_layout.nb_channels == 8 ? 1 : 0); + uint8_t data[2]; + + for (samplerate_index = 0; samplerate_index < 16; + samplerate_index++) + if (par->sample_rate + == ff_mpeg4audio_sample_rates[samplerate_index]) + break; + + init_put_bits(&pbc, data, sizeof(data)); + put_bits(&pbc, 5, par->profile + 1); //profile + put_bits(&pbc, 4, samplerate_index); //sample rate index + put_bits(&pbc, 4, channels); + put_bits(&pbc, 1, 0); //frame length - 1024 samples + put_bits(&pbc, 1, 0); //does not depend on core coder + put_bits(&pbc, 1, 0); //is not extension + flush_put_bits(&pbc); + + avio_w8(pb, data[0]); + avio_w8(pb, data[1]); + + av_log(s, AV_LOG_WARNING, "AAC sequence header: %02x %02x.\n", + data[0], data[1]); + } + avio_write(pb, par->extradata, par->extradata_size); +} + static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, int64_t ts, int stream_index) { int64_t data_size; AVIOContext *pb = s->pb; @@ -641,7 +684,9 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i if (par->codec_id == AV_CODEC_ID_AAC || par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_MPEG4 || par->codec_id == AV_CODEC_ID_HEVC - || par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9) { + || par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9 + || par->codec_id == AV_CODEC_ID_OPUS || par->codec_id == AV_CODEC_ID_FLAC + || par->codec_id == AV_CODEC_ID_AC3 || par->codec_id == AV_CODEC_ID_EAC3) { int64_t pos; avio_w8(pb, par->codec_type == AVMEDIA_TYPE_VIDEO ? @@ -650,39 +695,38 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i put_timestamp(pb, ts); avio_wb24(pb, 0); // streamid pos = avio_tell(pb); - if (par->codec_id == AV_CODEC_ID_AAC) { - avio_w8(pb, get_audio_flags(s, par)); - avio_w8(pb, 0); // AAC sequence header - - if (!par->extradata_size && (flv->flags & FLV_AAC_SEQ_HEADER_DETECT)) { - PutBitContext pbc; - int samplerate_index; - int channels = par->ch_layout.nb_channels - - (par->ch_layout.nb_channels == 8 ? 1 : 0); - uint8_t data[2]; - - for (samplerate_index = 0; samplerate_index < 16; - samplerate_index++) - if (par->sample_rate - == ff_mpeg4audio_sample_rates[samplerate_index]) - break; - - init_put_bits(&pbc, data, sizeof(data)); - put_bits(&pbc, 5, par->profile + 1); //profile - put_bits(&pbc, 4, samplerate_index); //sample rate index - put_bits(&pbc, 4, channels); - put_bits(&pbc, 1, 0); //frame length - 1024 samples - put_bits(&pbc, 1, 0); //does not depend on core coder - put_bits(&pbc, 1, 0); //is not extension - flush_put_bits(&pbc); - - avio_w8(pb, data[0]); - avio_w8(pb, data[1]); - - av_log(s, AV_LOG_WARNING, "AAC sequence header: %02x %02x.\n", - data[0], data[1]); + if (par->codec_type == AVMEDIA_TYPE_AUDIO) { + int extended_flv = par->codec_id == AV_CODEC_ID_OPUS + || par->codec_id == AV_CODEC_ID_FLAC + || par->codec_id == AV_CODEC_ID_AC3 + || par->codec_id == AV_CODEC_ID_EAC3; + + if (extended_flv) { + avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeSequenceStart); + + if (par->codec_id == AV_CODEC_ID_AAC) { + avio_write(pb, "mp4a", 4); + flv_write_aac_header(s, par); + } else if (par->codec_id == AV_CODEC_ID_OPUS) { + avio_write(pb, "Opus", 4); + av_assert0(par->extradata_size); + avio_write(pb, par->extradata, par->extradata_size); + } else if (par->codec_id == AV_CODEC_ID_FLAC) { + avio_write(pb, "fLaC", 4); + av_assert0(par->extradata_size); + avio_write(pb, par->extradata, par->extradata_size); + } else if (par->codec_id == AV_CODEC_ID_MP3) + avio_write(pb, ".mp3", 4); + else if (par->codec_id == AV_CODEC_ID_AC3) + avio_write(pb, "ac-3", 4); + else if (par->codec_id == AV_CODEC_ID_EAC3) + avio_write(pb, "ec-3", 4); + } else if (par->codec_id == AV_CODEC_ID_AAC) { + avio_w8(pb, get_audio_flags(s, par)); + avio_w8(pb, 0); // AAC sequence header + + flv_write_aac_header(s, par); } - avio_write(pb, par->extradata, par->extradata_size); } else { int track_idx = flv->video_track_idx_map[stream_index]; // If video stream has track_idx > 0 we need to send H.264 as extended video packet @@ -1018,13 +1062,20 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt) int64_t cur_offset = avio_tell(pb); int track_idx = flv->video_track_idx_map[pkt->stream_index]; + int extended_audio = par->codec_id == AV_CODEC_ID_OPUS + || par->codec_id == AV_CODEC_ID_FLAC + || par->codec_id == AV_CODEC_ID_AC3 + || par->codec_id == AV_CODEC_ID_EAC3; + if (par->codec_type == AVMEDIA_TYPE_AUDIO && !pkt->size) { av_log(s, AV_LOG_WARNING, "Empty audio Packet\n"); return AVERROR(EINVAL); } - if (par->codec_id == AV_CODEC_ID_VP6F || par->codec_id == AV_CODEC_ID_VP6A || - par->codec_id == AV_CODEC_ID_VP6 || par->codec_id == AV_CODEC_ID_AAC) + if (extended_audio) + flags_size = 5; + else if (par->codec_id == AV_CODEC_ID_VP6F || par->codec_id == AV_CODEC_ID_VP6A || + par->codec_id == AV_CODEC_ID_VP6 || par->codec_id == AV_CODEC_ID_AAC) flags_size = 2; else if (par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_MPEG4 || par->codec_id == AV_CODEC_ID_HEVC || par->codec_id == AV_CODEC_ID_AV1 || @@ -1043,7 +1094,8 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt) if (par->codec_id == AV_CODEC_ID_AAC || par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_MPEG4 || par->codec_id == AV_CODEC_ID_HEVC - || par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9) { + || par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9 + || par->codec_id == AV_CODEC_ID_OPUS || par->codec_id == AV_CODEC_ID_FLAC) { size_t side_size; uint8_t *side = av_packet_get_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, &side_size); if (side && side_size > 0 && (side_size != par->extradata_size || memcmp(side, par->extradata, side_size))) { @@ -1175,12 +1227,12 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt) avio_seek(pb, data_size + 10 - 3, SEEK_CUR); avio_wb32(pb, data_size + 11); } else { - int extended_flv = (par->codec_id == AV_CODEC_ID_H264 && track_idx) + int extended_video = (par->codec_id == AV_CODEC_ID_H264 && track_idx) || par->codec_id == AV_CODEC_ID_HEVC || par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9; - if (extended_flv) { + if (extended_video) { int h2645 = par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_HEVC; int pkttype = PacketTypeCodedFrames; @@ -1208,6 +1260,21 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt) avio_w8(pb, track_idx); if (h2645 && pkttype == PacketTypeCodedFrames) avio_wb24(pb, pkt->pts - pkt->dts); + } else if (extended_audio) { + avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeCodedFrames); + + if (par->codec_id == AV_CODEC_ID_AAC) + avio_write(pb, "mp4a", 4); + else if (par->codec_id == AV_CODEC_ID_OPUS) + avio_write(pb, "Opus", 4); + else if (par->codec_id == AV_CODEC_ID_FLAC) + avio_write(pb, "fLaC", 4); + else if (par->codec_id == AV_CODEC_ID_MP3) + avio_write(pb, ".mp3", 4); + else if (par->codec_id == AV_CODEC_ID_AC3) + avio_write(pb, "ac-3", 4); + else if (par->codec_id == AV_CODEC_ID_EAC3) + avio_write(pb, "ec-3", 4); } else { av_assert1(flags>=0); avio_w8(pb, flags);