From patchwork Wed May 22 18:27:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timo Rothenpieler X-Patchwork-Id: 49152 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:542:0:b0:460:55fa:d5ed with SMTP id 63csp563653vqf; Wed, 22 May 2024 11:28:07 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCX5+KgVK0p95DYvyx1OhV3KR2JFCOj6WWn8jqbzcL3Lxns3GiEZTb/bjGQhKuRCR+z9Zk6vUhewdQp0EcQTRRnGLZqlsZqLs090Qg== X-Google-Smtp-Source: AGHT+IEffSgwX5/kuE+O0Z1I1Qd7gz+9EtyRJgXgVoNem/snuwrvHEkay72wo50j9+0BANVQilPN X-Received: by 2002:a50:c192:0:b0:572:7bda:1709 with SMTP id 4fb4d7f45d1cf-578329a292fmr1815179a12.9.1716402487281; Wed, 22 May 2024 11:28:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716402487; cv=none; d=google.com; s=arc-20160816; b=itqVDOzmFFpNhAqfREwHKOsS8DArryu4Bt/oxu2kntQkpJJW6s6ycJYDmInw09/IQa nE4dq+OrtmQswArovnKJU7n+xlHn8zAGna4Gopqb05w0ib1m/HPDzwbAQaAiLnhbXBeB CZmKDoRoHAMoXjcoDQvE8tUkSvvKczxGJ+6Z6q8KSVBZEf5iuG+JYquD8767qq3gLv2A RVv8l0krLlJevr37d6ofC53J7hkqKOdnDC/5j50+Dewk5yMKG9MFmB4Ar28kQnFV89Z8 eg1IBlKPXo3itmCxrK+E0Bl9ag0qNMDKTIwaYN5muR6XY/7wxmBENMkYmmSOoxI+tXqy Hgnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=Qu/o9mVd66togPL2TeINPCc8LsKobpBT9+5xAOGc2tk=; fh=LnlYe9qYwgML7nWWXqAumr7YCmPjjpEPjQf6GasgJC0=; b=upUKwxQgAgxhWvFYP6xILDX6wTozfuY4LSTCjxi7ZcvBgZK7iJgRwmxVN+l5bYxwvz 5nQ8jpC673ejIibbhb7SodwYhbuS0eLKeqqLeZbRQeeHbQLHVwVDeMwVUHxsgnA96j4N p2OaYg18WzpCxbsaslhK/xie5keedoxnvu7z0xag4n6UadvVQY2/L3cLHfkKScG3NfSE m2GRpC8Wy/R+L0TeZXekKB+kHbAcHe70F6GdZFR55fB6CBcs81X0sL+Tvl8C/uyZtO3e 24xsSXYH1L8ztj10bDCvRQWLUhS9XxX5GEp2Eq8QzzR3bNFs5BdpRPp2yi9Wi7JVhbh4 /Ggw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b="SZoEB/tu"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5733beacb66si14943378a12.10.2024.05.22.11.28.06; Wed, 22 May 2024 11:28:07 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b="SZoEB/tu"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6EEE268D27C; Wed, 22 May 2024 21:28:04 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from btbn.de (btbn.de [144.76.60.213]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5D08E68D109 for ; Wed, 22 May 2024 21:27:57 +0300 (EEST) Received: from [authenticated] by btbn.de (Postfix) with ESMTPSA id B473527FFD90C; Wed, 22 May 2024 20:27:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rothenpieler.org; s=mail; t=1716402476; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HbmBwXBS2PV/9+A27W+Sqi1A2TpCNNtDNGD55k1g2o0=; b=SZoEB/tur/NiYW+M5mgfBnWOuubNRV5XR0JJybIgBqtOrNfVu5Pimm+FTocg6x/Sh1y53b LyqIBerEggD1GLBTNajvKMc/XP/BpQoQpsfiG8RO3fjmMaI7Ua6gUhh6suVsFXiquB/Uy+ sXC0fSmQ47oqjusArI9oorM7gYsf/XUduMUHceuboRwCMGBRofDLV451lIdH1bHt1juvr8 mmFs5JL1LRIqsFZ0cr43a7nkV9Yo1tw+XnUa+MUICPKKydA9stWJnuybYZ436/jos1LJum 6+GD6b28BM3PdmDDGjPt5YnoBuT+fT/JHcCgupGVYL8EDk2tKFMGToAyCAwhbg== From: Timo Rothenpieler To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 May 2024 20:27:39 +0200 Message-ID: <20240522182750.17340-1-timo@rothenpieler.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240521090316.782-14-timo@rothenpieler.org> References: <20240521090316.782-14-timo@rothenpieler.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 13/13] avformat/flvdec: support all multi-track modes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Timo Rothenpieler Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: SWcUt4Xwqppq --- libavformat/flvdec.c | 574 +++++++++++++++++++++++-------------------- 1 file changed, 311 insertions(+), 263 deletions(-) diff --git a/libavformat/flvdec.c b/libavformat/flvdec.c index 4f98ff348c..fe5a44a715 100644 --- a/libavformat/flvdec.c +++ b/libavformat/flvdec.c @@ -1273,6 +1273,7 @@ static int flv_read_packet(AVFormatContext *s, AVPacket *pkt) { FLVContext *flv = s->priv_data; int ret, i, size, flags; + int res = 0; enum FlvTagType type; int stream_type=-1; int64_t next, pos, meta_pos; @@ -1287,6 +1288,7 @@ static int flv_read_packet(AVFormatContext *s, AVPacket *pkt) int pkt_type = 0; uint8_t track_idx = 0; uint32_t codec_id = 0; + int multitrack_type = MultitrackTypeOneTrack; retry: /* pkt size is repeated at end. skip it */ @@ -1337,14 +1339,9 @@ retry: if (pkt_type == AudioPacketTypeMultitrack) { uint8_t types = avio_r8(s->pb); - int multitrack_type = types >> 4; + multitrack_type = types & 0xF0; pkt_type = types & 0xF; - if (multitrack_type != MultitrackTypeOneTrack) { - av_log(s, AV_LOG_ERROR, "Audio multitrack types other than MultitrackTypeOneTrack are unsupported!\n"); - return AVERROR_PATCHWELCOME; - } - multitrack = 1; size--; } @@ -1371,14 +1368,9 @@ retry: if (pkt_type == PacketTypeMultitrack) { uint8_t types = avio_r8(s->pb); - int multitrack_type = types >> 4; + multitrack_type = types & 0xF0; pkt_type = types & 0xF; - if (multitrack_type != MultitrackTypeOneTrack) { - av_log(s, AV_LOG_ERROR, "Multitrack types other than MultitrackTypeOneTrack are unsupported!\n"); - return AVERROR_PATCHWELCOME; - } - multitrack = 1; size--; } @@ -1447,293 +1439,349 @@ skip: goto leave; } - /* now find stream */ - for (i = 0; i < s->nb_streams; i++) { - st = s->streams[i]; - if (stream_type == FLV_STREAM_TYPE_AUDIO) { - if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO && - (s->audio_codec_id || flv_same_audio_codec(st->codecpar, flags, codec_id)) && - st->id == track_idx) - break; - } else if (stream_type == FLV_STREAM_TYPE_VIDEO) { - if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO && - (s->video_codec_id || flv_same_video_codec(st->codecpar, codec_id)) && - st->id == track_idx) - break; - } else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) { - if (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) - break; - } else if (stream_type == FLV_STREAM_TYPE_DATA) { - if (st->codecpar->codec_type == AVMEDIA_TYPE_DATA) - break; + for (;;) { + int track_size = size; + + if (multitrack_type != MultitrackTypeOneTrack) { + track_size = avio_rb24(s->pb); + size -= 3; } - } - if (i == s->nb_streams) { - static const enum AVMediaType stream_types[] = {AVMEDIA_TYPE_VIDEO, AVMEDIA_TYPE_AUDIO, AVMEDIA_TYPE_SUBTITLE, AVMEDIA_TYPE_DATA}; - st = create_stream(s, stream_types[stream_type], track_idx); - if (!st) - return AVERROR(ENOMEM); - } - av_log(s, AV_LOG_TRACE, "%d %X %d \n", stream_type, flags, st->discard); - if (flv->time_pos <= pos) { - dts += flv->time_offset; - } + /* now find stream */ + for (i = 0; i < s->nb_streams; i++) { + st = s->streams[i]; + if (stream_type == FLV_STREAM_TYPE_AUDIO) { + if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO && + (s->audio_codec_id || flv_same_audio_codec(st->codecpar, flags, codec_id)) && + st->id == track_idx) + break; + } else if (stream_type == FLV_STREAM_TYPE_VIDEO) { + if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO && + (s->video_codec_id || flv_same_video_codec(st->codecpar, codec_id)) && + st->id == track_idx) + break; + } else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) { + if (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) + break; + } else if (stream_type == FLV_STREAM_TYPE_DATA) { + if (st->codecpar->codec_type == AVMEDIA_TYPE_DATA) + break; + } + } + if (i == s->nb_streams) { + static const enum AVMediaType stream_types[] = {AVMEDIA_TYPE_VIDEO, AVMEDIA_TYPE_AUDIO, AVMEDIA_TYPE_SUBTITLE, AVMEDIA_TYPE_DATA}; + st = create_stream(s, stream_types[stream_type], track_idx); + if (!st) + return AVERROR(ENOMEM); + } + av_log(s, AV_LOG_TRACE, "%d %X %d \n", stream_type, flags, st->discard); - if ((s->pb->seekable & AVIO_SEEKABLE_NORMAL) && - ((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY || - stream_type == FLV_STREAM_TYPE_AUDIO)) - av_add_index_entry(st, pos, dts, size, 0, AVINDEX_KEYFRAME); + if (flv->time_pos <= pos) { + dts += flv->time_offset; + } - if ((st->discard >= AVDISCARD_NONKEY && !((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY || stream_type == FLV_STREAM_TYPE_AUDIO)) || - (st->discard >= AVDISCARD_BIDIR && ((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_DISP_INTER && stream_type == FLV_STREAM_TYPE_VIDEO)) || - st->discard >= AVDISCARD_ALL) { - avio_seek(s->pb, next, SEEK_SET); - ret = FFERROR_REDO; - goto leave; - } + if ((s->pb->seekable & AVIO_SEEKABLE_NORMAL) && + ((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY || + stream_type == FLV_STREAM_TYPE_AUDIO)) + av_add_index_entry(st, pos, dts, track_size, 0, AVINDEX_KEYFRAME); + + if ((st->discard >= AVDISCARD_NONKEY && !((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY || stream_type == FLV_STREAM_TYPE_AUDIO)) || + (st->discard >= AVDISCARD_BIDIR && ((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_DISP_INTER && stream_type == FLV_STREAM_TYPE_VIDEO)) || + st->discard >= AVDISCARD_ALL) { + avio_seek(s->pb, next, SEEK_SET); + ret = FFERROR_REDO; + goto leave; + } - // if not streamed and no duration from metadata then seek to end to find - // the duration from the timestamps - if ((s->pb->seekable & AVIO_SEEKABLE_NORMAL) && - (!s->duration || s->duration == AV_NOPTS_VALUE) && - !flv->searched_for_end) { - int final_size; - const int64_t pos = avio_tell(s->pb); - // Read the last 4 bytes of the file, this should be the size of the - // previous FLV tag. Use the timestamp of its payload as duration. - int64_t fsize = avio_size(s->pb); + // if not streamed and no duration from metadata then seek to end to find + // the duration from the timestamps + if ((s->pb->seekable & AVIO_SEEKABLE_NORMAL) && + (!s->duration || s->duration == AV_NOPTS_VALUE) && + !flv->searched_for_end) { + int final_size; + const int64_t pos = avio_tell(s->pb); + // Read the last 4 bytes of the file, this should be the size of the + // previous FLV tag. Use the timestamp of its payload as duration. + int64_t fsize = avio_size(s->pb); retry_duration: - avio_seek(s->pb, fsize - 4, SEEK_SET); - final_size = avio_rb32(s->pb); - if (final_size > 0 && final_size < fsize) { - // Seek to the start of the last FLV tag at position (fsize - 4 - final_size) - // but skip the byte indicating the type. - avio_seek(s->pb, fsize - 3 - final_size, SEEK_SET); - if (final_size == avio_rb24(s->pb) + 11) { - uint32_t ts = avio_rb24(s->pb); - ts |= (unsigned)avio_r8(s->pb) << 24; - if (ts) - s->duration = ts * (int64_t)AV_TIME_BASE / 1000; - else if (fsize >= 8 && fsize - 8 >= final_size) { - fsize -= final_size+4; - goto retry_duration; + avio_seek(s->pb, fsize - 4, SEEK_SET); + final_size = avio_rb32(s->pb); + if (final_size > 0 && final_size < fsize) { + // Seek to the start of the last FLV tag at position (fsize - 4 - final_size) + // but skip the byte indicating the type. + avio_seek(s->pb, fsize - 3 - final_size, SEEK_SET); + if (final_size == avio_rb24(s->pb) + 11) { + uint32_t ts = avio_rb24(s->pb); + ts |= (unsigned)avio_r8(s->pb) << 24; + if (ts) + s->duration = ts * (int64_t)AV_TIME_BASE / 1000; + else if (fsize >= 8 && fsize - 8 >= final_size) { + fsize -= final_size+4; + goto retry_duration; + } } } + + avio_seek(s->pb, pos, SEEK_SET); + flv->searched_for_end = 1; } - avio_seek(s->pb, pos, SEEK_SET); - flv->searched_for_end = 1; - } + if (stream_type == FLV_STREAM_TYPE_AUDIO && !enhanced_flv) { + int bits_per_coded_sample; + channels = (flags & FLV_AUDIO_CHANNEL_MASK) == FLV_STEREO ? 2 : 1; + sample_rate = 44100 << ((flags & FLV_AUDIO_SAMPLERATE_MASK) >> + FLV_AUDIO_SAMPLERATE_OFFSET) >> 3; + bits_per_coded_sample = (flags & FLV_AUDIO_SAMPLESIZE_MASK) ? 16 : 8; + if (!av_channel_layout_check(&st->codecpar->ch_layout) || + !st->codecpar->sample_rate || + !st->codecpar->bits_per_coded_sample) { + av_channel_layout_default(&st->codecpar->ch_layout, channels); + st->codecpar->sample_rate = sample_rate; + st->codecpar->bits_per_coded_sample = bits_per_coded_sample; + } + if (!st->codecpar->codec_id) { + flv_set_audio_codec(s, st, st->codecpar, + flags & FLV_AUDIO_CODECID_MASK); + flv->last_sample_rate = + sample_rate = st->codecpar->sample_rate; + flv->last_channels = + channels = st->codecpar->ch_layout.nb_channels; + } else { + AVCodecParameters *par = avcodec_parameters_alloc(); + if (!par) { + ret = AVERROR(ENOMEM); + goto leave; + } + par->sample_rate = sample_rate; + par->bits_per_coded_sample = bits_per_coded_sample; + flv_set_audio_codec(s, st, par, flags & FLV_AUDIO_CODECID_MASK); + sample_rate = par->sample_rate; + avcodec_parameters_free(&par); + } + } else if (stream_type == FLV_STREAM_TYPE_AUDIO) { + if (!st->codecpar->codec_id) + flv_set_audio_codec(s, st, st->codecpar, + codec_id ? codec_id : (flags & FLV_AUDIO_CODECID_MASK)); + + // These are not signalled in the flags anymore + channels = 0; + sample_rate = 0; + + if (pkt_type == AudioPacketTypeMultichannelConfig) { + int channel_order = avio_r8(s->pb); + channels = avio_r8(s->pb); + size -= 2; + track_size -= 2; + + av_channel_layout_uninit(&st->codecpar->ch_layout); + + if (channel_order == AudioChannelOrderCustom) { + ret = av_channel_layout_custom_init(&st->codecpar->ch_layout, channels); + if (ret < 0) + return ret; + + for (i = 0; i < channels; i++) { + uint8_t id = avio_r8(s->pb); + size--; + + if (id < 18) + st->codecpar->ch_layout.u.map[i].id = id; + else if (id >= 18 && id <= 23) + st->codecpar->ch_layout.u.map[i].id = id - 18 + AV_CHAN_LOW_FREQUENCY_2; + else if (id == 0xFE) + st->codecpar->ch_layout.u.map[i].id = AV_CHAN_UNUSED; + else + st->codecpar->ch_layout.u.map[i].id = AV_CHAN_UNKNOWN; + } + } else if (channel_order == AudioChannelOrderNative) { + uint64_t mask = avio_rb32(s->pb); + size -= 4; + track_size -= 4; + + // The first 18 entries in the mask match ours, but the remaining 6 entries start at AV_CHAN_LOW_FREQUENCY_2 + mask = (mask & 0x3FFFF) | ((mask & 0xFC0000) << (AV_CHAN_LOW_FREQUENCY_2 - 18)); + ret = av_channel_layout_from_mask(&st->codecpar->ch_layout, mask); + if (ret < 0) + return ret; + } else { + av_channel_layout_default(&st->codecpar->ch_layout, channels); + } + + av_log(s, AV_LOG_DEBUG, "Set channel data from MultiChannel info.\n"); - if (stream_type == FLV_STREAM_TYPE_AUDIO && !enhanced_flv) { - int bits_per_coded_sample; - channels = (flags & FLV_AUDIO_CHANNEL_MASK) == FLV_STEREO ? 2 : 1; - sample_rate = 44100 << ((flags & FLV_AUDIO_SAMPLERATE_MASK) >> - FLV_AUDIO_SAMPLERATE_OFFSET) >> 3; - bits_per_coded_sample = (flags & FLV_AUDIO_SAMPLESIZE_MASK) ? 16 : 8; - if (!av_channel_layout_check(&st->codecpar->ch_layout) || - !st->codecpar->sample_rate || - !st->codecpar->bits_per_coded_sample) { - av_channel_layout_default(&st->codecpar->ch_layout, channels); - st->codecpar->sample_rate = sample_rate; - st->codecpar->bits_per_coded_sample = bits_per_coded_sample; - } - if (!st->codecpar->codec_id) { - flv_set_audio_codec(s, st, st->codecpar, - flags & FLV_AUDIO_CODECID_MASK); - flv->last_sample_rate = - sample_rate = st->codecpar->sample_rate; - flv->last_channels = - channels = st->codecpar->ch_layout.nb_channels; - } else { - AVCodecParameters *par = avcodec_parameters_alloc(); - if (!par) { - ret = AVERROR(ENOMEM); goto leave; } - par->sample_rate = sample_rate; - par->bits_per_coded_sample = bits_per_coded_sample; - flv_set_audio_codec(s, st, par, flags & FLV_AUDIO_CODECID_MASK); - sample_rate = par->sample_rate; - avcodec_parameters_free(&par); + } else if (stream_type == FLV_STREAM_TYPE_VIDEO) { + int sret = flv_set_video_codec(s, st, codec_id, 1); + if (sret < 0) + return sret; + size -= sret; + track_size -= sret; + } else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) { + st->codecpar->codec_id = AV_CODEC_ID_TEXT; + } else if (stream_type == FLV_STREAM_TYPE_DATA) { + st->codecpar->codec_id = AV_CODEC_ID_NONE; // Opaque AMF data } - } else if (stream_type == FLV_STREAM_TYPE_AUDIO) { - if (!st->codecpar->codec_id) - flv_set_audio_codec(s, st, st->codecpar, - codec_id ? codec_id : (flags & FLV_AUDIO_CODECID_MASK)); - - // These are not signalled in the flags anymore - channels = 0; - sample_rate = 0; - if (pkt_type == AudioPacketTypeMultichannelConfig) { - int channel_order = avio_r8(s->pb); - channels = avio_r8(s->pb); - size -= 2; + if (st->codecpar->codec_id == AV_CODEC_ID_AAC || + st->codecpar->codec_id == AV_CODEC_ID_OPUS || + st->codecpar->codec_id == AV_CODEC_ID_FLAC || + st->codecpar->codec_id == AV_CODEC_ID_H264 || + st->codecpar->codec_id == AV_CODEC_ID_MPEG4 || + st->codecpar->codec_id == AV_CODEC_ID_HEVC || + st->codecpar->codec_id == AV_CODEC_ID_AV1 || + st->codecpar->codec_id == AV_CODEC_ID_VP9) { + int type = 0; + if (enhanced_flv) { + type = pkt_type; + } else { + type = avio_r8(s->pb); + size--; + track_size--; + } - av_channel_layout_uninit(&st->codecpar->ch_layout); + if (size < 0 || track_size < 0) { + ret = AVERROR_INVALIDDATA; + goto leave; + } - if (channel_order == AudioChannelOrderCustom) { - ret = av_channel_layout_custom_init(&st->codecpar->ch_layout, channels); - if (ret < 0) - return ret; + if (enhanced_flv && stream_type == FLV_STREAM_TYPE_VIDEO && flv->meta_color_info_flag) { + flv_update_video_color_info(s, st); // update av packet side data + flv->meta_color_info_flag = 0; + } - for (i = 0; i < channels; i++) { - uint8_t id = avio_r8(s->pb); - size--; - - if (id < 18) - st->codecpar->ch_layout.u.map[i].id = id; - else if (id >= 18 && id <= 23) - st->codecpar->ch_layout.u.map[i].id = id - 18 + AV_CHAN_LOW_FREQUENCY_2; - else if (id == 0xFE) - st->codecpar->ch_layout.u.map[i].id = AV_CHAN_UNUSED; - else - st->codecpar->ch_layout.u.map[i].id = AV_CHAN_UNKNOWN; + if (st->codecpar->codec_id == AV_CODEC_ID_MPEG4 || + (st->codecpar->codec_id == AV_CODEC_ID_H264 && (!enhanced_flv || type == PacketTypeCodedFrames)) || + (st->codecpar->codec_id == AV_CODEC_ID_HEVC && type == PacketTypeCodedFrames)) { + // sign extension + int32_t cts = (avio_rb24(s->pb) + 0xff800000) ^ 0xff800000; + pts = av_sat_add64(dts, cts); + if (cts < 0) { // dts might be wrong + if (!flv->wrong_dts) + av_log(s, AV_LOG_WARNING, + "Negative cts, previous timestamps might be wrong.\n"); + flv->wrong_dts = 1; + } else if (FFABS(dts - pts) > 1000*60*15) { + av_log(s, AV_LOG_WARNING, + "invalid timestamps %"PRId64" %"PRId64"\n", dts, pts); + dts = pts = AV_NOPTS_VALUE; } - } else if (channel_order == AudioChannelOrderNative) { - uint64_t mask = avio_rb32(s->pb); - size -= 4; - - // The first 18 entries in the mask match ours, but the remaining 6 entries start at AV_CHAN_LOW_FREQUENCY_2 - mask = (mask & 0x3FFFF) | ((mask & 0xFC0000) << (AV_CHAN_LOW_FREQUENCY_2 - 18)); - ret = av_channel_layout_from_mask(&st->codecpar->ch_layout, mask); - if (ret < 0) - return ret; - } else { - av_channel_layout_default(&st->codecpar->ch_layout, channels); + size -= 3; + track_size -= 3; } + if (type == 0 && (!st->codecpar->extradata || st->codecpar->codec_id == AV_CODEC_ID_AAC || + st->codecpar->codec_id == AV_CODEC_ID_OPUS || st->codecpar->codec_id == AV_CODEC_ID_FLAC || + st->codecpar->codec_id == AV_CODEC_ID_H264 || st->codecpar->codec_id == AV_CODEC_ID_HEVC || + st->codecpar->codec_id == AV_CODEC_ID_AV1 || st->codecpar->codec_id == AV_CODEC_ID_VP9)) { + AVDictionaryEntry *t; + + if (st->codecpar->extradata) { + if ((ret = flv_queue_extradata(flv, s->pb, multitrack ? track_idx : stream_type, track_size, multitrack)) < 0) + return ret; + ret = FFERROR_REDO; + goto leave; + } + if ((ret = flv_get_extradata(s, st, track_size)) < 0) + return ret; - av_log(s, AV_LOG_DEBUG, "Set channel data from MultiChannel info.\n"); - - goto leave; - } - } else if (stream_type == FLV_STREAM_TYPE_VIDEO) { - int sret = flv_set_video_codec(s, st, codec_id, 1); - if (sret < 0) - return sret; - size -= sret; - } else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) { - st->codecpar->codec_id = AV_CODEC_ID_TEXT; - } else if (stream_type == FLV_STREAM_TYPE_DATA) { - st->codecpar->codec_id = AV_CODEC_ID_NONE; // Opaque AMF data - } + /* Workaround for buggy Omnia A/XE encoder */ + t = av_dict_get(s->metadata, "Encoder", NULL, 0); + if (st->codecpar->codec_id == AV_CODEC_ID_AAC && t && !strcmp(t->value, "Omnia A/XE")) + st->codecpar->extradata_size = 2; - if (st->codecpar->codec_id == AV_CODEC_ID_AAC || - st->codecpar->codec_id == AV_CODEC_ID_OPUS || - st->codecpar->codec_id == AV_CODEC_ID_FLAC || - st->codecpar->codec_id == AV_CODEC_ID_H264 || - st->codecpar->codec_id == AV_CODEC_ID_MPEG4 || - st->codecpar->codec_id == AV_CODEC_ID_HEVC || - st->codecpar->codec_id == AV_CODEC_ID_AV1 || - st->codecpar->codec_id == AV_CODEC_ID_VP9) { - int type = 0; - if (enhanced_flv) { - type = pkt_type; - } else { - type = avio_r8(s->pb); - size--; + ret = FFERROR_REDO; + goto leave; + } } - if (size < 0) { - ret = AVERROR_INVALIDDATA; + /* skip empty or broken data packets */ + if (size <= 0 || track_size < 0) { + ret = FFERROR_REDO; goto leave; } - if (enhanced_flv && stream_type == FLV_STREAM_TYPE_VIDEO && flv->meta_color_info_flag) { - flv_update_video_color_info(s, st); // update av packet side data - flv->meta_color_info_flag = 0; + /* skip empty data track */ + if (!track_size) + goto next_track; + + ret = av_get_packet(s->pb, pkt, track_size); + if (ret < 0) + return ret; + + track_size -= ret; + size -= ret; + + pkt->dts = dts; + pkt->pts = pts == AV_NOPTS_VALUE ? dts : pts; + pkt->stream_index = st->index; + pkt->pos = pos; + if (!multitrack && flv->new_extradata[stream_type]) { + ret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, + flv->new_extradata[stream_type], + flv->new_extradata_size[stream_type]); + if (ret >= 0) { + flv->new_extradata[stream_type] = NULL; + flv->new_extradata_size[stream_type] = 0; + } + } else if (multitrack + && flv->mt_extradata_cnt > track_idx + && flv->mt_extradata[track_idx]) { + ret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, + flv->mt_extradata[track_idx], + flv->mt_extradata_sz[track_idx]); + if (ret >= 0) { + flv->mt_extradata[track_idx] = NULL; + flv->mt_extradata_sz[track_idx] = 0; + } + } + if (stream_type == FLV_STREAM_TYPE_AUDIO && !enhanced_flv && + (sample_rate != flv->last_sample_rate || + channels != flv->last_channels)) { + flv->last_sample_rate = sample_rate; + flv->last_channels = channels; + ff_add_param_change(pkt, channels, 0, sample_rate, 0, 0); } - if (st->codecpar->codec_id == AV_CODEC_ID_MPEG4 || - (st->codecpar->codec_id == AV_CODEC_ID_H264 && (!enhanced_flv || type == PacketTypeCodedFrames)) || - (st->codecpar->codec_id == AV_CODEC_ID_HEVC && type == PacketTypeCodedFrames)) { - // sign extension - int32_t cts = (avio_rb24(s->pb) + 0xff800000) ^ 0xff800000; - pts = av_sat_add64(dts, cts); - if (cts < 0) { // dts might be wrong - if (!flv->wrong_dts) - av_log(s, AV_LOG_WARNING, - "Negative cts, previous timestamps might be wrong.\n"); - flv->wrong_dts = 1; - } else if (FFABS(dts - pts) > 1000*60*15) { - av_log(s, AV_LOG_WARNING, - "invalid timestamps %"PRId64" %"PRId64"\n", dts, pts); - dts = pts = AV_NOPTS_VALUE; - } - size -= 3; + if (stream_type == FLV_STREAM_TYPE_AUDIO || + (flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY || + stream_type == FLV_STREAM_TYPE_SUBTITLE || + stream_type == FLV_STREAM_TYPE_DATA) + pkt->flags |= AV_PKT_FLAG_KEY; + + ret = ff_buffer_packet(s, pkt); + if (ret < 0) + return ret; + res = FFERROR_REDO; + + if (track_size) { + av_log(s, AV_LOG_WARNING, "Track size mismatch: %d!\n", track_size); + avio_skip(s->pb, track_size); + size -= track_size; } - if (type == 0 && (!st->codecpar->extradata || st->codecpar->codec_id == AV_CODEC_ID_AAC || - st->codecpar->codec_id == AV_CODEC_ID_OPUS || st->codecpar->codec_id == AV_CODEC_ID_FLAC || - st->codecpar->codec_id == AV_CODEC_ID_H264 || st->codecpar->codec_id == AV_CODEC_ID_HEVC || - st->codecpar->codec_id == AV_CODEC_ID_AV1 || st->codecpar->codec_id == AV_CODEC_ID_VP9)) { - AVDictionaryEntry *t; - - if (st->codecpar->extradata) { - if ((ret = flv_queue_extradata(flv, s->pb, multitrack ? track_idx : stream_type, size, multitrack)) < 0) - return ret; - ret = FFERROR_REDO; - goto leave; - } - if ((ret = flv_get_extradata(s, st, size)) < 0) - return ret; - /* Workaround for buggy Omnia A/XE encoder */ - t = av_dict_get(s->metadata, "Encoder", NULL, 0); - if (st->codecpar->codec_id == AV_CODEC_ID_AAC && t && !strcmp(t->value, "Omnia A/XE")) - st->codecpar->extradata_size = 2; + if (!size) + break; - ret = FFERROR_REDO; - goto leave; +next_track: + if (multitrack_type == MultitrackTypeOneTrack) { + av_log(s, AV_LOG_ERROR, "Attempted to read next track in single-track mode.\n"); + return AVERROR_BUG; } - } - - /* skip empty data packets */ - if (!size) { - ret = FFERROR_REDO; - goto leave; - } - ret = av_get_packet(s->pb, pkt, size); - if (ret < 0) - return ret; - pkt->dts = dts; - pkt->pts = pts == AV_NOPTS_VALUE ? dts : pts; - pkt->stream_index = st->index; - pkt->pos = pos; - if (!multitrack && flv->new_extradata[stream_type]) { - int sret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, - flv->new_extradata[stream_type], - flv->new_extradata_size[stream_type]); - if (sret >= 0) { - flv->new_extradata[stream_type] = NULL; - flv->new_extradata_size[stream_type] = 0; + if (multitrack_type == MultitrackTypeManyTracksManyCodecs) { + codec_id = avio_rb32(s->pb); + size -= 4; } - } else if (multitrack - && flv->mt_extradata_cnt > track_idx - && flv->mt_extradata[track_idx]) { - int sret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, - flv->mt_extradata[track_idx], - flv->mt_extradata_sz[track_idx]); - if (sret >= 0) { - flv->mt_extradata[track_idx] = NULL; - flv->mt_extradata_sz[track_idx] = 0; + + track_idx = avio_r8(s->pb); + size--; + + if (avio_feof(s->pb)) { + av_log(s, AV_LOG_WARNING, "premature EOF\n"); + /* return REDO so that any potentially queued up packages can be drained first */ + return FFERROR_REDO; } } - if (stream_type == FLV_STREAM_TYPE_AUDIO && !enhanced_flv && - (sample_rate != flv->last_sample_rate || - channels != flv->last_channels)) { - flv->last_sample_rate = sample_rate; - flv->last_channels = channels; - ff_add_param_change(pkt, channels, 0, sample_rate, 0, 0); - } - - if (stream_type == FLV_STREAM_TYPE_AUDIO || - (flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY || - stream_type == FLV_STREAM_TYPE_SUBTITLE || - stream_type == FLV_STREAM_TYPE_DATA) - pkt->flags |= AV_PKT_FLAG_KEY; leave: last = avio_rb32(s->pb); @@ -1755,7 +1803,7 @@ leave: if (ret >= 0) flv->last_ts = pkt->dts; - return ret; + return ret ? ret : res; } static int flv_read_seek(AVFormatContext *s, int stream_index,