mbox series

[FFmpeg-devel,RFC,00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel

Message ID 20240521090316.782-1-timo@rothenpieler.org
Headers show
Series flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel | expand

Message

Timo Rothenpieler May 21, 2024, 9:02 a.m. UTC
This is based on the preliminary spec for enhanced rtmp v2:
https://veovera.org/docs/enhanced/enhanced-rtmp-v2

The spec is not final, and can still undergo breaking changes, hence this set is purely for comments and review, and not ready to be merged until the final v2 spec is published.

There are no samples out in the wild yet, so testing interoperability with other software has not happened yet either.
Specially the two other multitrack modes, where multiple tracks are in the same packet, have not been tested at all, since no software can write such files.

The set can also be found on GitHub, where ignoring whitespaces makes specially the last patch a lot more readable:
https://github.com/BtbN/FFmpeg/tree/enhanced-flv


Dennis Sädtler via ffmpeg-devel (2):
  avformat/flvenc: Implement support for multi-track video
  avformat/flvdec: Add support for demuxing multi-track FLV

Timo Rothenpieler (11):
  avformat/flvenc: add enhanced audio codecs
  avformat/flvenc: remove !size check for audio packets
  avformat/flvdec: add enhanced audio codecs
  avformat/flvenc: refactor fourcc writing
  avformat/flvenc: write enhanced rtmp multichannel info for audio with
    more than two channels
  avformat/flvdec: parse enhanced rtmp multichannel info
  avformat/flvenc: add support for writing multi track audio
  avformat/flvdec: add support for reading multi track audio
  avformat/rtmpproto: add more enhanced rtmp codecs
  avformat/flvdec: stop shadowing local variables
  avformat/flvdec: support all multi-track modes

 libavformat/flv.h       |  21 ++
 libavformat/flvdec.c    | 654 +++++++++++++++++++++++++++-------------
 libavformat/flvenc.c    | 443 +++++++++++++++++++++------
 libavformat/rtmpproto.c |  11 +-
 4 files changed, 819 insertions(+), 310 deletions(-)

Comments

Steven Liu May 23, 2024, 1:57 a.m. UTC | #1
Timo Rothenpieler <timo@rothenpieler.org> 于2024年5月21日周二 17:03写道:
>
> This is based on the preliminary spec for enhanced rtmp v2:
> https://veovera.org/docs/enhanced/enhanced-rtmp-v2
>
> The spec is not final, and can still undergo breaking changes, hence this set is purely for comments and review, and not ready to be merged until the final v2 spec is published.
>
> There are no samples out in the wild yet, so testing interoperability with other software has not happened yet either.
> Specially the two other multitrack modes, where multiple tracks are in the same packet, have not been tested at all, since no software can write such files.
Yes, there have reconnect function in v2
And i cannot sure there maybe have some problem in multitrack mode,
different codecs type, change amount the tracks, maybe need more
process, need check GOP、 keyframes 、VFR mode, because they will get
different timestamps.
>
> The set can also be found on GitHub, where ignoring whitespaces makes specially the last patch a lot more readable:
> https://github.com/BtbN/FFmpeg/tree/enhanced-flv
>
>
> Dennis Sädtler via ffmpeg-devel (2):
>   avformat/flvenc: Implement support for multi-track video
>   avformat/flvdec: Add support for demuxing multi-track FLV
>
> Timo Rothenpieler (11):
>   avformat/flvenc: add enhanced audio codecs
>   avformat/flvenc: remove !size check for audio packets
>   avformat/flvdec: add enhanced audio codecs
>   avformat/flvenc: refactor fourcc writing
>   avformat/flvenc: write enhanced rtmp multichannel info for audio with
>     more than two channels
>   avformat/flvdec: parse enhanced rtmp multichannel info
>   avformat/flvenc: add support for writing multi track audio
>   avformat/flvdec: add support for reading multi track audio
>   avformat/rtmpproto: add more enhanced rtmp codecs
>   avformat/flvdec: stop shadowing local variables
>   avformat/flvdec: support all multi-track modes
>
>  libavformat/flv.h       |  21 ++
>  libavformat/flvdec.c    | 654 +++++++++++++++++++++++++++-------------
>  libavformat/flvenc.c    | 443 +++++++++++++++++++++------
>  libavformat/rtmpproto.c |  11 +-
>  4 files changed, 819 insertions(+), 310 deletions(-)
>
> --
> 2.43.2
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".


Thanks
Steven
Andrew Sayers May 28, 2024, 3:14 p.m. UTC | #2
On Tue, May 21, 2024 at 11:02:09AM +0200, Timo Rothenpieler wrote:
> This is based on the preliminary spec for enhanced rtmp v2:
> https://veovera.org/docs/enhanced/enhanced-rtmp-v2
> 
> The spec is not final, and can still undergo breaking changes, hence this set is purely for comments and review, and not ready to be merged until the final v2 spec is published.
> 
> There are no samples out in the wild yet, so testing interoperability with other software has not happened yet either.
> Specially the two other multitrack modes, where multiple tracks are in the same packet, have not been tested at all, since no software can write such files.
> 
> The set can also be found on GitHub, where ignoring whitespaces makes specially the last patch a lot more readable:
> https://github.com/BtbN/FFmpeg/tree/enhanced-flv
> 

I ran this against a little review bot I'm working on.
Please do s/\* / \*/g on the following:


avformat/flvenc: add support for writing multi track audio

> +static void flv_write_multichannel_header(AVFormatContext* s, AVCodecParameters* par, int64_t ts, int stream_index)


avformat/flvenc: write enhanced rtmp multichannel info for audio with more than two channels

> +static void flv_write_multichannel_body(AVFormatContext* s, AVCodecParameters* par)
> +static int flv_get_multichannel_body_size(AVCodecParameters* par)
> +static void flv_write_multichannel_header(AVFormatContext* s, AVCodecParameters* par, int64_t ts)


avformat/flvenc: add enhanced audio codecs

> +static void flv_write_aac_header(AVFormatContext* s, AVCodecParameters* par)


avformat/flvenc: Implement support for multi-track video

> +static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, int64_t ts, int stream_index) {
Timo Rothenpieler May 28, 2024, 5:11 p.m. UTC | #3
On 28.05.2024 17:14, Andrew Sayers wrote:
> On Tue, May 21, 2024 at 11:02:09AM +0200, Timo Rothenpieler wrote:
>> This is based on the preliminary spec for enhanced rtmp v2:
>> https://veovera.org/docs/enhanced/enhanced-rtmp-v2
>>
>> The spec is not final, and can still undergo breaking changes, hence this set is purely for comments and review, and not ready to be merged until the final v2 spec is published.
>>
>> There are no samples out in the wild yet, so testing interoperability with other software has not happened yet either.
>> Specially the two other multitrack modes, where multiple tracks are in the same packet, have not been tested at all, since no software can write such files.
>>
>> The set can also be found on GitHub, where ignoring whitespaces makes specially the last patch a lot more readable:
>> https://github.com/BtbN/FFmpeg/tree/enhanced-flv
>>
> 
> I ran this against a little review bot I'm working on.
> Please do s/\* / \*/g on the following:

The usage of that is already inconsistent throughout the flv codebase, 
so running that would make the patch touch a lot of unrelated things.

Seems more sane to just do a general style-cleanup after the fact.