| Message ID | 20220616195534.5278-1-anton@khirnov.net |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel,01/35] fftools/ffmpeg_mux: add private muxer context | expand |
| Context | Check | Description |
|---|---|---|
| andriy/make_x86 | success | Make finished |
| andriy/make_fate_x86 | success | Make fate finished |
The current version of this set can also be found in my tree git://git.khirnov.net/libav branch ffmpeg_mt/mux
On Fri, Jun 17, 2022 at 12:27:18PM +0200, Anton Khirnov wrote: > The current version of this set can also be found in my tree > git://git.khirnov.net/libav > branch ffmpeg_mt/mux There are really many files changing, its hard to say for sure that all are the same issue, but basically it all seems more or less frames in some streams including cases where there are hugely more or 0 Here are some examples: ffmpeg -i matrixbench_mpeg2.mpg -vcodec rawvideo -pix_fmt rgb555 -allow_raw_vfw 1 -vframes 1 -bitexact file-rgb555.mkv the new file is much bigger (due to the audio track) -rw-r----- 1 michael michael 2765813 Jul 8 16:57 file-rgb555.mkv -rw-r----- 1 michael michael 834643 Jul 8 17:02 file-rgb555-ref.mkv another one: ./ffmpeg -y -i vlcticket/8344/DVR_NVR_IP\ Camera01_20130321162325_20130321162358_576877.mp4 -vframes 1 -aframes 1 -bitexact -f framecrc - This appears to loose the video stream #channel_layout_name 1: mono -0, 0, 0, 1, 2880000, 0x4136bc92 1, 112, 112, 320, 640, 0x2cd73b36 sample in https://samples.ffmpeg.org/camera-dvr/hikvision/ This one fails a bit worse than before (ffmpeg succeeds before besides producing errors as well) my notes say this worked better only before 04aa09c4bcf2d5a634a35da3a3ae3fc1abe30ef8 the file is a little big and i havnt found it anywhere online, i will try to send it privately to you ffmpeg -i 2014-10-17\ 11.31\ i95Dev\ -\ Carlo\ Pazolini\ _\ KWI\ -\ Meeting.g2m -bitexact -max_muxing_queue_size 8000 -vframes 2 file-g2m5.avi Metadata: DeviceConformanceTemplate: L2 WMFSDKNeeded : 0.0.0.0000 WMFSDKVersion : 12.0.7601.17514 IsVBR : 1 WM/ToolVersion : 6.4.3 Build 1767 WM/ToolName : GoToMeeting BitRateFrom the writer: 97087 Audio samples : 34341 Video samples : 3740 recording time : Fri, 17 Oct 2014 12:28:16 Eastern Daylight Time Duration: 00:57:13.86, start: 0.000000, bitrate: 100 kb/s Stream #0:0: Audio: wmav2 (a[1][0][0] / 0x0161), 44100 Hz, mono, fltp, 48 kb/s Stream #0:1: Data: none, 2 kb/s Stream #0:2: Video: g2m (G2M5 / 0x354D3247), rgb24, 1440x900, 49 kb/s, 1k tbr, 1k tbn Stream mapping: Stream #0:2 -> #0:0 (g2m (native) -> mpeg4 (native)) Stream #0:0 -> #0:1 (wmav2 (native) -> mp3 (libmp3lame)) Press [q] to stop, [?] for help [libmp3lame @ 0x55c17cf03140] Queue input is backward in time Output #0, avi, to 'file-g2m5.avi': Metadata: DeviceConformanceTemplate: L2 WMFSDKNeeded : 0.0.0.0000 WMFSDKVersion : 12.0.7601.17514 IsVBR : 1 WM/ToolVersion : 6.4.3 Build 1767 WM/ToolName : GoToMeeting BitRateFrom the writer: 97087 Audio samples : 34341 Video samples : 3740 recording time : Fri, 17 Oct 2014 12:28:16 Eastern Daylight Time Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p(tv, progressive), 1440x900, q=2-31, 200 kb/s, 1k fps, 1k tbn Metadata: encoder : Lavc mpeg4 Side data: cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 44100 Hz, mono, fltp Metadata: encoder : Lavc libmp3lame [avi @ 0x55c17cf31340] Too large number of skipped frames 194184 > 600000kbits/s speed= 140x av_interleaved_write_frame(): Invalid argument Error muxing a packet for output file #0 [avi @ 0x55c17cf31340] Too large number of skipped frames 194085 > 60000 frame= 2 fps=1.3 q=2.0 Lsize= 1855kB time=00:03:14.18 bitrate= 78.3kbits/s speed= 129x video:149kB audio:1517kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 11.381945% Conversion failed! ./ffmpeg -i tickets/1666/avc-intra-panasonic-AG-HPX301E.mov -vframes 3 -aframes 2 -bitexact -f framecrc - duplicate behavior of the a/vframe issue above, one stream disappears sample in https://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket1666/ [...]
Quoting Michael Niedermayer (2022-07-08 18:58:11) > On Fri, Jun 17, 2022 at 12:27:18PM +0200, Anton Khirnov wrote: > > The current version of this set can also be found in my tree > > git://git.khirnov.net/libav > > branch ffmpeg_mt/mux > > There are really many files changing, its hard to say for sure that all are the > same issue, but basically it all seems more or less frames in some streams > including cases where there are hugely more or 0 > > Here are some examples: > > > ffmpeg -i matrixbench_mpeg2.mpg -vcodec rawvideo -pix_fmt rgb555 -allow_raw_vfw 1 -vframes 1 -bitexact file-rgb555.mkv > > the new file is much bigger (due to the audio track) > > -rw-r----- 1 michael michael 2765813 Jul 8 16:57 file-rgb555.mkv > -rw-r----- 1 michael michael 834643 Jul 8 17:02 file-rgb555-ref.mkv Where can I find this file? > another one: > ./ffmpeg -y -i vlcticket/8344/DVR_NVR_IP\ Camera01_20130321162325_20130321162358_576877.mp4 -vframes 1 -aframes 1 -bitexact -f framecrc - > > This appears to loose the video stream > > #channel_layout_name 1: mono > -0, 0, 0, 1, 2880000, 0x4136bc92 > 1, 112, 112, 320, 640, 0x2cd73b36 > > sample in https://samples.ffmpeg.org/camera-dvr/hikvision/ The sync queue got confused by stale frame durations, which are now overwritten in the updated version of 22/35. But do note that there may be valid situations where a stream will "disappear" when you specify -frames constraints on multiple streams, because -frames is supposed to cut the whole file once the constraint is reached, not just the stream it applies to. If you want a specific number of frames in each stream, you should rather use the trim/atrim filters. See also my discussion with Andreas under 24/35 in this thread. > This one fails a bit worse than before (ffmpeg succeeds before besides producing errors as well) > my notes say this worked better only before 04aa09c4bcf2d5a634a35da3a3ae3fc1abe30ef8 > the file is a little big and i havnt found it anywhere online, i will try to > send it privately to you > > ffmpeg -i 2014-10-17\ 11.31\ i95Dev\ -\ Carlo\ Pazolini\ _\ KWI\ -\ Meeting.g2m -bitexact -max_muxing_queue_size 8000 -vframes 2 file-g2m5.avi > > Metadata: > DeviceConformanceTemplate: L2 > WMFSDKNeeded : 0.0.0.0000 > WMFSDKVersion : 12.0.7601.17514 > IsVBR : 1 > WM/ToolVersion : 6.4.3 Build 1767 > WM/ToolName : GoToMeeting > BitRateFrom the writer: 97087 > Audio samples : 34341 > Video samples : 3740 > recording time : Fri, 17 Oct 2014 12:28:16 Eastern Daylight Time > Duration: 00:57:13.86, start: 0.000000, bitrate: 100 kb/s > Stream #0:0: Audio: wmav2 (a[1][0][0] / 0x0161), 44100 Hz, mono, fltp, 48 kb/s > Stream #0:1: Data: none, 2 kb/s > Stream #0:2: Video: g2m (G2M5 / 0x354D3247), rgb24, 1440x900, 49 kb/s, 1k tbr, 1k tbn > Stream mapping: > Stream #0:2 -> #0:0 (g2m (native) -> mpeg4 (native)) > Stream #0:0 -> #0:1 (wmav2 (native) -> mp3 (libmp3lame)) > Press [q] to stop, [?] for help > [libmp3lame @ 0x55c17cf03140] Queue input is backward in time > Output #0, avi, to 'file-g2m5.avi': > Metadata: > DeviceConformanceTemplate: L2 > WMFSDKNeeded : 0.0.0.0000 > WMFSDKVersion : 12.0.7601.17514 > IsVBR : 1 > WM/ToolVersion : 6.4.3 Build 1767 > WM/ToolName : GoToMeeting > BitRateFrom the writer: 97087 > Audio samples : 34341 > Video samples : 3740 > recording time : Fri, 17 Oct 2014 12:28:16 Eastern Daylight Time > Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p(tv, progressive), 1440x900, q=2-31, 200 kb/s, 1k fps, 1k tbn > Metadata: > encoder : Lavc mpeg4 > Side data: > cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A > Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 44100 Hz, mono, fltp > Metadata: > encoder : Lavc libmp3lame > [avi @ 0x55c17cf31340] Too large number of skipped frames 194184 > 600000kbits/s speed= 140x > av_interleaved_write_frame(): Invalid argument > Error muxing a packet for output file #0 > [avi @ 0x55c17cf31340] Too large number of skipped frames 194085 > 60000 > frame= 2 fps=1.3 q=2.0 Lsize= 1855kB time=00:03:14.18 bitrate= 78.3kbits/s speed= 129x > video:149kB audio:1517kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 11.381945% > Conversion failed! The new code ends up sending a few more audio packets to the muxer, so av_interleaved_write_frame() returns an error, which results in a non-zero exit status. In the current code, the same error appears during flushing the muxing queue in av_write_trailer() and is actually returned from av_write_trailer(), but ffmpeg for some reason does not treat av_write_trailer() errors the same way. I do not know why that is, seems like a bug to me. In any case, conversion is quite broken both before and after my patches, so I would say it is not a problem with my code. > ./ffmpeg -i tickets/1666/avc-intra-panasonic-AG-HPX301E.mov -vframes 3 -aframes 2 -bitexact -f framecrc - > duplicate behavior of the a/vframe issue above, one stream disappears > > sample in https://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket1666/ Again, should be resolved by the updated patch.
On Wed, Jul 13, 2022 at 12:58:54PM +0200, Anton Khirnov wrote: > Quoting Michael Niedermayer (2022-07-08 18:58:11) > > On Fri, Jun 17, 2022 at 12:27:18PM +0200, Anton Khirnov wrote: > > > The current version of this set can also be found in my tree > > > git://git.khirnov.net/libav > > > branch ffmpeg_mt/mux > > > > There are really many files changing, its hard to say for sure that all are the > > same issue, but basically it all seems more or less frames in some streams > > including cases where there are hugely more or 0 > > > > Here are some examples: > > > > > > ffmpeg -i matrixbench_mpeg2.mpg -vcodec rawvideo -pix_fmt rgb555 -allow_raw_vfw 1 -vframes 1 -bitexact file-rgb555.mkv > > > > the new file is much bigger (due to the audio track) > > > > -rw-r----- 1 michael michael 2765813 Jul 8 16:57 file-rgb555.mkv > > -rw-r----- 1 michael michael 834643 Jul 8 17:02 file-rgb555-ref.mkv > > Where can I find this file? the 2 file-rgb555.mkv files are the outputs from ffmpeg from the matrixbench_mpeg2 file and teh command line above. From before and after the change IIRC matrixbench_mpeg2.mpg should be here: https://samples.ffmpeg.org/benchmark/testsuite1/ thx [...]
Quoting Michael Niedermayer (2022-07-14 00:12:59) > On Wed, Jul 13, 2022 at 12:58:54PM +0200, Anton Khirnov wrote: > > Quoting Michael Niedermayer (2022-07-08 18:58:11) > > > On Fri, Jun 17, 2022 at 12:27:18PM +0200, Anton Khirnov wrote: > > > > The current version of this set can also be found in my tree > > > > git://git.khirnov.net/libav > > > > branch ffmpeg_mt/mux > > > > > > There are really many files changing, its hard to say for sure that all are the > > > same issue, but basically it all seems more or less frames in some streams > > > including cases where there are hugely more or 0 > > > > > > Here are some examples: > > > > > > > > > ffmpeg -i matrixbench_mpeg2.mpg -vcodec rawvideo -pix_fmt rgb555 -allow_raw_vfw 1 -vframes 1 -bitexact file-rgb555.mkv > > > > > > the new file is much bigger (due to the audio track) > > > > > > -rw-r----- 1 michael michael 2765813 Jul 8 16:57 file-rgb555.mkv > > > -rw-r----- 1 michael michael 834643 Jul 8 17:02 file-rgb555-ref.mkv > > > > Where can I find this file? > > the 2 file-rgb555.mkv files are the outputs from ffmpeg from the matrixbench_mpeg2 > file and teh command line above. From before and after the change IIRC > > matrixbench_mpeg2.mpg should be here: > https://samples.ffmpeg.org/benchmark/testsuite1/ The output is ~815kB in the current branch, same as before. So I assume the problem got fixed by the changes I did for one of the other samples. Thanks for testing and let me know if you find any other issues.
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 69a368b8d1..c6ffe9fdf6 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -581,9 +581,12 @@ typedef struct OutputStream { int64_t error[4]; } OutputStream; +typedef struct Muxer Muxer; + typedef struct OutputFile { int index; + Muxer *mux; const AVOutputFormat *format; AVFormatContext *ctx; @@ -594,8 +597,6 @@ typedef struct OutputFile { uint64_t limit_filesize; /* filesize limit expressed in bytes */ int shortest; - - int header_written; } OutputFile; extern InputStream **input_streams; @@ -694,6 +695,7 @@ int hw_device_setup_for_filter(FilterGraph *fg); int hwaccel_decode_init(AVCodecContext *avctx); +int of_muxer_init(OutputFile *of); /* open the muxer when all the streams are initialized */ int of_check_init(OutputFile *of); int of_write_trailer(OutputFile *of); diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c index a55fd18f8f..e47a55c4e9 100644 --- a/fftools/ffmpeg_mux.c +++ b/fftools/ffmpeg_mux.c @@ -32,6 +32,10 @@ #include "libavformat/avformat.h" #include "libavformat/avio.h" +struct Muxer { + int header_written; +}; + static void close_all_output_streams(OutputStream *ost, OSTFinished this_stream, OSTFinished others) { int i; @@ -64,7 +68,7 @@ void of_write_packet(OutputFile *of, AVPacket *pkt, OutputStream *ost, ost->frame_number++; } - if (!of->header_written) { + if (!of->mux->header_written) { AVPacket *tmp_pkt; /* the muxer is not initialized yet, buffer the packet */ if (!av_fifo_can_write(ost->muxing_queue)) { @@ -182,7 +186,7 @@ static int print_sdp(void) AVFormatContext **avc; for (i = 0; i < nb_output_files; i++) { - if (!output_files[i]->header_written) + if (!output_files[i]->mux->header_written) return 0; } @@ -246,7 +250,7 @@ int of_check_init(OutputFile *of) return ret; } //assert_avoptions(of->opts); - of->header_written = 1; + of->mux->header_written = 1; av_dump_format(of->ctx, of->index, of->ctx->url, 1); nb_output_dumped++; @@ -282,7 +286,7 @@ int of_write_trailer(OutputFile *of) { int ret; - if (!of->header_written) { + if (!of->mux->header_written) { av_log(NULL, AV_LOG_ERROR, "Nothing was written into output file %d (%s), because " "at least one of its streams received no packets.\n", @@ -313,5 +317,19 @@ void of_close(OutputFile **pof) avformat_free_context(s); av_dict_free(&of->opts); + av_freep(&of->mux); + av_freep(pof); } + +int of_muxer_init(OutputFile *of) +{ + Muxer *mux = av_mallocz(sizeof(*mux)); + + if (!mux) + return AVERROR(ENOMEM); + + of->mux = mux; + + return 0; +} diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 398067da96..ab8d307c3c 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -2947,6 +2947,12 @@ loop_end: exit_program(1); } + err = of_muxer_init(of); + if (err < 0) { + av_log(NULL, AV_LOG_FATAL, "Error initializing internal muxing state\n"); + exit_program(1); + } + return 0; }