| Message ID | 20240204090233.1157950-1-jdek@itanimul.li |
|---|---|
| Headers | show |
| Series | Remove SDL2 output devices | expand |
> On Feb 4, 2024, at 17:02, J. Dekker <jdek@itanimul.li> wrote: > > With the addition of threading in ffmpeg.c, the SDL2 devices no longer have the > 'main' thread. This means that both the SDL2 and OpenGL output device are broken > in master. Rather than attempting to fix it, they should be removed instead as > there are better alternatives for debugging or viewing streams. Please note they are broken only with fftools, they work as before when used as libavdevice in theory. Please take this into consideration. I have no use case by myself. > > The 'pipe:' output can be used with a real video player such as mpv, vlc, or > even ffplay. For cases where the user was an application using the API they > should supply their own renderer. > > J. Dekker (2): > avdevice: remove sdl2 outdev > avdevice: remove OpenGL device > > MAINTAINERS | 3 - > configure | 16 - > doc/outdevs.texi | 105 --- > libavdevice/Makefile | 2 - > libavdevice/alldevices.c | 2 - > libavdevice/opengl_enc.c | 1313 -------------------------------------- > libavdevice/sdl2.c | 370 ----------- > 7 files changed, 1811 deletions(-) > delete mode 100644 libavdevice/opengl_enc.c > delete mode 100644 libavdevice/sdl2.c > > -- > 2.43.0 > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Quoting J. Dekker (2024-02-04 10:02:31) > With the addition of threading in ffmpeg.c, the SDL2 devices no longer have the > 'main' thread. This means that both the SDL2 and OpenGL output device are broken > in master. Rather than attempting to fix it, they should be removed instead as > there are better alternatives for debugging or viewing streams. > > The 'pipe:' output can be used with a real video player such as mpv, vlc, or > even ffplay. For cases where the user was an application using the API they > should supply their own renderer. I am in favor of this. Even when they did "work", they were toys unsuitable for any serious use.
Quoting Zhao Zhili (2024-02-04 10:19:11) > > On Feb 4, 2024, at 17:02, J. Dekker <jdek@itanimul.li> wrote: > > > > With the addition of threading in ffmpeg.c, the SDL2 devices no longer have the > > 'main' thread. This means that both the SDL2 and OpenGL output device are broken > > in master. Rather than attempting to fix it, they should be removed instead as > > there are better alternatives for debugging or viewing streams. > > Please note they are broken only with fftools, they work as before when used as libavdevice > in theory. In other words, unlike any normal muxer, they are broken for any multithreaded caller and always have been.
FFmpeg project leader never left, it is still Michael. But now there are his minions like Anton and others. FFmpeg is already dead project.
On Sun, 4 Feb 2024, J. Dekker wrote: > With the addition of threading in ffmpeg.c, the SDL2 devices no longer have the > 'main' thread. This means that both the SDL2 and OpenGL output device are broken > in master. Rather than attempting to fix it, they should be removed instead as > there are better alternatives for debugging or viewing streams. Actually they work here on a linux box with OpenSuse 15.5. So even if they are broken on some setups, they are not broken everywhere, or not more broken than they used to be. Also, poper deprecation is needed here, since not only the CLI tools might use these. Especially since there is no drop-in replacement. > > The 'pipe:' output can be used with a real video player such as mpv, vlc, or > even ffplay. For cases where the user was an application using the API they > should supply their own renderer. Yeah, but I never liked when people piped uncompressed data... Not everything that the devices support can be serialized, it is extra CPU, latency of the receiving app reading from pipe is a question... I'd be a lot more happy with this if we'd offer some replacement which has no issues. Maybe a libplacebo based outdev. Regards, Marton
Le 4 février 2024 11:11:12 GMT+01:00, Marton Balint <cus@passwd.hu> a écrit : >Actually they work here on a linux box with OpenSuse 15.5. So even if they >are broken on some setups, they are not broken everywhere, or not more broken than they used to be. No. They were always broken in terms of the design, and they are more technically broken than before because the threading rework exposed the design bugs from within fftools. No sane application would use this. If it doesn't even work in fftools, it should be removed. >Also, poper deprecation is needed here, since not only the CLI tools might use these. Especially since there is no drop-in replacement. First it's not what would be considered an API. The removal shouldn't break source compatibility, so deprecation won't get us anything here. Where would you even put the deprecation guards? And then deprecation only makes sense if it can be fixed. Nobody has come forward with a practical solution to make it work, probably because there is not one, at least on MacOS. >> The 'pipe:' output can be used with a real video player such as mpv, vlc, or >> even ffplay. For cases where the user was an application using the API they >> should supply their own renderer. > >Yeah, but I never liked when people piped uncompressed data... Not everything that the devices support can be serialized, it is extra CPU, latency of the receiving app reading from pipe is a question... That sounds pretty minor problems for something that's purely meant for testing, and well, at least piping works. >I'd be a lot more happy with this if we'd offer some replacement which has no issues. Maybe a libplacebo based outdev. That's orthogonal, and you're welcome to provide patches. But AFAICT, any video output device would suffer the same problems on the same platforms. You simply can't treat video output as a generic pipeline component, at least on Windows and especially MacOS.
Le 4 février 2024 10:02:31 GMT+01:00, "J. Dekker" <jdek@itanimul.li> a écrit : >With the addition of threading in ffmpeg.c, the SDL2 devices no longer have the >'main' thread. This means that both the SDL2 and OpenGL output device are broken >in master. Rather than attempting to fix it, they should be removed instead as >there are better alternatives for debugging or viewing streams. This is as agreed after discussed in yesterday's technical meeting. So obviously I support this patchset.
On Sun, Feb 04, 2024 at 10:02:31AM +0100, J. Dekker wrote: > With the addition of threading in ffmpeg.c, the SDL2 devices no longer have the > 'main' thread. This means that both the SDL2 and OpenGL output device are broken > in master. Rather than attempting to fix it, they should be removed instead as > there are better alternatives for debugging or viewing streams. > > The 'pipe:' output can be used with a real video player such as mpv, vlc, or > even ffplay. For cases where the user was an application using the API they > should supply their own renderer. we should point to our tools (ffplay in this case) first and foremost. Also if they can be used, an example is needed in the documentation. This could be in place of the removed device Alternatively, a flag could be added to devices that specifies that they need to be called from the main thread This flag could switch tools which want to support this into single threaded mode, if it doesnt fit naturally in their architecture. While tools not supporting it could simply fail with a "unsupported" message thx [...]
On Sun, 4 Feb 2024, Rémi Denis-Courmont wrote: > > > Le 4 février 2024 11:11:12 GMT+01:00, Marton Balint <cus@passwd.hu> a écrit : >> Actually they work here on a linux box with OpenSuse 15.5. So even if they >> are broken on some setups, they are not broken everywhere, or not more broken than they used to be. > > No. They were always broken in terms of the design, and they are more technically broken than before because the threading rework exposed the design bugs from within fftools. > > No sane application would use this. If it doesn't even work in fftools, it should be removed. As I wrote earlier, it works for me. > >> Also, poper deprecation is needed here, since not only the CLI tools might use these. Especially since there is no drop-in replacement. > > First it's not what would be considered an API. The removal shouldn't break source compatibility, so deprecation won't get us anything here. Where would you even put the deprecation guards? See what Anton did with the BKTR device. > > And then deprecation only makes sense if it can be fixed. Nobody has come forward with a practical solution to make it work, probably because there is not one, at least on MacOS. See Michael's suggestions with the thread safety flag. Regards, Marton
On date Sunday 2024-02-04 10:02:31 +0100, J. Dekker wrote: > With the addition of threading in ffmpeg.c, the SDL2 devices no longer have the > 'main' thread. This means that both the SDL2 and OpenGL output device are broken > in master. Rather than attempting to fix it, they should be removed instead as > there are better alternatives for debugging or viewing streams. > > The 'pipe:' output can be used with a real video player such as mpv, vlc, or > even ffplay. For cases where the user was an application using the API they > should supply their own renderer. > > J. Dekker (2): > avdevice: remove sdl2 outdev > avdevice: remove OpenGL device I am against this. 1. If at all, the devices should be deprecated and removed only after a given timeframe, so users can switch to something else (you don't know what they are using). 2. > With the addition of threading in ffmpeg.c, the SDL2 devices no longer have the > 'main' thread. This implies a misunderstanding of what these components are. If they are broken with ffmpeg.c this is not a good reason to remove them (ffmpeg.c is not the only user). Also, it was already suggested some way to fix it, it's not like they are "broken by design", it is just that the assumptions done previuosly apply no more. Probably ffmpeg.c should not use the main thread or make this selectable. If this cannot be supported in a given OS, the feature should be disabled only for that OS.
Removing SDL2 sounds like a very bad idea. There are many examples which are using these output devices, and all these examples would be broken. A quick search in my book http://www.astro-electronic.de/FFmpeg_Book.pdf finds about 40 occurences for "-f sdl" or "-f sdl2". Michael
> On Feb 6, 2024, at 16:08, Michael Koch <astroelectronic@t-online.de> wrote: > > Removing SDL2 sounds like a very bad idea. There are many examples which are using these output devices, and all these examples would be broken. A quick search in my book > http://www.astro-electronic.de/FFmpeg_Book.pdf > finds about 40 occurences for "-f sdl" or "-f sdl2". Those examples are broken already before the patch. We are not talking about that in this thread. Please reference https://www.mail-archive.com/ffmpeg-devel@ffmpeg.org/msg155921.html > > Michael > > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Zhao Zhili (12024-02-06):
> Those examples are broken already before the patch.
Funny that the people who actually use the feature had not noticed.
On Tue, Feb 6, 2024 at 9:08 AM Michael Koch <astroelectronic@t-online.de> wrote: > Removing SDL2 sounds like a very bad idea. There are many examples which > are using these output devices, and all these examples would be broken. > A quick search in my book > http://www.astro-electronic.de/FFmpeg_Book.pdf > finds about 40 occurences for "-f sdl" or "-f sdl2". > would it be easier/possible to fix SDL to work on any thread, instead of keeping this odd architecture in the codebase?
I didn't notice any problems with -f sdl2. I just tested again with Windows 11 and the latest FFmpeg build from Gyan, just 2 days old. ffmpeg -re -f lavfi -i testsrc2=s=800x600 -t 10 -f sdl2 - Works without any problems. Michael
> On Feb 7, 2024, at 04:51, Michael Koch <astroelectronic@t-online.de> wrote: > > I didn't notice any problems with -f sdl2. I just tested again with Windows 11 and the latest FFmpeg build from Gyan, just 2 days old. > > ffmpeg -re -f lavfi -i testsrc2=s=800x600 -t 10 -f sdl2 - > > Works without any problems. It works until move or resize the window. As far as I know, it’s broken on Linux/macOS/Windows, so not “works without any problems”. SDL should be run in main thread, that’s all, although we can do render in separate thread with vulkan. It’s easy to create a video sink filter to do render, and setup a window on fftools/ffmpeg main thread, or implement the whole preview task just inside fftools. Not everyone agrees on the preview feature with ffmpeg, not to mention libSDL. > > Michael > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> It works until move or resize the window.
yes, that's right. I didn't notice because I didn't try to move or resize the window.
My point is: Removing SDL would break many examples that can be found in the internet.
Michael
Quoting Marton Balint (2024-02-04 11:11:12) > > The 'pipe:' output can be used with a real video player such as mpv, vlc, or > > even ffplay. For cases where the user was an application using the API they > > should supply their own renderer. > > Yeah, but I never liked when people piped uncompressed data... Not > everything that the devices support can be serialized, For instance? What do these devices support that e.g. NUT does not? > it is extra CPU, latency of the receiving app reading from pipe is a > question... People keep saying this in defence of these devices, but never support such claims with any data. I have doubts this is an actual problem in practice. > I'd be a lot more happy with this if we'd offer some replacement which has > no issues. Maybe a libplacebo based outdev. I don't think it's possible to have such a replacement, neither should we try. Muxer API is simply the wrong abstraction for playback.
Anton Khirnov (12024-02-07): > For instance? What do these devices support that e.g. NUT does not? Returning the latency of the device. > neither should we try. This is the libav mindset we do not want in FFmpeg.
Quoting Stefano Sabatini (2024-02-05 01:02:20) > This implies a misunderstanding of what these components are. If > they are broken with ffmpeg.c this is not a good reason to remove > them (ffmpeg.c is not the only user). They are broken with _any_ caller that happens to call libavformat from a thread other than the main one. Since libavformat API does not impose any such restrictions on its callers, these devices are broken in general. > Also, it was already suggested some way to fix it There is no way to fix them other than impose new restrictions on callers... > it's not like they are "broken by design", ...so they are precisely broken by design.
Anton Khirnov (12024-02-07):
> ...so they are precisely broken by design.
Words words words.
Words to try and hide that something used to work for people and now you
are done with it it no longer works.
Exactly the kind of attitude that killed libav, killing FFmpeg now.
On Wed, Feb 7, 2024 at 6:38 PM Nicolas George <george@nsup.org> wrote: > Anton Khirnov (12024-02-07): > > ...so they are precisely broken by design. > > Words words words. > > Words to try and hide that something used to work for people and now you > are done with it it no longer works. > > Exactly the kind of attitude that killed libav, killing FFmpeg now. > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On Wed, Feb 7, 2024 at 6:40 PM Vittorio Giovara <vittorio.giovara@gmail.com> wrote: > On Wed, Feb 7, 2024 at 6:38 PM Nicolas George <george@nsup.org> wrote: > > > Anton Khirnov (12024-02-07): > > > ...so they are precisely broken by design. > > > > Words words words. > > > > Words to try and hide that something used to work for people and now you > > are done with it it no longer works. > > > > Exactly the kind of attitude that killed libav, killing FFmpeg now. > > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > -- > Vittorio > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > What about subtitles support, they are also very broken. When subtitles support will be removed soonTM?