| Message ID | AM7PR03MB66602D9804F6511C194637DC8F709@AM7PR03MB6660.eurprd03.prod.outlook.com |
|---|---|
| Headers | show |
| Series | Switch mmaldec to receive_frame API | expand |
Built and tested successfully on a Rpi 3b+ through mpv. Thanks for this patchset! On Thu, Dec 9, 2021 at 9:01 PM Andreas Rheinhardt <andreas.rheinhardt@outlook.com> wrote: > > This is an updated and extended version of [1] by Ho Ming Shun > together with some additions by me. Notice that I am unable > to test these patches (not even compilation), so I hope that > others (Ho Ming Shun) do so. > > [1]: https://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285992.html > > Andreas Rheinhardt (2): > avcodec/mmaldec: Avoid creating unnecessary reference, simplify code > avcodec/mmaldec: Deduplicate AVClasses > > Ho Ming Shun (3): > avcodec/mmaldec: use decoupled dataflow > avcodec/mmaldec: re-use AVPacket for extra_data > avcodec/mmaldec: fix pointer type warning > > libavcodec/mmaldec.c | 81 +++++++++++++++++++++++--------------------- > 1 file changed, 42 insertions(+), 39 deletions(-) > > -- > 2.32.0 > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
While we are on the topic of mmaldec, would you mind reviewing and applying this patch as well: https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210916134652.22781-1-cyph1984@gmail.com/ On Fri, Dec 10, 2021 at 11:48 AM Ming Shun Ho <cyph1984@gmail.com> wrote: > > Built and tested successfully on a Rpi 3b+ through mpv. > > Thanks for this patchset! > > On Thu, Dec 9, 2021 at 9:01 PM Andreas Rheinhardt > <andreas.rheinhardt@outlook.com> wrote: > > > > This is an updated and extended version of [1] by Ho Ming Shun > > together with some additions by me. Notice that I am unable > > to test these patches (not even compilation), so I hope that > > others (Ho Ming Shun) do so. > > > > [1]: https://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285992.html > > > > Andreas Rheinhardt (2): > > avcodec/mmaldec: Avoid creating unnecessary reference, simplify code > > avcodec/mmaldec: Deduplicate AVClasses > > > > Ho Ming Shun (3): > > avcodec/mmaldec: use decoupled dataflow > > avcodec/mmaldec: re-use AVPacket for extra_data > > avcodec/mmaldec: fix pointer type warning > > > > libavcodec/mmaldec.c | 81 +++++++++++++++++++++++--------------------- > > 1 file changed, 42 insertions(+), 39 deletions(-) > > > > -- > > 2.32.0 > > > > _______________________________________________ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > To unsubscribe, visit link above, or email > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On 12/9/2021 7:01 AM, Andreas Rheinhardt wrote: > This is an updated and extended version of [1] by Ho Ming Shun > together with some additions by me. Notice that I am unable > to test these patches (not even compilation), so I hope that > others (Ho Ming Shun) do so. > > [1]: https://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285992.html > > Andreas Rheinhardt (2): > avcodec/mmaldec: Avoid creating unnecessary reference, simplify code > avcodec/mmaldec: Deduplicate AVClasses > > Ho Ming Shun (3): > avcodec/mmaldec: use decoupled dataflow > avcodec/mmaldec: re-use AVPacket for extra_data > avcodec/mmaldec: fix pointer type warning > > libavcodec/mmaldec.c | 81 +++++++++++++++++++++++--------------------- > 1 file changed, 42 insertions(+), 39 deletions(-) > This is fantastic work! I'm seeing significantly improved decode latency here. I've been carrying an out-of-tree hack[0] for years to address the extremely poor performance of mmaldec due to this async decoding behavior. The refactoring in this patch set _almost_ matches my dirty hack. With a realtime 60 FPS stream (so 1 packet per 16 ms) via Moonlight-Qt[1]: - Current master: ~250 ms - This patch set: ~18 ms - My hack: ~14 ms With this patchset, I usually end up receiving frame N-1 when submitting frame N. My hack manages to avoid asynchronous decoding altogether, so I'm always receiving frame N after sending frame N for decoding. The absolute latency numbers are very close, but going over a frame of latency is significant. The effective latency difference when it comes to drawing the resulting image is 16 ms on a 60 Hz monitor due to V-sync. I assume my hack is unsuitable for inclusion in FFmpeg, but perhaps we can figure out a way to avoid having to patch FFmpeg to get that behavior. Maybe an AVOption to set MAX_DELAYED_FRAMES rather than hardcoding it? In any case, this is a _major_ improvement for my use so feel free to add my Tested-by to this patch series: Tested-by: Cameron Gutman <aicommander@gmail.com> Regards, Cam [0]: https://github.com/cgutman/FFmpeg/commit/193a0320afd5f316da407208eb2fdceea888ff64 [1]: https://github.com/moonlight-stream/moonlight-qt
It would definitely have to be an option. I tried lowering MAX_DELAYED_FRAMES myself. While it might work for live video streams, games streaming in your case, RTSP from CCTV cameras in mine, lower MAX_DELAYED_FRAMES seems to cause a decoding stall in any video with re-ordered frames. Thanks for testing this patchset! On Sun, Dec 12, 2021 at 9:09 AM Cameron Gutman <aicommander@gmail.com> wrote: > On 12/9/2021 7:01 AM, Andreas Rheinhardt wrote: > > This is an updated and extended version of [1] by Ho Ming Shun > > together with some additions by me. Notice that I am unable > > to test these patches (not even compilation), so I hope that > > others (Ho Ming Shun) do so. > > > > [1]: > https://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285992.html > > > > Andreas Rheinhardt (2): > > avcodec/mmaldec: Avoid creating unnecessary reference, simplify code > > avcodec/mmaldec: Deduplicate AVClasses > > > > Ho Ming Shun (3): > > avcodec/mmaldec: use decoupled dataflow > > avcodec/mmaldec: re-use AVPacket for extra_data > > avcodec/mmaldec: fix pointer type warning > > > > libavcodec/mmaldec.c | 81 +++++++++++++++++++++++--------------------- > > 1 file changed, 42 insertions(+), 39 deletions(-) > > > > This is fantastic work! I'm seeing significantly improved decode latency > here. I've been carrying an out-of-tree hack[0] for years to address the > extremely poor performance of mmaldec due to this async decoding behavior. > > The refactoring in this patch set _almost_ matches my dirty hack. > > With a realtime 60 FPS stream (so 1 packet per 16 ms) via Moonlight-Qt[1]: > - Current master: ~250 ms > - This patch set: ~18 ms > - My hack: ~14 ms > > With this patchset, I usually end up receiving frame N-1 when submitting > frame N. My hack manages to avoid asynchronous decoding altogether, so > I'm always receiving frame N after sending frame N for decoding. > > The absolute latency numbers are very close, but going over a frame of > latency is significant. The effective latency difference when it comes to > drawing the resulting image is 16 ms on a 60 Hz monitor due to V-sync. > > I assume my hack is unsuitable for inclusion in FFmpeg, but perhaps we > can figure out a way to avoid having to patch FFmpeg to get that behavior. > Maybe an AVOption to set MAX_DELAYED_FRAMES rather than hardcoding it? > > In any case, this is a _major_ improvement for my use so feel free to add > my Tested-by to this patch series: > > Tested-by: Cameron Gutman <aicommander@gmail.com> > > > > Regards, > Cam > > > [0]: > https://github.com/cgutman/FFmpeg/commit/193a0320afd5f316da407208eb2fdceea888ff64 > [1]: https://github.com/moonlight-stream/moonlight-qt >