diff mbox series

[FFmpeg-devel,v2,1/1] lavc/qsvdec: fix dead loop of qsv decoding

Message ID OSZP286MB214217936FF2CDDD56722971D5E7A@OSZP286MB2142.JPNP286.PROD.OUTLOOK.COM
State New
Headers show
Series [FFmpeg-devel,v2,1/1] lavc/qsvdec: fix dead loop of qsv decoding | expand

Checks

Context Check Description
yinshiyou/make_loongarch64 success Make finished
yinshiyou/make_fate_loongarch64 success Make fate finished
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished

Commit Message

Ting Hu Aug. 29, 2023, 7:49 a.m. UTC 
From: tinghu3 <siriushu@hotmail.com>

MFXVideoDECODE_DecodeFrameAsync always return MFX_WRN_DEVICE_BUSY in special scenario.

Signed-off-by: tinghu3 <siriushu@hotmail.com>
---
 libavcodec/qsvdec.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

Comments

Ting Hu Aug. 31, 2023, 1:34 a.m. UTC | #1 
I encountered this problem in video conference scenario. Receive more over 9 RTP h264 video streams simultaneously.

获取 Outlook for iOS<https://aka.ms/o0ukef>
________________________________
发件人: Anton Khirnov <anton@khirnov.net>
发送时间: Tuesday, August 29, 2023 3:57:00 PM
收件人: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
抄送: tinghu3 <siriushu@hotmail.com>
主题: Re: [FFmpeg-devel] [PATCH v2 1/1] lavc/qsvdec: fix dead loop of qsv decoding

Quoting Ting Hu (2023-08-29 09:49:06)
> From: tinghu3 <siriushu@hotmail.com>
>
> MFXVideoDECODE_DecodeFrameAsync always return MFX_WRN_DEVICE_BUSY in special scenario.

"special scenario"? What special scenario? Is this documented anywhere?
This seems like a bug in libmfx/libvpl.

--
Anton Khirnov
Xiang, Haihao Sept. 14, 2023, 6:11 a.m. UTC | #2 
> This seems like a bug in libmfx/libvpl.

I agree there should be a bug in libmfx/libvpl, but we have to avoid this dead
loop in FFmpeg. 

> I encountered this problem in video conference scenario. Receive more over 9
> RTP h264 video streams simultaneously.

Could you file a bug to https://github.com/oneapi-src/oneVPL-intel-gpu , then
document this issue in your patch ? 

Thanks
Haihao

> 
> 获取 Outlook for iOS<https://aka.ms/o0ukef>
> ________________________________
> 发件人: Anton Khirnov <anton@khirnov.net>
> 发送时间: Tuesday, August 29, 2023 3:57:00 PM
> 收件人: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> 抄送: tinghu3 <siriushu@hotmail.com>
> 主题: Re: [FFmpeg-devel] [PATCH v2 1/1] lavc/qsvdec: fix dead loop of qsv
> decoding
> 
> Quoting Ting Hu (2023-08-29 09:49:06)
> > From: tinghu3 <siriushu@hotmail.com>
> > 
> > MFXVideoDECODE_DecodeFrameAsync always return MFX_WRN_DEVICE_BUSY in special
> > scenario.
> 
> "special scenario"? What special scenario? Is this documented anywhere?
> This seems like a bug in libmfx/libvpl.
> 
> --
> Anton Khirnov
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Zhao Zhili Sept. 14, 2023, 7:08 a.m. UTC | #3 
> On Sep 14, 2023, at 14:11, Xiang, Haihao <haihao.xiang-at-intel.com@ffmpeg.org> wrote:
> 
> 
>> This seems like a bug in libmfx/libvpl.
> 
> I agree there should be a bug in libmfx/libvpl, but we have to avoid this dead
> loop in FFmpeg.

Is it possible that a normal case which do take more than 500ms now break out
early and return AVERROR(EBUSY)?

On the other hand, this isn’t the only place where dead loops can happen.
Third party encoder/decoder wrappers, especially those operating system framework
wrappers (e.g., Android MediaCodec) have the same risks. Something like a
interrupt callback in avcodec can be useful if more cases have been found. 

> 
> 
>> I encountered this problem in video conference scenario. Receive more over 9
>> RTP h264 video streams simultaneously.
> 
> Could you file a bug to https://github.com/oneapi-src/oneVPL-intel-gpu , then
> document this issue in your patch ? 
> 
> Thanks
> Haihao
> 
>> 
>> 获取 Outlook for iOS<https://aka.ms/o0ukef>
>> ________________________________
>> 发件人: Anton Khirnov <anton@khirnov.net>
>> 发送时间: Tuesday, August 29, 2023 3:57:00 PM
>> 收件人: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
>> 抄送: tinghu3 <siriushu@hotmail.com>
>> 主题: Re: [FFmpeg-devel] [PATCH v2 1/1] lavc/qsvdec: fix dead loop of qsv
>> decoding
>> 
>> Quoting Ting Hu (2023-08-29 09:49:06)
>>> From: tinghu3 <siriushu@hotmail.com>
>>> 
>>> MFXVideoDECODE_DecodeFrameAsync always return MFX_WRN_DEVICE_BUSY in special
>>> scenario.
>> 
>> "special scenario"? What special scenario? Is this documented anywhere?
>> This seems like a bug in libmfx/libvpl.
>> 
>> --
>> Anton Khirnov
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> 
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Ting Hu Sept. 15, 2023, 6:36 a.m. UTC | #4 
Ok, I will file a bug ASAP.

Best regards
Sirius
________________________________
发件人: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> 代表 Xiang, Haihao <haihao.xiang-at-intel.com@ffmpeg.org>
发送时间: Thursday, September 14, 2023 2:11:52 PM
收件人: anton@khirnov.net <anton@khirnov.net>; ffmpeg-devel@ffmpeg.org <ffmpeg-devel@ffmpeg.org>
主题: Re: [FFmpeg-devel] [PATCH v2 1/1] lavc/qsvdec: fix dead loop of qsv decoding


> This seems like a bug in libmfx/libvpl.

I agree there should be a bug in libmfx/libvpl, but we have to avoid this dead
loop in FFmpeg.

> I encountered this problem in video conference scenario. Receive more over 9
> RTP h264 video streams simultaneously.

Could you file a bug to https://github.com/oneapi-src/oneVPL-intel-gpu , then
document this issue in your patch ?

Thanks
Haihao

>
> 获取 Outlook for iOS<https://aka.ms/o0ukef>
> ________________________________
> 发件人: Anton Khirnov <anton@khirnov.net>
> 发送时间: Tuesday, August 29, 2023 3:57:00 PM
> 收件人: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> 抄送: tinghu3 <siriushu@hotmail.com>
> 主题: Re: [FFmpeg-devel] [PATCH v2 1/1] lavc/qsvdec: fix dead loop of qsv
> decoding
>
> Quoting Ting Hu (2023-08-29 09:49:06)
> > From: tinghu3 <siriushu@hotmail.com>
> >
> > MFXVideoDECODE_DecodeFrameAsync always return MFX_WRN_DEVICE_BUSY in special
> > scenario.
>
> "special scenario"? What special scenario? Is this documented anywhere?
> This seems like a bug in libmfx/libvpl.
>
> --
> Anton Khirnov
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Xiang, Haihao Oct. 16, 2023, 8:26 a.m. UTC | #5 
On Do, 2023-09-14 at 15:08 +0800, zhilizhao(赵志立) wrote:
> 
> 
> > On Sep 14, 2023, at 14:11, Xiang, Haihao
> > <haihao.xiang-at-intel.com@ffmpeg.org> wrote:
> > 
> > 
> > > This seems like a bug in libmfx/libvpl.
> > 
> > I agree there should be a bug in libmfx/libvpl, but we have to avoid this
> > dead
> > loop in FFmpeg.
> 
> Is it possible that a normal case which do take more than 500ms now break out
> early and return AVERROR(EBUSY)?

Thanks for your question. Yes, it is possible. So we can't use this way to fix
the problem. 

BRs
Haihao

> 
> On the other hand, this isn’t the only place where dead loops can happen.
> Third party encoder/decoder wrappers, especially those operating system
> framework
> wrappers (e.g., Android MediaCodec) have the same risks. Something like a
> interrupt callback in avcodec can be useful if more cases have been found. 
> 
> > 
> > 
> > > I encountered this problem in video conference scenario. Receive more over
> > > 9
> > > RTP h264 video streams simultaneously.
> > 
> > Could you file a bug to https://github.com/oneapi-src/oneVPL-intel-gpu ,
> > then
> > document this issue in your patch ? 
> > 
> > Thanks
> > Haihao
> > 
> > > 
> > > 获取 Outlook for iOS<https://aka.ms/o0ukef>
> > > ________________________________
> > > 发件人: Anton Khirnov <anton@khirnov.net>
> > > 发送时间: Tuesday, August 29, 2023 3:57:00 PM
> > > 收件人: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> > > 抄送: tinghu3 <siriushu@hotmail.com>
> > > 主题: Re: [FFmpeg-devel] [PATCH v2 1/1] lavc/qsvdec: fix dead loop of qsv
> > > decoding
> > > 
> > > Quoting Ting Hu (2023-08-29 09:49:06)
> > > > From: tinghu3 <siriushu@hotmail.com>
> > > > 
> > > > MFXVideoDECODE_DecodeFrameAsync always return MFX_WRN_DEVICE_BUSY in
> > > > special
> > > > scenario.
> > > 
> > > "special scenario"? What special scenario? Is this documented anywhere?
> > > This seems like a bug in libmfx/libvpl.
> > > 
> > > --
> > > Anton Khirnov
> > > _______________________________________________
> > > ffmpeg-devel mailing list
> > > ffmpeg-devel@ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > 
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > 
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff mbox series

Patch

diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c
index da700f25e9..6286612ce6 100644
--- a/libavcodec/qsvdec.c
+++ b/libavcodec/qsvdec.c
@@ -703,6 +703,7 @@  static int qsv_decode(AVCodecContext *avctx, QSVContext *q,
     mfxSyncPoint *sync;
     mfxBitstream bs = { { { 0 } } };
     int ret;
+    int max_count = 0;
 
     if (avpkt->size) {
         bs.Data       = avpkt->data;
@@ -728,9 +729,18 @@  static int qsv_decode(AVCodecContext *avctx, QSVContext *q,
 
         ret = MFXVideoDECODE_DecodeFrameAsync(q->session, avpkt->size ? &bs : NULL,
                                               insurf, &outsurf, sync);
-        if (ret == MFX_WRN_DEVICE_BUSY)
+        if (ret == MFX_WRN_DEVICE_BUSY) {
+            /* Check the max wait time 500ms to avoid dead loop */
+            if (++max_count == 1000) {
+                av_log(avctx, AV_LOG_ERROR,
+                        "MFX decoder returns device busy that reachs timeout 500ms \n");
+                av_freep(&sync);
+                return AVERROR(EBUSY);
+            }
             av_usleep(500);
-
+        } else if (ret == MFX_ERR_MORE_SURFACE) {
+            max_count = 0;
+        }
     } while (ret == MFX_WRN_DEVICE_BUSY || ret == MFX_ERR_MORE_SURFACE);
 
     if (ret == MFX_ERR_INCOMPATIBLE_VIDEO_PARAM) {