| Message ID | 20190320185333.GA5021@akuma.local |
|---|---|
| State | New |
| Headers | show |
Hi all! On 2019-03-20 19:56 +0100, Alexander Strasser wrote: > I had found that when capturing video for some hours from > USB Camera-B4.09.24.1 (Manufacturer: OmniVision Technologies, Inc.), > sometimes when invoking the ioctl DQBUF, the returned buffer is not > filled with the size we expect for the raw video frame. > > Here are two examples for such occurrences: > > [video4linux2,v4l2 @ 0x258b440] Dequeued v4l2 buffer contains 609596 bytes, but 614400 were expected. Flags: 0x00000001. > /dev/video1: Invalid data found when processing input > > [video4linux2,v4l2 @ 0x225f440] Dequeued v4l2 buffer contains 609508 bytes, but 614400 were expected. Flags: 0x00000001. > /dev/video1: Invalid data found when processing input > > The ffmpeg CLI tool will then stop capturing and exit. > > To avoid this, return FFERROR_REDO instead. I have not seen the > issue appearing twice or more often in a row. It was more like > one single error every two hours. After thinking about the FFERROR_REDO approach some more, I think it is not ideal in the case where a driver permanently delivers frames of unexpected size. The FFmpeg lib call would not return to the client, potentially freezing the running program. So maybe the zero-sized package approach, like we do with the corrupted frames, is more robust here? Any opinions? Best regards, Alexander > > While searching for the above quoted error message I found a ticket > that mentions this type of problem on our tracker. > > Fixes #4795 > --- > libavdevice/v4l2.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/libavdevice/v4l2.c b/libavdevice/v4l2.c > index 1b9c6e7..5a1b324 100644 > --- a/libavdevice/v4l2.c > +++ b/libavdevice/v4l2.c > @@ -542,7 +542,7 @@ static int mmap_read_frame(AVFormatContext *ctx, AVPacket *pkt) > "Dequeued v4l2 buffer contains %d bytes, but %d were expected. Flags: 0x%08X.\n", > buf.bytesused, s->frame_size, buf.flags); > enqueue_buffer(s, &buf); > - return AVERROR_INVALIDDATA; > + return FFERROR_REDO; > } > } > > --
2019-03-20 22:26 GMT+01:00, Alexander Strasser <eclipse7@gmx.net>: > Hi all! > > On 2019-03-20 19:56 +0100, Alexander Strasser wrote: >> I had found that when capturing video for some hours from >> USB Camera-B4.09.24.1 (Manufacturer: OmniVision Technologies, Inc.), >> sometimes when invoking the ioctl DQBUF, the returned buffer is not >> filled with the size we expect for the raw video frame. >> >> Here are two examples for such occurrences: >> >> [video4linux2,v4l2 @ 0x258b440] Dequeued v4l2 buffer contains 609596 >> bytes, but 614400 were expected. Flags: 0x00000001. >> /dev/video1: Invalid data found when processing input >> >> [video4linux2,v4l2 @ 0x225f440] Dequeued v4l2 buffer contains 609508 >> bytes, but 614400 were expected. Flags: 0x00000001. >> /dev/video1: Invalid data found when processing input >> >> The ffmpeg CLI tool will then stop capturing and exit. >> >> To avoid this, return FFERROR_REDO instead. I have not seen the >> issue appearing twice or more often in a row. It was more like >> one single error every two hours. > > > After thinking about the FFERROR_REDO approach some more, I think it > is not ideal in the case where a driver permanently delivers frames > of unexpected size. The FFmpeg lib call would not return to the client, > potentially freezing the running program. > > So maybe the zero-sized package approach, like we do with the corrupted > frames, is more robust here? Any opinions? You could check if a sane frame was already received before the wrong size. Carl Eugen
On Wed, 20 Mar 2019, Carl Eugen Hoyos wrote: > 2019-03-20 22:26 GMT+01:00, Alexander Strasser <eclipse7@gmx.net>: >> Hi all! >> >> On 2019-03-20 19:56 +0100, Alexander Strasser wrote: >>> I had found that when capturing video for some hours from >>> USB Camera-B4.09.24.1 (Manufacturer: OmniVision Technologies, Inc.), >>> sometimes when invoking the ioctl DQBUF, the returned buffer is not >>> filled with the size we expect for the raw video frame. >>> >>> Here are two examples for such occurrences: >>> >>> [video4linux2,v4l2 @ 0x258b440] Dequeued v4l2 buffer contains 609596 >>> bytes, but 614400 were expected. Flags: 0x00000001. >>> /dev/video1: Invalid data found when processing input >>> >>> [video4linux2,v4l2 @ 0x225f440] Dequeued v4l2 buffer contains 609508 >>> bytes, but 614400 were expected. Flags: 0x00000001. >>> /dev/video1: Invalid data found when processing input >>> >>> The ffmpeg CLI tool will then stop capturing and exit. >>> >>> To avoid this, return FFERROR_REDO instead. I have not seen the >>> issue appearing twice or more often in a row. It was more like >>> one single error every two hours. >> >> >> After thinking about the FFERROR_REDO approach some more, I think it >> is not ideal in the case where a driver permanently delivers frames >> of unexpected size. The FFmpeg lib call would not return to the client, >> potentially freezing the running program. >> >> So maybe the zero-sized package approach, like we do with the corrupted >> frames, is more robust here? Any opinions? > > You could check if a sane frame was already received before the > wrong size. Maybe just set the packet corrupt flag and log an error, but return the partial buffer? Regards, Marton
Marton Balint (12019-03-21): > Maybe just set the packet corrupt flag and log an error, but return the > partial buffer? I think it may be the best solution. But it would be interesting to know how the tools react to it. Regards,
Hi Marton, hi Nicolas! On 2019-03-21 10:07 +0100, Nicolas George wrote: > Marton Balint (12019-03-21): > > Maybe just set the packet corrupt flag and log an error, but return the > > partial buffer? > > I think it may be the best solution. But it would be interesting to know > how the tools react to it. I agree that it might be the most sophisticated solution. But it is not so clear how this would work out with ffmpeg and all other client apps. It might also be more tricky/involved to implement correctly. Personally I don't want to invest much more time into this ATM. I would really like to achieve to prevent aborting the capture in case of a frame once in a while being returned too short. I believe it would be a big win that can hopefully be implemented without causing harm. After considering the 3 variants I have implemented so far I am now at the following conclusion: 1. Return AVERROR(EAGAIN) => this is not an option because it is not expected when using blocking mode 2. Return FFERROR_REDO => this could be used, but has the not completely unlikely downside to not return to the caller for a very long time. That is in the case the driver starts emitting too short frames in a row for a long time. 3. Return zero-sized packets => This works and is consistent with how we handle frames flagged to be corrupted (V4L2_BUF_FLAG_ERROR). See commit 28f20d2ff487aa589643d8f70eaf614b78839685 . Choosing from the above 3 options, IMHO 3 is the best. If someone decides to improve it at a later point, one could improve the V4L2_BUF_FLAG_ERROR case too. Do you think that going with 3 now is fine? If yes I would send that patch for review. Thanks for your comments! Alexander
diff --git a/libavdevice/v4l2.c b/libavdevice/v4l2.c index 1b9c6e7..5a1b324 100644 --- a/libavdevice/v4l2.c +++ b/libavdevice/v4l2.c @@ -542,7 +542,7 @@ static int mmap_read_frame(AVFormatContext *ctx, AVPacket *pkt) "Dequeued v4l2 buffer contains %d bytes, but %d were expected. Flags: 0x%08X.\n", buf.bytesused, s->frame_size, buf.flags); enqueue_buffer(s, &buf); - return AVERROR_INVALIDDATA; + return FFERROR_REDO; } }
I had found that when capturing video for some hours from USB Camera-B4.09.24.1 (Manufacturer: OmniVision Technologies, Inc.), sometimes when invoking the ioctl DQBUF, the returned buffer is not filled with the size we expect for the raw video frame. Here are two examples for such occurrences: [video4linux2,v4l2 @ 0x258b440] Dequeued v4l2 buffer contains 609596 bytes, but 614400 were expected. Flags: 0x00000001. /dev/video1: Invalid data found when processing input [video4linux2,v4l2 @ 0x225f440] Dequeued v4l2 buffer contains 609508 bytes, but 614400 were expected. Flags: 0x00000001. /dev/video1: Invalid data found when processing input The ffmpeg CLI tool will then stop capturing and exit. To avoid this, return FFERROR_REDO instead. I have not seen the issue appearing twice or more often in a row. It was more like one single error every two hours. While searching for the above quoted error message I found a ticket that mentions this type of problem on our tracker. Fixes #4795 --- libavdevice/v4l2.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --