From patchwork Sat Sep 10 08:22:57 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Oliver Collyer X-Patchwork-Id: 523 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.140.134 with SMTP id o128csp688326vsd; Sat, 10 Sep 2016 01:23:13 -0700 (PDT) X-Received: by 10.194.191.162 with SMTP id gz2mr6296383wjc.182.1473495793405; Sat, 10 Sep 2016 01:23:13 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id lu4si6578451wjb.101.2016.09.10.01.23.12; Sat, 10 Sep 2016 01:23:13 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@mac.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE dis=NONE) header.from=mac.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F2DED689F0E; Sat, 10 Sep 2016 11:22:58 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from pv33p04im-asmtp001.me.com (pv33p04im-asmtp001.me.com [17.143.181.10]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 82035689ECF for ; Sat, 10 Sep 2016 11:22:52 +0300 (EEST) Received: from process-dkim-sign-daemon.pv33p04im-asmtp001.me.com by pv33p04im-asmtp001.me.com (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016)) id <0ODA00A004LEJ000@pv33p04im-asmtp001.me.com> for ffmpeg-devel@ffmpeg.org; Sat, 10 Sep 2016 08:23:01 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mac.com; s=4d515a; t=1473495781; bh=Q7JPLfPWQm6VgyyEc33n7Iv00COcep5fowOGuMFiM9U=; h=Content-type:MIME-version:Subject:From:Date:Message-id:To; b=uZ7bL79OHmkNgQG46iV9/ZXe4kFExnbIC1FL3B6SfaUtngFwtJyvI7tYJOQ4s0br3 2z0XmoAOlBJBUf2w7tpp61PNyXsWlzhneuNJH+r1h/aI3RSvleI3oFxiCNZlvL4x15 b6CYVujPRLobNfGjDkPl4QdiP58Kwon38m2SFX21YXLVwORrhID25XNPCxOBbTANEV S6NBWEnjlsqRxLUrhkfyvAXUyogSum77Pr1atKLm/xzNAUBRdt91W91yNGj4xGbSqZ A0vuJ1TTwU/VtlKpabPY7RRJcNlRH5DrDVIrO4ttm2+PGmCViK6vXBD+Y+PNp0DkKj B7P6J7PDanMOQ== Received: from ovs-laptop.ov (88.233.73.222.dynamic.ttnet.com.tr [88.233.73.222]) by pv33p04im-asmtp001.me.com (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016)) with ESMTPSA id <0ODA00LB24M9PQ30@pv33p04im-asmtp001.me.com> for ffmpeg-devel@ffmpeg.org; Sat, 10 Sep 2016 08:23:00 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-09-09_14:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1015 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1603290000 definitions=main-1609100116 MIME-version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Oliver Collyer In-reply-to: Date: Sat, 10 Sep 2016 11:22:57 +0300 Message-id: <9444DC78-7A1D-4016-B6FF-9BF96F4579CE@mac.com> References: To: FFmpeg development discussions and patches X-Mailer: Apple Mail (2.3124) Subject: Re: [FFmpeg-devel] Patches related to bug 4988 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" So I’ve spent some more time investigating this issue. It actually affects the two USB capture devices I have, the Magewell XI100DUSB-HDMI and the Inogeni DVI2USB. Each of them suffers similar (but not identical) problems with corrupted buffers at the start of the capture. One workaround is to issue “modprobe -r uvcvideo && modprobe uvcvideo” to re-load the uvcvideo module each time before running FFmpeg. This works for both devices and the captures then proceed as normal. Alternatively, below is a patch for FFmpeg that also deals with the problem. Currently when ffmpeg encounters the error flag it just sets the buffers “bytesused” to zero and carries on as normal. The trouble with this is that these corrupted buffers have a zero timestamp so it leads to FFmpeg spewing out warnings for the duration of the capture once the non-corrupted buffers arrive with normal timestamps, due to the difference between them. So the first thing the patch does is discard the corrupted buffers rather than attempting to use them. I think this is a more sensible approach anyway - it seems counterproductive to try and handle a buffer as normal that we have already been told is corrupt. Unfortunately this doesn’t completely solve the problem because even if we discard the corrupted buffers it seems that for a short time afterwards the valid buffers don’t arrive at the usual regular intervals, so you can still get many warnings inside FFmpeg due to the initial irregular timestamps. Both devices exhibit the same behaviour and it reproduces on different machines. So the other thing the patch does is ignore a fixed number of valid buffers following the last corrupted one. I found 3-4 buffers is usually enough to ignore, but to be on the safe side I have set this to 8. This solves the issues and prevents any more warnings, but I don’t really think hardcoding a number of buffers to ignore is really a clean solution is it? Does anyone have any better ideas? Maybe this behaviour that ignores buffers should be an argument? Maybe it should only do this at the start of the capture and not after corrupted buffers encountered once valid buffers have been received? As mentioned in my earlier message I had also written a patch adding an option “-timestamps discard” to discard the timestamps altogether as another solution but this seemed like overkill as once the capture has “settled down” all the timestamps are just fine. I’ve also been trying to get the underlying issue fixed by the v4l2/linuxtv developers, and also attempted to involve the manufacturers of these devices, but nobody has really been able to look at it properly yet. Anyway, patch below. diff --git a/libavdevice/v4l2.c b/libavdevice/v4l2.c old mode 100644 new mode 100755 index ddf331d..7b4a826 --- a/libavdevice/v4l2.c +++ b/libavdevice/v4l2.c @@ -79,6 +79,7 @@ struct video_data { int buffers; volatile int buffers_queued; + int buffers_ignore; void **buf_start; unsigned int *buf_len; char *standard; @@ -519,7 +520,9 @@ static int mmap_read_frame(AVFormatContext *ctx, AVPacket *pkt) av_log(ctx, AV_LOG_WARNING, "Dequeued v4l2 buffer contains corrupted data (%d bytes).\n", buf.bytesused); - buf.bytesused = 0; + s->buffers_ignore = 8; + enqueue_buffer(s, &buf); + return FFERROR_REDO; } else #endif { @@ -529,14 +532,28 @@ static int mmap_read_frame(AVFormatContext *ctx, AVPacket *pkt) s->frame_size = buf.bytesused; if (s->frame_size > 0 && buf.bytesused != s->frame_size) { - av_log(ctx, AV_LOG_ERROR, + av_log(ctx, AV_LOG_WARNING, "Dequeued v4l2 buffer contains %d bytes, but %d were expected. Flags: 0x%08X.\n", buf.bytesused, s->frame_size, buf.flags); + s->buffers_ignore = 8; enqueue_buffer(s, &buf); - return AVERROR_INVALIDDATA; + return FFERROR_REDO; } } + + /* if we just encounted some corrupted buffers then we ignore the next few + * legitimate buffers because they can arrive at irregular intervals, causing + * the timestamps of the input and output streams to be out-of-sync and FFmpeg + * to continually emit warnings. */ + if (s->buffers_ignore) { + av_log(ctx, AV_LOG_WARNING, + "Ignoring dequeued v4l2 buffer due to earlier corruption.\n"); + s->buffers_ignore --; + enqueue_buffer(s, &buf); + return FFERROR_REDO; + } + /* Image is at s->buff_start[buf.index] */ if (avpriv_atomic_int_get(&s->buffers_queued) == FFMAX(s->buffers / 8, 1)) { /* when we start getting low on queued buffers, fall back on copying data */ @@ -608,6 +625,7 @@ static int mmap_start(AVFormatContext *ctx) } } s->buffers_queued = s->buffers; + s->buffers_ignore = 0; type = V4L2_BUF_TYPE_VIDEO_CAPTURE; if (v4l2_ioctl(s->fd, VIDIOC_STREAMON, &type) < 0) {