From patchwork Tue Dec 21 12:12:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 32780 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp5360594iog; Tue, 21 Dec 2021 04:13:40 -0800 (PST) X-Google-Smtp-Source: ABdhPJzIIPVlGFKshSHpwt0cTb/U6gNznLHki2DooEx5eELvvmVB7DvDbVqEzirshA2AmT82f+91 X-Received: by 2002:aa7:d957:: with SMTP id l23mr2958032eds.116.1640088820375; Tue, 21 Dec 2021 04:13:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1640088820; cv=none; d=google.com; s=arc-20160816; b=keunslx7mBvidNPcQYl1FPoVELK3UPkZwBSWcuiudYSF2F7M7XNnHWot/KFMvNn6Sv cZGZRBG2f0JLOrjBKfLHKgkavmFBi7mKHTTMYfRw0DoFa57W1TY8pIgWIRR/OyrPseA/ w2g7UZ3FzWsj7kyATf5dGOe5zJ1UHNHRV75D/FN56Ql595WDdeL6C8aQoRAe2tDEsabI odcxphfIOku0eaZgMG5XcihZBDCpDO5O40H7lzMYhJVuIvE5mmmOf3ipos8GOVbswr6K J9MadZVExs7xIYr/Qkv/KsaeRFzYOcd3D4T/hyPMZolhqOPnAuvX1QRXlb7Hs7nOl/K7 L9Lg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=FvjjeTXn/ZrJRmlwOsosC8ne9WQHJ1SgG/qOy+hTado=; b=p74bRG1SHsOxdhLS7NFN2D/zqsM+kyEiodis5dbtIiTdNfcshcIR8lu1sHW6F2mAg4 s2I2uIGs1IQMcKzNEHc90vbOAWLfHaoFlynwwafsi9JcbGCmEUEvaBF3U0U+RxHeFDF+ wpOoeQQ2VteoYgo7bM/dsJmqRgMOzpLCOC1YXQ+/ey20OHhd1wGVvbViFgFAAyIsTSgg NN3BWKDOZeot6w+QtsuUs/4D4QxpzYMIxLY4Uj0ocAwTJ0uoE2/WukQucZzdm3a3FzBL 1plhDaEm8xUZgJia7X099JnBMI0St/E8+rahlkAZ1vD1a89QRou/Fu1ckdXoWHxGUc1w L+zQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=DUl+NLMb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id nb7si14314936ejc.685.2021.12.21.04.13.27; Tue, 21 Dec 2021 04:13:40 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=DUl+NLMb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A0A8068AEFD; Tue, 21 Dec 2021 14:13:14 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com [209.85.167.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3C8E468AEE9 for ; Tue, 21 Dec 2021 14:13:08 +0200 (EET) Received: by mail-lf1-f50.google.com with SMTP id bq20so27810031lfb.4 for ; Tue, 21 Dec 2021 04:13:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4bcGWxKvvhbdpCAlfdAHzJCUilMj1GVE3tthIhcqlAk=; b=DUl+NLMbMBfFhX8tdl9Se6LwgczjPVNaRxt/uUX2B9xEF+9c+cYXX3c5zhzLPOf1t+ 34c3s0R6aiVLDUoyL1n3LzlrkyTj0AxDSFK+8G/TLfIff3R7WiLh32kNCkUOCZrKJqxW vR8Y3al7ehtZZ5TOd0vNtvIsZ/gRdUp2GFujsfVY2ZYJTEovQ72vh7qaqyiIB5WV+olo IguhV2AMVHTsW0NvIc3ld02FOsCoBrvbRTH+hjzduY3H0r7d7MEF/gNqty2qlA0kYXW8 0LxU4uqVHOWaSriL3pF2l7whJCzn1hHqAaMpB0It8/elJV3+hsLHVIv8VuKqSxysZRge 4jxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4bcGWxKvvhbdpCAlfdAHzJCUilMj1GVE3tthIhcqlAk=; b=lWeoMBbgcfUGypylEH4FvvlrMd7r/0tXcpn+K6p6mCxp6qHlEXKuCyWhaw9U8M7ojA j/SKjwghUk9yqnMXbnijXy2WGVHEVRldl3/wYBem3BfQ2iGZDrQ/BXlSUISK39YRBv/X rMgaHwCCVtM2rx1A6bd8xkmwAbN0vVl0hA5Qaa0oCBq7+1Qs5HXvULTNDDPkb+RrwgCY JMJ1LmMuu1JAYfX0jPazuBIOU7BVo/VCgfR3FeEv9W0HUS9wnPokWBMny0eTfrgbADee LKxF3n+galjNIrGo5+con1J2SHnnxWSEGhnG8euAWIsZJweXjBGlIhkgXZCVY0nblyMc qfbQ== X-Gm-Message-State: AOAM530Ys5p1+IyLJHH+stFkbiLsH7wWqLyfkK77z/JEGwoWjUmqn1dY F2H8Rsb/x8+kcWmpsnArsb+FbpsymVRRzw== X-Received: by 2002:ac2:59c8:: with SMTP id x8mr2772632lfn.573.1640088787255; Tue, 21 Dec 2021 04:13:07 -0800 (PST) Received: from localhost.localdomain ([196.244.192.13]) by smtp.gmail.com with ESMTPSA id c20sm2724671ljf.37.2021.12.21.04.13.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Dec 2021 04:13:06 -0800 (PST) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Tue, 21 Dec 2021 13:12:29 +0100 Message-Id: <20211221121239.1201-3-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20211221121239.1201-1-dcnieho@gmail.com> References: <20211221121239.1201-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v6 02/12] avdevice/dshow: implement option to use device video timestamps X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: F3RXxYWyf0qt The dshow avdevice ignores timestamps for video frames provided by the DirectShow device, instead using wallclock time, apparently because the implementer of this code had a device that provided unreliable timestamps. Me (and others) would like to use the device's timestamps. The new use_video_device_timestamps option for dshow device enables them to do so. Since the majority of video devices out there probably provide fine timestamps, this patch sets the default to using the device timestamps, which means best fidelity timestamps are used by default. Using the new option, the user can switch this off and revert to the old behavior, so a fall back remains available in case the device provides broken timestamps. add use_video_device_timestamps to docs. Closes: #8620 Signed-off-by: Diederick Niehorster --- doc/indevs.texi | 6 ++++++ libavdevice/dshow.c | 1 + libavdevice/dshow_capture.h | 1 + libavdevice/dshow_pin.c | 11 ++++++----- 4 files changed, 14 insertions(+), 5 deletions(-) diff --git a/doc/indevs.texi b/doc/indevs.texi index 5be647f70a..9d8020311a 100644 --- a/doc/indevs.texi +++ b/doc/indevs.texi @@ -625,6 +625,12 @@ Save the currently used video capture filter device and its parameters (if the filter supports it) to a file. If a file with the same name exists it will be overwritten. +@item use_video_device_timestamps +If set to @option{false}, the timestamp for video frames will be +derived from the wallclock instead of the timestamp provided by +the capture device. This allows working around devices that +provide unreliable timestamps. + @end table @subsection Examples diff --git a/libavdevice/dshow.c b/libavdevice/dshow.c index cc0bef0474..5e6eb9c85d 100644 --- a/libavdevice/dshow.c +++ b/libavdevice/dshow.c @@ -1310,6 +1310,7 @@ static const AVOption options[] = { { "audio_device_save", "save audio capture filter device (and properties) to file", OFFSET(audio_filter_save_file), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, DEC }, { "video_device_load", "load video capture filter device (and properties) from file", OFFSET(video_filter_load_file), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, DEC }, { "video_device_save", "save video capture filter device (and properties) to file", OFFSET(video_filter_save_file), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, DEC }, + { "use_video_device_timestamps", "use device instead of wallclock timestamps for video frames", OFFSET(use_video_device_timestamps), AV_OPT_TYPE_BOOL, {.i64 = 1}, 0, 1, DEC }, { NULL }, }; diff --git a/libavdevice/dshow_capture.h b/libavdevice/dshow_capture.h index 06ded2ba96..5a2691518c 100644 --- a/libavdevice/dshow_capture.h +++ b/libavdevice/dshow_capture.h @@ -312,6 +312,7 @@ struct dshow_ctx { char *audio_filter_save_file; char *video_filter_load_file; char *video_filter_save_file; + int use_video_device_timestamps; IBaseFilter *device_filter[2]; IPin *device_pin[2]; diff --git a/libavdevice/dshow_pin.c b/libavdevice/dshow_pin.c index 3dae405e65..8e56dccbfe 100644 --- a/libavdevice/dshow_pin.c +++ b/libavdevice/dshow_pin.c @@ -309,10 +309,14 @@ long ff_dshow_meminputpin_Receive(DShowMemInputPin *this, IMediaSample *sample) if (!sample) return E_POINTER; + priv_data = pin->filter->priv_data; + s = priv_data; + ctx = s->priv_data; + IMediaSample_GetTime(sample, &orig_curtime, &dummy); orig_curtime += pin->filter->start_time; IReferenceClock_GetTime(clock, &graphtime); - if (devtype == VideoDevice) { + if (devtype == VideoDevice && !ctx->use_video_device_timestamps) { /* PTS from video devices is unreliable. */ IReferenceClock_GetTime(clock, &curtime); } else { @@ -322,7 +326,7 @@ long ff_dshow_meminputpin_Receive(DShowMemInputPin *this, IMediaSample *sample) like 437650244077016960 which FFmpeg doesn't like. TODO figure out math. For now just drop them. */ av_log(NULL, AV_LOG_DEBUG, - "dshow dropping initial (or ending) audio frame with odd PTS too high %"PRId64"\n", curtime); + "dshow dropping initial (or ending) frame with odd PTS too high %"PRId64"\n", curtime); return S_OK; } curtime += pin->filter->start_time; @@ -330,9 +334,6 @@ long ff_dshow_meminputpin_Receive(DShowMemInputPin *this, IMediaSample *sample) buf_size = IMediaSample_GetActualDataLength(sample); IMediaSample_GetPointer(sample, &buf); - priv_data = pin->filter->priv_data; - s = priv_data; - ctx = s->priv_data; index = pin->filter->stream_index; av_log(NULL, AV_LOG_VERBOSE, "dshow passing through packet of type %s size %8d "