From patchwork Mon Dec 13 07:21:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 32403 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp5072322iog; Sun, 12 Dec 2021 23:22:23 -0800 (PST) X-Google-Smtp-Source: ABdhPJy+x27sshiCflrGAgeBMyt5RlslLgERkRgEzjajGYdat6gOEpdYYf5kpgX3DeJnvE54N+Au X-Received: by 2002:a17:906:4c95:: with SMTP id q21mr41776574eju.485.1639380143012; Sun, 12 Dec 2021 23:22:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639380143; cv=none; d=google.com; s=arc-20160816; b=kovyLax4vLk/mCSGNFyJPHzi7BXZ8PquGPredBmP3v2qxcFMjeHz2Wy6Ee+mRMnP7x MPJhKHb51vIKmVOIDoWe/PfI0cvVUeo5ZVqyDpCh9Si5E1BVILUhG5kD+yOm24R//HbX nPdXuOqpScLaYKCO57x60bLO+WGu9pVOyGM1OmQpU6qtmE2n9LlxrPHOR6H9ajdNdlGv Ny60XjI1gWKg+LVBgPAc04pGP/KxCFjT+CFg0KnkkxPe6Vxdfm02fnI3E3pVWH2mY+wZ /GpMFTIJj7H4eZE7kh2plOf2cv5cH5A6XsdzptdNb/d/su7lXak9m2w+JpcyB0nJCVV7 GyXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=m/s5EPvZLYo4YA+rc8/KGNmrJHguYkJqkJTYyTZicHs=; b=slgUZBS6UL2lBNj3PWgbBvhLzlsbCoDzXXxOtqCuX478TMxVIPcSXimpQt1QEomTqM pOhqheRJFPFqn99xkC8EpLONpLpuR83gwqfnaUyktEj8yGiKqlYhrJfOojc6X8brpTLx 0t7Kd6dUikQU3KNRrVEx4xZq8iKewobEBxr4w6OaFV1HAJxLq/6KIvkFjwJX+m32eGUw +OcHa3F+EbNPam96A6yYTJWgk6sMLFZKiwXGTn0NHygOgN86VvnZrwtVG0GWaSjzwD1o kS6krsEk/nCgbKcUgKMJdiLP/a1vY3g+91Zxvd67rE8Dc4WizM2Ev6u7hS5RgcrhObwX 5ajQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=PE6pyY1C; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ga42si20962841ejc.319.2021.12.12.23.22.22; Sun, 12 Dec 2021 23:22:23 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=PE6pyY1C; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DA6BD68AF0A; Mon, 13 Dec 2021 09:22:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f182.google.com (mail-lj1-f182.google.com [209.85.208.182]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 85CD8680979 for ; Mon, 13 Dec 2021 09:22:10 +0200 (EET) Received: by mail-lj1-f182.google.com with SMTP id 13so22266166ljj.11 for ; Sun, 12 Dec 2021 23:22:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=n4ECdPBbt8xqX/bDaPPPd4LEZ+alPN7BkD4xWH+ML2s=; b=PE6pyY1CiNHG5t6d1BOBGN0fLtWQc/vpa3rFu3NzTIvH+X1gc5Xhwsckj4m1VSIooL dXf0fSaErIPmBSJL8gPk3EMmCTBfiCF7S2mCEHPX4oxSyj2LyL9m6e7xsHhd26LXT24s KEt2Lpz3l8fjIpExjWJ8B3xmR4OKOIn1Yx+Zz2vXz1Jq8lpzJs7yauLYaqPe/izmSSzP EMTQoynEa1/Y+lTkPOj+pnc8lC4lY69SN3oewORv/ZvqnVpTyCXyj8EKOOp2dUL7gQG9 0Nd6AyE4ZetTapBVDdZUvwpr/wLPtd8waf/9Zk0Ug+zDaEDKSeDV7I9Rejr2XCa6WJgv VUAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=n4ECdPBbt8xqX/bDaPPPd4LEZ+alPN7BkD4xWH+ML2s=; b=SuEZ7/3uJY3Ad4vC5D9u3zggjZXF3QSbpz8QHgblIYUXFUZjscYcTAcKKz1hFYfKYH K3zEsoRdIpxacL7fU30BazbEPqDg6lIZRz/NK7JDks48mzs4us/D36uj5lX+L1PDGtaa iAL9vNqq+Y7ql6QoJHdkWnNXu+n6/lpEvcrb5hnK5E1uiyVjGn1unc5isk9AqhGv2Hd6 xYsQ71LVZ/sRki6qQOvtyn5plgkIge4XPC083ll/26S8vB/FBtmW+VqmYd7kmEB0tJsZ FVop0t+LmYMhqw8egOdkQWSc74zI1Dlm1llC9coWY+4CErhkuG5HU/mo2gICddB3K506 nOJw== X-Gm-Message-State: AOAM532jqeWCR3N3xqbhBfonbFWu0Jq0n7WY6eKAc8jVjwW3iKtxAn4H nHfLk9lSUNJAUnfeLKvj//hDwoDT0jcjGA== X-Received: by 2002:a05:651c:2119:: with SMTP id a25mr28216533ljq.131.1639380129514; Sun, 12 Dec 2021 23:22:09 -0800 (PST) Received: from localhost.localdomain (84-217-56-54.customers.ownit.se. [84.217.56.54]) by smtp.gmail.com with ESMTPSA id t7sm1342357lfl.260.2021.12.12.23.22.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Dec 2021 23:22:09 -0800 (PST) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Mon, 13 Dec 2021 08:21:31 +0100 Message-Id: <20211213072143.993-2-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20211213072143.993-1-dcnieho@gmail.com> References: <20211213072143.993-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 01/13] avdevice/dshow: prevent NULL access X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: xFk0NpfcHfUg list_options true would crash when both a video and an audio device were specified as input. Crash would occur on line 784 because ctx->device_unique_name[otherDevType] would be NULL Signed-off-by: Diederick Niehorster --- libavdevice/dshow.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/libavdevice/dshow.c b/libavdevice/dshow.c index ef78781865..cc0bef0474 100644 --- a/libavdevice/dshow.c +++ b/libavdevice/dshow.c @@ -708,9 +708,9 @@ dshow_list_device_options(AVFormatContext *avctx, ICreateDevEnum *devenum, if ((r = dshow_cycle_devices(avctx, devenum, devtype, sourcetype, &device_filter, &device_unique_name)) < 0) return r; ctx->device_filter[devtype] = device_filter; + ctx->device_unique_name[devtype] = device_unique_name; if ((r = dshow_cycle_pins(avctx, devtype, sourcetype, device_filter, NULL)) < 0) return r; - av_freep(&device_unique_name); return 0; } @@ -1143,6 +1143,7 @@ static int dshow_read_header(AVFormatContext *avctx) } } } + // don't exit yet, allow it to list crossbar options in dshow_open_device } if (ctx->device_name[VideoDevice]) { if ((r = dshow_open_device(avctx, devenum, VideoDevice, VideoSourceDevice)) < 0 || From patchwork Mon Dec 13 07:21:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 32404 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp5072445iog; Sun, 12 Dec 2021 23:22:32 -0800 (PST) X-Google-Smtp-Source: ABdhPJy1xHXuOogH0mHuMkBZNe6/9sqXDL/OozBnfDDqwsztWTOZoreM6faH9OQ3yH2reKMCM3cf X-Received: by 2002:aa7:ca46:: with SMTP id j6mr62393080edt.234.1639380151960; Sun, 12 Dec 2021 23:22:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639380151; cv=none; d=google.com; s=arc-20160816; b=02v/pkiTku17XtJVbWYAn38AbDQs0XAuV33+Ci8yfqi0b83i0axorNmEJxXlAFMvO3 gtKPxmqNlhQ9brx4ymT0MUBWGdisNWk8+sSUlm+/qBQpI7g9ojJ3WIPz7ucHJtIYs6Y9 to0mB1Fi5SuBL/c9xOBGiB/uJVj+s9W8T58R0v79nV0MoKKlM1rVJ3GDrnKXtrD6OsxM 7oXg+WODw7u3vq1HtYvgoFHtv+zVXi9YUmCAwhtipzkamQVIEkrg/mi+XpBn8YYQziYS TvJaXqYMER/XZYbuBpyyBObnluR2qhPNNvbWZT7C2bkfE375LsDYYrLZ6rAPnqg/3jlM yrLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=cb6qFZCRmw9uKI83TutOZzOW3dr94jHYFbzqbF3Z6KM=; b=Jux1kLPShDG3lRqDj/QxDtKm1FUyLsKqWtK2HZh8Ftx7fMdMc5ExDRri1ug1tj+MrI fka/jefcLekDF5FNJQWKOoHJClYg23meDGylmKzgQ23PTjPdwvX9RyDYDybd4EpJujbl 2IDF9TdNG8S6sQhsmUMIRYlwcWlkFceAKOVGGmL28PTf4cLxnKTTJPqbRhbBvybSTa8z On9d25GMHn/zNvChDFAcXour0Ct64OU1yVKJUFiHVx6efp5ypYpxEAKwzFgD9CSf/oEW 5hDycKsFeA++7QCMovRRz+PcZdTxCfRwZoMp8m4JwwNdz0dLcZh0NYqR+GJGAJSdgAOM 5hnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=l0evdDdG; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hu8si17979540ejc.322.2021.12.12.23.22.31; Sun, 12 Dec 2021 23:22:31 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=l0evdDdG; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F32CB68AEEA; Mon, 13 Dec 2021 09:22:23 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f52.google.com (mail-lf1-f52.google.com [209.85.167.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id ECF0C68AEEA for ; Mon, 13 Dec 2021 09:22:16 +0200 (EET) Received: by mail-lf1-f52.google.com with SMTP id d10so29067581lfg.6 for ; Sun, 12 Dec 2021 23:22:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5pFUjJjSlGSp8OHwT/JcqzdN1IRZqm+bZ2QkppFpUzM=; b=l0evdDdGc1bdYFOq6n01Ptiw1vntdBUoK55PuoEyN1x/ppAe63mi2nJxC38hip0OR6 dQxiRBcc/grR4pVjcftNudahOHbZ7wZAMEFFpkcn+daF24qD6q5BcoMdTz8CD1FPVQid rGEutzMpZjoGm71bFTBblbUB8de9Vj2mwN1MU5mCpGUmrQX9+YXipMG3eu6UeeGwQuvd iV0Bhb46XllU9/Vl9E0PFknoBnNBqx8dWrGI0D4lxH63BSwDVVsiarTOkofiaJRrkER7 9t9UbqHZ6EcaUfS2xqwfpTvtRHcNLgMjxJW/J4cN/mFPxkao4cVQX/BHJXaqMu0bUUtD qkCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5pFUjJjSlGSp8OHwT/JcqzdN1IRZqm+bZ2QkppFpUzM=; b=uSucDI1PVni7r7ofqy1pk1R+xt06SJIMZMrs59gJi9yuKvepUFDuWDRgAi6SEy+QAD Nrt2i2ZVDzI6Hk0aWwkm6Gxgn22Phd81z3ACIMG5mhGsR2Ov/DEChsgQIBJZc2zL6hjv loW7XR7ncI/cSjRbXnuUZpFCtnClyRFrO9Rumhca1ZCk/1q/Z/MHUlHqw4Wg25I2E3A/ eqRL9llyH+MpJEZ9FEs9YttMTDXcZBUe0iTKKZw8haA8TpVhldnXN4U9457cqmka7muz /DWM3He9l+sQm80UFol6Y68u1J8pGrvt/Gpr5xNF1zq7iuseBBfeJ/szHgP2S7RIANKT 0ufA== X-Gm-Message-State: AOAM532E6kAjzpHsJXj2nSNANTbsPXiqtv2Lp7HLY/tGBtu39QdUYx2x nutmMBfogN6XQg0jjYPyqPThb98kISDN7A== X-Received: by 2002:a05:6512:23a4:: with SMTP id c36mr27663942lfv.634.1639380135897; Sun, 12 Dec 2021 23:22:15 -0800 (PST) Received: from localhost.localdomain (84-217-56-54.customers.ownit.se. [84.217.56.54]) by smtp.gmail.com with ESMTPSA id f17sm1313501lfk.145.2021.12.12.23.22.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Dec 2021 23:22:15 -0800 (PST) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Mon, 13 Dec 2021 08:21:32 +0100 Message-Id: <20211213072143.993-3-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20211213072143.993-1-dcnieho@gmail.com> References: <20211213072143.993-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 02/13] avdevice/dshow: implement option to use device video timestamps X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: qPgRCvSgU+H0 The dshow avdevice ignores timestamps for video frames provided by the DirectShow device, instead using wallclock time, apparently because the implementer of this code had a device that provided unreliable timestamps. Me (and others) would like to use the device's timestamps. The new use_video_device_timestamps option for dshow device enables them to do so. Since the majority of video devices out there probably provide fine timestamps, this patch sets the default to using the device timestamps, which means best fidelity timestamps are used by default. Using the new option, the user can switch this off and revert to the old behavior, so a fall back remains available in case the device provides broken timestamps. Closes: #8620 Signed-off-by: Diederick Niehorster --- libavdevice/dshow.c | 1 + libavdevice/dshow_capture.h | 1 + libavdevice/dshow_pin.c | 11 ++++++----- 3 files changed, 8 insertions(+), 5 deletions(-) diff --git a/libavdevice/dshow.c b/libavdevice/dshow.c index cc0bef0474..5e6eb9c85d 100644 --- a/libavdevice/dshow.c +++ b/libavdevice/dshow.c @@ -1310,6 +1310,7 @@ static const AVOption options[] = { { "audio_device_save", "save audio capture filter device (and properties) to file", OFFSET(audio_filter_save_file), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, DEC }, { "video_device_load", "load video capture filter device (and properties) from file", OFFSET(video_filter_load_file), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, DEC }, { "video_device_save", "save video capture filter device (and properties) to file", OFFSET(video_filter_save_file), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, DEC }, + { "use_video_device_timestamps", "use device instead of wallclock timestamps for video frames", OFFSET(use_video_device_timestamps), AV_OPT_TYPE_BOOL, {.i64 = 1}, 0, 1, DEC }, { NULL }, }; diff --git a/libavdevice/dshow_capture.h b/libavdevice/dshow_capture.h index 06ded2ba96..5a2691518c 100644 --- a/libavdevice/dshow_capture.h +++ b/libavdevice/dshow_capture.h @@ -312,6 +312,7 @@ struct dshow_ctx { char *audio_filter_save_file; char *video_filter_load_file; char *video_filter_save_file; + int use_video_device_timestamps; IBaseFilter *device_filter[2]; IPin *device_pin[2]; diff --git a/libavdevice/dshow_pin.c b/libavdevice/dshow_pin.c index 3dae405e65..8e56dccbfe 100644 --- a/libavdevice/dshow_pin.c +++ b/libavdevice/dshow_pin.c @@ -309,10 +309,14 @@ long ff_dshow_meminputpin_Receive(DShowMemInputPin *this, IMediaSample *sample) if (!sample) return E_POINTER; + priv_data = pin->filter->priv_data; + s = priv_data; + ctx = s->priv_data; + IMediaSample_GetTime(sample, &orig_curtime, &dummy); orig_curtime += pin->filter->start_time; IReferenceClock_GetTime(clock, &graphtime); - if (devtype == VideoDevice) { + if (devtype == VideoDevice && !ctx->use_video_device_timestamps) { /* PTS from video devices is unreliable. */ IReferenceClock_GetTime(clock, &curtime); } else { @@ -322,7 +326,7 @@ long ff_dshow_meminputpin_Receive(DShowMemInputPin *this, IMediaSample *sample) like 437650244077016960 which FFmpeg doesn't like. TODO figure out math. For now just drop them. */ av_log(NULL, AV_LOG_DEBUG, - "dshow dropping initial (or ending) audio frame with odd PTS too high %"PRId64"\n", curtime); + "dshow dropping initial (or ending) frame with odd PTS too high %"PRId64"\n", curtime); return S_OK; } curtime += pin->filter->start_time; @@ -330,9 +334,6 @@ long ff_dshow_meminputpin_Receive(DShowMemInputPin *this, IMediaSample *sample) buf_size = IMediaSample_GetActualDataLength(sample); IMediaSample_GetPointer(sample, &buf); - priv_data = pin->filter->priv_data; - s = priv_data; - ctx = s->priv_data; index = pin->filter->stream_index; av_log(NULL, AV_LOG_VERBOSE, "dshow passing through packet of type %s size %8d " From patchwork Mon Dec 13 07:21:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 32405 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp5072595iog; Sun, 12 Dec 2021 23:22:42 -0800 (PST) X-Google-Smtp-Source: ABdhPJy2mCk4khyepgJbEJ0OCO362NDFWelthPDU+6oXg4JB1k8NxoCeXEYKKpixz5Ix4k3LHLit X-Received: by 2002:a05:6402:2686:: with SMTP id w6mr61475532edd.141.1639380162108; Sun, 12 Dec 2021 23:22:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639380162; cv=none; d=google.com; s=arc-20160816; b=o3jiQeHGBmXVnGNS/SoKPmUtux6f0bkV2R0yNyC6nMsKAcBN1mQZENDs+dqjcKZ+Lu v2dlBQhwpe72jxc0o+SsD0KV/rvgiAVGdSwvpXOLUhAxfVbk4Q/IVleifXcZAEXVP3XF TVe8NkJAjA6f2RS0tXqdm7IM4wISCKtF3GHDhz+NIPL7KH+5B7J9JkX4tcgDJAPWJaya vXpsKW5EI2TjKWJTlZuaugyJcseTFjOQTWf7wXpLhNpR4z/Su6ZOYecsOIdNK7rnh9+u TzPzLg22dxoIr2v0PVTdSjlpcldYabS57B2GMQCxO7SS3Fe8o1A+JzLzlCZU4XWBWFGp J9Mg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=ww9zrGVs3Sf3WtAuexO6dUed3d83/nuLkVjfn9hvYc8=; b=glUiqsqD6Ek/l9RBRSwzP2Fq+XiiF2fHwgz3dRAjSj+FE2h1HGm25lPDxQ6kIlSmVk /Wv6d0HogddtedNqcvIj2Cnc832sWigFYNtsXNiTyl4Mnl+3POSnwRWSmU2r7FV7NE3v aBbNLYJlj5R27RtPSEZ5h8sitw2SRAV1ygzcsXQJQGGcQ9tSaOUK+RPQIf7sg0Yxo72O s+RsjlEpfbOTZ5TWyO6JezB/962JFU5/ONCltGzCNIl18qbucCQnrKO82VtTVI3W092F 3HuV9i86f17Pt7RRZhEBLAWlEzlTzd3ynntRTK780ZN2+IX54wBO6AUcq/sLA4Cr83Xd 08jA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=G4vvr0DO; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s2si15149646edx.402.2021.12.12.23.22.41; Sun, 12 Dec 2021 23:22:42 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=G4vvr0DO; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E8A5B68AF2A; Mon, 13 Dec 2021 09:22:29 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f52.google.com (mail-lf1-f52.google.com [209.85.167.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 296EE68807F for ; Mon, 13 Dec 2021 09:22:23 +0200 (EET) Received: by mail-lf1-f52.google.com with SMTP id t26so29068876lfk.9 for ; Sun, 12 Dec 2021 23:22:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dnt8sXpAh/1gJkyMC/XJOEO2KpkwOeytqXo40qew90g=; b=G4vvr0DOVRgxjAono1fwh9pLnqBfe6IhKosKnGGOLXR1AKDPaB288SdmCJYRWpMqOP PPhg9nyPQPHPqz4gfKi3nhZ0CurNrps4ew/0DK83waNtykrznQbf/UsUhCnlfgEWz4aE oaScUIrHY3wPGCzz6McWtCjoxD8IwIjAwzMF4X7kXUmmGvGxjB3JzfFGfaUyqfjRWf0P h3R45UH/D/4bOd9FXTlGcJso2TaMvIK4GXMC+CI0qLbtcLykBiw94aoRuc3UmLRsNO9x +7ZMQkJA3exAS3DADvYW6DFXQiPMXBQKUhy8knMZs6oDIomuKdE9ErmBScKNUyM84arz O+8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dnt8sXpAh/1gJkyMC/XJOEO2KpkwOeytqXo40qew90g=; b=7rC2IxMB+W4lEDA63/otlDKT+YK3pOHVfX8HAt1wKuM8q0Vrg5EPzLw2h9RwEsyUJv x04DFR+pOuOWBa2l8I/Zmsw1fa9xn5hUouZo00mISdgJlYzx3HgbnaLqsCh61t5VJbEb O9OyozpKHRwwsqmyGXvswUSPS4wxeq16DirCEnAtG+98q1dp/kGDHtW6h07WPFt7+uqz vWAS/ypdiIHYOLB4srlUzPqwQXEB2VSklMt22jPf5IhEUUcITkQZdawxdINRQsk/p0yt UGF8+VorvQnJwC6jU2LoTGDAW/5ljH52wnKPQUzgJESzNMZJ5YaqM8tHOrwKfSPBmpYU Ja1Q== X-Gm-Message-State: AOAM533erJjYjt670tlDPMWbjuAEoYjfMA/jeYO5h0jfewwR+iaN2pLr JKM/R43GTIlac+7XDf/qZV/0y30UygUIMA== X-Received: by 2002:a05:6512:1151:: with SMTP id m17mr28075074lfg.414.1639380142327; Sun, 12 Dec 2021 23:22:22 -0800 (PST) Received: from localhost.localdomain (84-217-56-54.customers.ownit.se. [84.217.56.54]) by smtp.gmail.com with ESMTPSA id g4sm1311327lfv.288.2021.12.12.23.22.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Dec 2021 23:22:21 -0800 (PST) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Mon, 13 Dec 2021 08:21:33 +0100 Message-Id: <20211213072143.993-4-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20211213072143.993-1-dcnieho@gmail.com> References: <20211213072143.993-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 03/13] avdevice/dshow: add use_video_device_timestamps to docs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Twt9mM6hEVXT Signed-off-by: Diederick Niehorster --- doc/indevs.texi | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/doc/indevs.texi b/doc/indevs.texi index 5be647f70a..9d8020311a 100644 --- a/doc/indevs.texi +++ b/doc/indevs.texi @@ -625,6 +625,12 @@ Save the currently used video capture filter device and its parameters (if the filter supports it) to a file. If a file with the same name exists it will be overwritten. +@item use_video_device_timestamps +If set to @option{false}, the timestamp for video frames will be +derived from the wallclock instead of the timestamp provided by +the capture device. This allows working around devices that +provide unreliable timestamps. + @end table @subsection Examples From patchwork Mon Dec 13 07:21:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 32406 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp5072694iog; Sun, 12 Dec 2021 23:22:52 -0800 (PST) X-Google-Smtp-Source: ABdhPJy3NBUtYxR2D2ZECVV/L6x4jW6g4iT+Dx9Lf516Qrn9ttjBHTUtUu8lEVDz7wonqz8wkiHh X-Received: by 2002:a05:6402:11d2:: with SMTP id j18mr64523357edw.318.1639380171947; Sun, 12 Dec 2021 23:22:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639380171; cv=none; d=google.com; s=arc-20160816; b=tt7xb4G21FzK65+Tg3NIcay04fQOpCTc6fWB9UskDgPTHRacjh7R/lo0aOC7lvQfBz 7ABPa3mN7ca8sRSqIFH2uyMHPL415UqyYUxwYpo+DaBhY+LnEKXq2eflrNgphU2B/Qlx NC5qooj51kfn9nLPjmVrborVRBVULbNN9DpNWRiCgh3DNvDYTtLoIG6iuC61wdssIJTp bANSEuJ9Lm9DAo8eia5uaaZghC+bfSYwAZ7oMykDzeBOULzhClN6YStf5Pv1aWS59FIO Wc71P0Ho8Jdkk81rNidbRZKLXAmGS0f4F/lTggRS+XoFwVtEiFde35yPczW0rghMK2Uu bQ2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=pa/7AoG/Tn1IKWdkErCmAui2uj/lRNPvOKfWL9u3LsY=; b=IkAQMWZDRvT2JBsS8zGHixWDmE0nfvXgRbzBPxEJfJCE1rpKMh4beMWF2ZOaYSsyHd q/HtkDh6NT9efGWgTXdXXARbHlBG65TOkn9heX1I1S3/otUCX9TRZ1F/qIYtFKswog6j NsFizUdMCCIJjS1NyuNsKz1BsOUZWVHSOh8QyK9xvMLumflp7uvZk5rCHDGpHXMrnFGM WH2wFjMdh6i9x1feTpHHzNxOdz28jg4ImiBteR/hhUXiD9nowp65sD2ekykds4igCZzp jI9wJT5eIgQTVA7OD23+2sVDqWAJguNAUs0yXrOzXw0LU10w+MccuLGtR1hsJuLsjYXt /qzg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=XyWnIL4G; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s5si20633133edd.18.2021.12.12.23.22.51; Sun, 12 Dec 2021 23:22:51 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=XyWnIL4G; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0AE9B68AF1C; Mon, 13 Dec 2021 09:22:33 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f48.google.com (mail-lf1-f48.google.com [209.85.167.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id ED5DC68AF2C for ; Mon, 13 Dec 2021 09:22:29 +0200 (EET) Received: by mail-lf1-f48.google.com with SMTP id b40so29028032lfv.10 for ; Sun, 12 Dec 2021 23:22:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tLBGkU73btm2UY8VXqRWlX4bo6pvicPCF5lFX+SbDcM=; b=XyWnIL4GulGWLTaaor5amkXHm/ZtyURxeZ/lcQ9m6Z/7xeoG67GstM1W0Z70fYuiWm FpcjgWfkKzpW4M+COBoSiQidUydWW538bE8qUTLhsc3eC6qDwRLU/u95wrzPRgrPunzF YvVfU2D9U5ZsT4CVMXyejDWrt1Q5qEZr0Gfry1MILvDtvYiPwb4WURa6YeEKIO1UHj+i LOKKTq+aKjS9zDySoSSo5Tn2R1nC13ygI3hOvzthjTMEnDYJpNHhORzeG1usYjR9YTkY ONrpanR6lpeL5x72KOfE4sptEuKX5EBHK97EkGVvXl3SB1bw3IQ0rC7Lc0KrW8ZsBP9h xB/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tLBGkU73btm2UY8VXqRWlX4bo6pvicPCF5lFX+SbDcM=; b=JT7ea6erM3BZAP8xijLqCp0V+C2CEscYvju+iw0toQSXoKfW3yeEtLFFbeWCztB6g0 hPcGmX5arvEoeK9pEOTC4vJ/1WJieh/OOCu9mOzRCM9G0Qish4F1VhyESqqmHaVscfis CFzRFVe2vS0zR4+Xz554Z9FwCUXg/Xn08PbRagGYO0pkBMhZl8p/ZmhoQ0hfNt00N53H gA7Wujx+CPgb+mcWEd8/6ccvSBFLvOAHP8awOLn3a8MhsdsYyS8ebpiPOa6X1faEDrZw TSUPSiqBSRIf4nlWMZ4R1blrQLaSoat4ZZtMu3TwkDZR9Nrw1lsn3sC3DpsoyROOzUtj 2iLQ== X-Gm-Message-State: AOAM533LJNiQI+HEAHvlmzYPDVwc/y3gizFpw4ghyNNmyqEb4cq+ma8r 9zVREdxjviqSqwomlVs0XRMHv8ODdCJ58g== X-Received: by 2002:a05:6512:1148:: with SMTP id m8mr26943297lfg.456.1639380148722; Sun, 12 Dec 2021 23:22:28 -0800 (PST) Received: from localhost.localdomain (84-217-56-54.customers.ownit.se. [84.217.56.54]) by smtp.gmail.com with ESMTPSA id w6sm1321544lfr.11.2021.12.12.23.22.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Dec 2021 23:22:28 -0800 (PST) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Mon, 13 Dec 2021 08:21:34 +0100 Message-Id: <20211213072143.993-5-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20211213072143.993-1-dcnieho@gmail.com> References: <20211213072143.993-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 04/13] avdevice/dshow: query graph and sample time only once X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: J8si14Q4OH/n No need to query twice, use value we've already unconditionally got. Improve variable names Signed-off-by: Diederick Niehorster --- libavdevice/dshow_pin.c | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/libavdevice/dshow_pin.c b/libavdevice/dshow_pin.c index 8e56dccbfe..1d0e880480 100644 --- a/libavdevice/dshow_pin.c +++ b/libavdevice/dshow_pin.c @@ -295,9 +295,10 @@ long ff_dshow_meminputpin_Receive(DShowMemInputPin *this, IMediaSample *sample) uint8_t *buf; int buf_size; /* todo should be a long? */ int index; - int64_t curtime; - int64_t orig_curtime; + int64_t chosentime; + int64_t sampletime; int64_t graphtime; + int use_sample_time = 1; const char *devtypename = (devtype == VideoDevice) ? "video" : "audio"; IReferenceClock *clock = pin->filter->clock; int64_t dummy; @@ -313,24 +314,27 @@ long ff_dshow_meminputpin_Receive(DShowMemInputPin *this, IMediaSample *sample) s = priv_data; ctx = s->priv_data; - IMediaSample_GetTime(sample, &orig_curtime, &dummy); - orig_curtime += pin->filter->start_time; + IMediaSample_GetTime(sample, &sampletime, &dummy); IReferenceClock_GetTime(clock, &graphtime); if (devtype == VideoDevice && !ctx->use_video_device_timestamps) { /* PTS from video devices is unreliable. */ - IReferenceClock_GetTime(clock, &curtime); + chosentime = graphtime; + use_sample_time = 0; } else { - IMediaSample_GetTime(sample, &curtime, &dummy); - if(curtime > 400000000000000000LL) { + if (sampletime > 400000000000000000LL) { /* initial frames sometimes start < 0 (shown as a very large number here, - like 437650244077016960 which FFmpeg doesn't like. + like 437650244077016960 which FFmpeg doesn't like). TODO figure out math. For now just drop them. */ av_log(NULL, AV_LOG_DEBUG, - "dshow dropping initial (or ending) frame with odd PTS too high %"PRId64"\n", curtime); + "dshow dropping initial (or ending) frame with odd PTS too high %"PRId64"\n", sampletime); return S_OK; } - curtime += pin->filter->start_time; + chosentime = sampletime; } + // media sample time is relative to graph start time + sampletime += pin->filter->start_time; + if (use_sample_time) + chosentime += pin->filter->start_time; buf_size = IMediaSample_GetActualDataLength(sample); IMediaSample_GetPointer(sample, &buf); @@ -338,8 +342,8 @@ long ff_dshow_meminputpin_Receive(DShowMemInputPin *this, IMediaSample *sample) av_log(NULL, AV_LOG_VERBOSE, "dshow passing through packet of type %s size %8d " "timestamp %"PRId64" orig timestamp %"PRId64" graph timestamp %"PRId64" diff %"PRId64" %s\n", - devtypename, buf_size, curtime, orig_curtime, graphtime, graphtime - orig_curtime, ctx->device_name[devtype]); - pin->filter->callback(priv_data, index, buf, buf_size, curtime, devtype); + devtypename, buf_size, chosentime, sampletime, graphtime, graphtime - sampletime, ctx->device_name[devtype]); + pin->filter->callback(priv_data, index, buf, buf_size, chosentime, devtype); return S_OK; } From patchwork Mon Dec 13 07:21:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 32407 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp5072804iog; Sun, 12 Dec 2021 23:23:01 -0800 (PST) X-Google-Smtp-Source: ABdhPJxiTBjW9XBIWYLWyZhYj9ic9pru2w0o6NTp4dmxhpRf7TxlPSC2WpyX+6lOfGS7PGd7XTnU X-Received: by 2002:aa7:df9a:: with SMTP id b26mr60619311edy.107.1639380181449; Sun, 12 Dec 2021 23:23:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639380181; cv=none; d=google.com; s=arc-20160816; b=Uu7tH0VSTa48AQtveAoL/VNddjVwUeeu7cZkqImY0DAoFlpGetOKpMWPvxKgXGeHjm NCbrs8sW8MsuVEM8Movz+Xj+xpW3nmu+uKWeV6QvwDBki7+wdqLAwR1os0TEnR8RwpKO Kfp7UFlaH77kWIZtcTH1W61z2nB5407PvL1B8YplnlKj6ZJIA08vzv/dkJJDH0NXjEtB InTbwYuwwYc8vu9v5rQQ9d/e3gVi3osCvmflc3jkq3GFa5QXeixNvnH7KtRn+8aEolJ6 A390Rz+imU9vwILqsgBCIy0nPWuC9VU2SGi8xBYdsQrSbcvyK4ZEoxQM8rRdBaefgRux d+Ag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=1GNbKic4/hTLN8/M2EuCDzyPjx1FXZ2iUnm51nzwk7Y=; b=mZJbOBmGSFOr9JX/7zDhDe36Orp4Wxh8hCu6zihnxWPNi+/e5jRHGvHFzDKzrZAKhs fLBU6fgOlt2qSx/6OVn1DpAEhvtZ+ldDrDCP61YEmNJ5/3vt3yOjYTpqK1phPx/qYYqm VHvo5ymco4bmdUBuw/GaTO5pZU81+vDHnB7NF4wfWGwZJIshKm8rGibKR8qcFSxa6570 /wFfnVD5tTwvBpLb3+ZR7VqfFp2l76fR+L099poif3sHiyunyM1x2I6ljb319y1qxIKX j9YDmbp2ObP9a0WtXcm4vfAvgEZSru58gLERfD3MdhJOxYxGwJMTY2n2TElAcBSp+YsR iycA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=PDvXF3b+; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hd43si18255466ejc.484.2021.12.12.23.23.01; Sun, 12 Dec 2021 23:23:01 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=PDvXF3b+; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AFEFA68AF32; Mon, 13 Dec 2021 09:22:38 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f169.google.com (mail-lj1-f169.google.com [209.85.208.169]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D442368AF19 for ; Mon, 13 Dec 2021 09:22:36 +0200 (EET) Received: by mail-lj1-f169.google.com with SMTP id 207so22305974ljf.10 for ; Sun, 12 Dec 2021 23:22:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=adnbYYXt3y1rSvkqYNUbMSM3zzoFE+m4ORetAFLtEoA=; b=PDvXF3b+h8Nm5pZ+2F1JivXLErBXRjV7FoTEE7MH3N9PEt1KCjnjvo2eMklmR9seld lL3IaPJJDynuAQMYKF7oBMCpGhjGrJVXlGwA8CjekIut/c1Qdqe35wJLLviLOtgBGRhy 9E6FcfRDn7Yxv/AihqOFfzUBIDz7B4QZrDyX0d2NR2FKFvH3Oy9hFEyujyblr30oQAbp wExzKWEAHehCY5xnGSa8cNlZVOrj3eQgUZNw99WLAVV+ZqM+U7Z2GZAP9+SX5DmqjK0s mDelW2TkOusKSFT+H7OurbTWr6hSbk1HGj3/MtF2aGcPanHj+m9ISFnZjXDuUIxDFsLv e+Ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=adnbYYXt3y1rSvkqYNUbMSM3zzoFE+m4ORetAFLtEoA=; b=l6xcmW2GW9TnvcTefkjIkaL6QqyA+xAs/usJs+HXds/qlYwQQUkHWSU/F58i3FGgoJ PZcwjjrE87R+GS30ARPRvBbzNe8bo7b7CQ/hYdmYABkbu0n6XK2XHq+50mMWOfgTXNt/ J3OklCMpHk+Y90GsCT4gBCjVMUGQvtqf7LxHNDPW4hfog7qAMMmQTKGgsgfOgGPwFKWR foS/jvRebOYkwpcq8brRz+choPiSfaCrlF+/ncaYna/8FqX3OwnAciXhPVECXRp3EbLw Y/pS+g7uQFUY/BxN5hO32EBp6rdlgFwdQuBLqMX95BlkeFYKkO2FsosbBcuCHPRaSadC RnJw== X-Gm-Message-State: AOAM533uugk7ZJatWW9q3cuCN2EveWRJgyePiVT84KdWPJmNNVslcBQO 7vAAbUQPJm+us8Bpr3lp2VwVZJoVWnQiHw== X-Received: by 2002:a2e:b8d0:: with SMTP id s16mr27827416ljp.496.1639380155128; Sun, 12 Dec 2021 23:22:35 -0800 (PST) Received: from localhost.localdomain (84-217-56-54.customers.ownit.se. [84.217.56.54]) by smtp.gmail.com with ESMTPSA id bj11sm1318383ljb.62.2021.12.12.23.22.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Dec 2021 23:22:34 -0800 (PST) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Mon, 13 Dec 2021 08:21:35 +0100 Message-Id: <20211213072143.993-6-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20211213072143.993-1-dcnieho@gmail.com> References: <20211213072143.993-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 05/13] avdevice/dshow: handle unknown sample time X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ksqr+E05RpGy GetTime may return an error indication that the sample has not timestamps, or may return a NULL start time. In those cases, fall back to graph time. Better debug message in case sample dropped: could now be audio or video frame Signed-off-by: Diederick Niehorster --- libavdevice/dshow_pin.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/libavdevice/dshow_pin.c b/libavdevice/dshow_pin.c index 1d0e880480..310f48c85e 100644 --- a/libavdevice/dshow_pin.c +++ b/libavdevice/dshow_pin.c @@ -295,14 +295,15 @@ long ff_dshow_meminputpin_Receive(DShowMemInputPin *this, IMediaSample *sample) uint8_t *buf; int buf_size; /* todo should be a long? */ int index; - int64_t chosentime; - int64_t sampletime; - int64_t graphtime; + int64_t chosentime = 0; + int64_t sampletime = 0; + int64_t graphtime = 0; int use_sample_time = 1; const char *devtypename = (devtype == VideoDevice) ? "video" : "audio"; IReferenceClock *clock = pin->filter->clock; int64_t dummy; struct dshow_ctx *ctx; + HRESULT hr; dshowdebug("ff_dshow_meminputpin_Receive(%p)\n", this); @@ -314,22 +315,26 @@ long ff_dshow_meminputpin_Receive(DShowMemInputPin *this, IMediaSample *sample) s = priv_data; ctx = s->priv_data; - IMediaSample_GetTime(sample, &sampletime, &dummy); + hr = IMediaSample_GetTime(sample, &sampletime, &dummy); IReferenceClock_GetTime(clock, &graphtime); if (devtype == VideoDevice && !ctx->use_video_device_timestamps) { /* PTS from video devices is unreliable. */ chosentime = graphtime; use_sample_time = 0; } else { - if (sampletime > 400000000000000000LL) { + if (hr == VFW_E_SAMPLE_TIME_NOT_SET || sampletime == 0) { + chosentime = graphtime; + use_sample_time = 0; + } + else if (sampletime > 400000000000000000LL) { /* initial frames sometimes start < 0 (shown as a very large number here, like 437650244077016960 which FFmpeg doesn't like). TODO figure out math. For now just drop them. */ av_log(NULL, AV_LOG_DEBUG, "dshow dropping initial (or ending) frame with odd PTS too high %"PRId64"\n", sampletime); return S_OK; - } - chosentime = sampletime; + } else + chosentime = sampletime; } // media sample time is relative to graph start time sampletime += pin->filter->start_time; From patchwork Mon Dec 13 07:21:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 32408 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp5073212iog; Sun, 12 Dec 2021 23:23:32 -0800 (PST) X-Google-Smtp-Source: ABdhPJwG0dCdHsqnj7kpIOH4LNoBeP2sc4wQvp1CvDIMpEiUlg8T4HYRlHi7JZtaYxi6NMy0+Ges X-Received: by 2002:aa7:dc15:: with SMTP id b21mr63236684edu.237.1639380212156; Sun, 12 Dec 2021 23:23:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639380212; cv=none; d=google.com; s=arc-20160816; b=L1cyjllu4IXaEVJQ7f2MV9HxQvTBGzAQJ7n3PsnsUKynqzWJYkkZnN1N/LilmAd6b3 n1H+63snC0taFeS5nNcdpsLnuxQMKWrlDpGzLo31fcJzRJ16XKKh27dAFvi2hAypIzO3 CSjWcuiSGzHZwRV9kHA7w/IqWTQh+FmuksZsAwHobWzFqJMF67o2xUP3zPhrOtYplRyC gS5Xwl9Tsmf9XMYBKbE0Nn9ESsbohvx71e7DwfRTUg+olHq5Y1fjYFhgUaA4/ueYauOj Mx+4UTaGzXTWh8uqvQgq5YPS4mktXw8NHp4ZWukQBBW8Z+wjIJiVISvmUwpUvJL96HeO zyiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=P/Nb+R18r7YiYrlW0B0+ap1kvBaWBeFlqtv4rvWiICE=; b=ep0qABdxjg1h6CLbCgDLgcQXY8yyFa5UpraZH4jfw0CJBZMHC3Vj3QJ85yO+XBjl4b jnwhYZNkvaauv669+9jqNFgjuI2zkdH3nYC5NMfKhzIhcgTkt6a+LF8AAxtuZea3xUAH rTyQFgO+4pCzLTV8k7bCYt3EZclhvIlWiBeGL6+Box6DKpBkMyYhKhg5NdoATSz35Kqx +hjcD1ypnu4NgdgxSw9MW5l0hoczqMZkFuqcvbu4HhFLPyAoZ0hnJWNizj60WH5wF6Pb SoLvcRYPgajTL92hHBYf7zdzTma5ZWuaMAlj5xP9eeaE7gwa/pCAKzCArvnGD+zPoVKQ cwIQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=q6rYa5pT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cw13si18339349ejc.125.2021.12.12.23.23.31; Sun, 12 Dec 2021 23:23:32 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=q6rYa5pT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3A2D668AEFD; Mon, 13 Dec 2021 09:22:57 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f48.google.com (mail-lf1-f48.google.com [209.85.167.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5E25A68AEFD for ; Mon, 13 Dec 2021 09:22:55 +0200 (EET) Received: by mail-lf1-f48.google.com with SMTP id bi37so29096156lfb.5 for ; Sun, 12 Dec 2021 23:22:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Orw5EJxf/TzKUx8s11toOLkErxFmfQr8jVi7Kb/rKMU=; b=q6rYa5pTqk8piUnlMl6oBH4toL6Nj/BltZGQOePxQwF1F9u5nZt+A6WY2BROVhSQKn 6VcWkF3BaybpbYJa9Om2xWSS7Ms1m308CuOlKpQ0wGRdlMUMRLpDBxaP2mw323BAs7w0 WoNA/1jgN9NH29zHa0YWO2wj2SDQR0QhjVyDpQC5QhnaYwJX6RtidAUbYpXick/BaTO/ 7S2+N8530NCed3Pz8Dc9P0QlRzSiKUicCYYXKVfZWL5ip4UMfGtEgWSnotRngSPT+nBL yl5+0UuBa5Zp2PoH2xHqFgf6KqUukky0z2siVfyq85jtiMd75FEYi0vimwozztNjKYUI HzPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Orw5EJxf/TzKUx8s11toOLkErxFmfQr8jVi7Kb/rKMU=; b=dhLVb9kVgKJr5m6qtGf/QkLGVqWDidRN7xX08Z+mHPD/Z+rv5MfG+z+uKX2dcPAap9 b/IvNlQiZ0VKdZNQ/AnTz3CS5HBqyh5sgQFOD7xnaDXT+DrudVJBTwsvCiqZv+ntxAau ZpVJfZv+UxQEdCaNgUWcZel/hPyCGYealY2S5qAo9Nk8jP8+0oSu44hrSPQp5FqgxF8k aFw0G1ReSLNvePpGsx5OvyB0kmTh+OZbv6/qjH/kXHfsm6S+eIMQao3bef8APui7aWmw iT0n5T8ntbvKsH//rNQmurwuvfUYbGNTZRQZT4vACw1k8E/OF1PF5c1Bygn39RfxWT6V tJsQ== X-Gm-Message-State: AOAM531NAvL2Ffq8hCBrP4EQU2UY6tlf8XfWR1q0vdu43nj8GuYszUSs W/jD4b04Anqb75ziXhBSk5qxts2vVK9XVQ== X-Received: by 2002:a05:6512:3402:: with SMTP id i2mr27565935lfr.447.1639380174394; Sun, 12 Dec 2021 23:22:54 -0800 (PST) Received: from localhost.localdomain (84-217-56-54.customers.ownit.se. [84.217.56.54]) by smtp.gmail.com with ESMTPSA id x133sm1309366lff.282.2021.12.12.23.22.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Dec 2021 23:22:54 -0800 (PST) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Mon, 13 Dec 2021 08:21:38 +0100 Message-Id: <20211213072143.993-9-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20211213072143.993-1-dcnieho@gmail.com> References: <20211213072143.993-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 08/13] avdevice/dshow: list_devices: show media type(s) per device X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: MEdiO56YOyDy the list_devices option of dshow didn't indicate whether a specific device provides audio or video output. This patch iterates through all media formats of all pins exposed by the device to see what types it provides for capture, and prints this to the console for each device. Importantly, this now allows to find devices that provide both audio and video, and devices that provide neither. Signed-off-by: Diederick Niehorster --- libavdevice/dshow.c | 103 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 98 insertions(+), 5 deletions(-) diff --git a/libavdevice/dshow.c b/libavdevice/dshow.c index 8c257ca8e7..f25537db5c 100644 --- a/libavdevice/dshow.c +++ b/libavdevice/dshow.c @@ -197,6 +197,79 @@ fail: return; } +static void dshow_get_device_media_types(AVFormatContext *avctx, enum dshowDeviceType devtype, + enum dshowSourceFilterType sourcetype, IBaseFilter *device_filter, + enum AVMediaType **media_types, int *nb_media_types) +{ + struct dshow_ctx *ctx = avctx->priv_data; + IEnumPins *pins = 0; + IPin *pin; + int has_audio = 0, has_video = 0; + + if (IBaseFilter_EnumPins(device_filter, &pins) != S_OK) + return; + + while (IEnumPins_Next(pins, 1, &pin, NULL) == S_OK) { + IKsPropertySet *p = NULL; + PIN_INFO info = { 0 }; + GUID category; + DWORD r2; + IEnumMediaTypes *types = NULL; + AM_MEDIA_TYPE *type; + + if (IPin_QueryPinInfo(pin, &info) != S_OK) + goto next; + IBaseFilter_Release(info.pFilter); + + if (info.dir != PINDIR_OUTPUT) + goto next; + if (IPin_QueryInterface(pin, &IID_IKsPropertySet, (void **) &p) != S_OK) + goto next; + if (IKsPropertySet_Get(p, &ROPSETID_Pin, AMPROPERTY_PIN_CATEGORY, + NULL, 0, &category, sizeof(GUID), &r2) != S_OK) + goto next; + if (!IsEqualGUID(&category, &PIN_CATEGORY_CAPTURE)) + goto next; + + if (IPin_EnumMediaTypes(pin, &types) != S_OK) + goto next; + + // enumerate media types exposed by pin + // NB: don't know if a pin can expose both audio and video, check 'm all to be safe + IEnumMediaTypes_Reset(types); + while (IEnumMediaTypes_Next(types, 1, &type, NULL) == S_OK) { + if (IsEqualGUID(&type->majortype, &MEDIATYPE_Video)) { + has_video = 1; + } else if (IsEqualGUID(&type->majortype, &MEDIATYPE_Audio)) { + has_audio = 1; + } + CoTaskMemFree(type); + } + + next: + if (types) + IEnumMediaTypes_Release(types); + if (p) + IKsPropertySet_Release(p); + if (pin) + IPin_Release(pin); + } + + IEnumPins_Release(pins); + + if (has_audio || has_video) { + int nb_types = has_audio + has_video; + *media_types = av_malloc_array(nb_types, sizeof(enum AVMediaType)); + if (*media_types) { + if (has_audio) + (*media_types)[0] = AVMEDIA_TYPE_AUDIO; + if (has_video) + (*media_types)[0 + has_audio] = AVMEDIA_TYPE_VIDEO; + *nb_media_types = nb_types; + } + } +} + /** * Cycle through available devices using the device enumerator devenum, * retrieve the device with type specified by devtype and return the @@ -242,6 +315,8 @@ dshow_cycle_devices(AVFormatContext *avctx, ICreateDevEnum *devenum, LPOLESTR olestr = NULL; LPMALLOC co_malloc = NULL; AVDeviceInfo *device = NULL; + enum AVMediaType *media_types = NULL; + int nb_media_types = 0; int i; r = CoGetMalloc(1, &co_malloc); @@ -286,6 +361,12 @@ dshow_cycle_devices(AVFormatContext *avctx, ICreateDevEnum *devenum, // success, loop will end now } } else { + // get media types exposed by pins of device + if (IMoniker_BindToObject(m, 0, 0, &IID_IBaseFilter, (void* ) &device_filter) == S_OK) { + dshow_get_device_media_types(avctx, devtype, sourcetype, device_filter, &media_types, &nb_media_types); + IBaseFilter_Release(device_filter); + device_filter = NULL; + } if (device_list) { device = av_mallocz(sizeof(AVDeviceInfo)); if (!device) @@ -306,12 +387,26 @@ dshow_cycle_devices(AVFormatContext *avctx, ICreateDevEnum *devenum, device = NULL; // copied into array, make sure not freed below } else { - av_log(avctx, AV_LOG_INFO, " \"%s\"\n", friendly_name); - av_log(avctx, AV_LOG_INFO, " Alternative name \"%s\"\n", unique_name); + av_log(avctx, AV_LOG_INFO, "\"%s\"", friendly_name); + if (nb_media_types > 0 && media_types) { + const char* media_type = av_get_media_type_string(media_types[0]); + av_log(NULL, AV_LOG_INFO, " (%s", media_type ? media_type : "unknown"); + for (int i = 1; i < nb_media_types; ++i) { + media_type = av_get_media_type_string(media_types[i]); + av_log(NULL, AV_LOG_INFO, ", %s", media_type ? media_type : "unknown"); + } + av_log(NULL, AV_LOG_INFO, ")"); + } else { + av_log(NULL, AV_LOG_INFO, " (none)"); + } + av_log(avctx, AV_LOG_INFO, "\n"); + av_log(avctx, AV_LOG_INFO, " Alternative name \"%s\"\n", unique_name); } } -fail1: + fail1: + if (media_types) + av_freep(&media_types); if (device) { if (device->device_name) av_freep(&device->device_name); @@ -1186,9 +1281,7 @@ static int dshow_read_header(AVFormatContext *avctx) } if (ctx->list_devices) { - av_log(avctx, AV_LOG_INFO, "DirectShow video devices (some may be both video and audio devices)\n"); dshow_cycle_devices(avctx, devenum, VideoDevice, VideoSourceDevice, NULL, NULL, NULL); - av_log(avctx, AV_LOG_INFO, "DirectShow audio devices\n"); dshow_cycle_devices(avctx, devenum, AudioDevice, AudioSourceDevice, NULL, NULL, NULL); ret = AVERROR_EXIT; goto error; From patchwork Mon Dec 13 07:21:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 32409 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp5073346iog; Sun, 12 Dec 2021 23:23:42 -0800 (PST) X-Google-Smtp-Source: ABdhPJxijc5NSm8tGJNzEHzWarsCgs38Sxpi6e3mNL8crJa6BOkfrXFVM+nQNT0Aws2gKfa5hemi X-Received: by 2002:a05:6402:22a5:: with SMTP id cx5mr60932175edb.334.1639380222303; Sun, 12 Dec 2021 23:23:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639380222; cv=none; d=google.com; s=arc-20160816; b=CvWLgHxfEFx7wvjQhmxllyjf1GyFwkDmd9ajQku1Si8D8S4jgnzVEyz7VbKgt6Lite MYvtE7zqCOEmTO6JqHRfFJ8yrDIhZ4BQ7tAmNgI606N9S5sE+vhER2EOrfKRHSssV5eW vpQaDvWnWXHFhEFdaKTLQMXUhkSGrqwWSkIc77UcrkdAffjli1zW9bCMZZK+ShIBtVvN L08D1X8xDVCMd1YpfxcD7KtB79s+vSx0blLwL9sVpG4G1ZBtkexk7Smh9+kNotDRtXit 2LNq5Ue16t6HgKAFZGD8ws+XxlLe50XY62TlUs53BJfFiZ1DG4t5c+4wDalcI+uK2AFz q2Iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=SCl7vd8MdT/kLxTictHzeBbuzmbrvPwz4mbp5NW0t6c=; b=qB1bE9A86jeDIsWOsNB4TBINAcF90Nd5It03P24V1f1N7OWhG/F21rHNgjGq/Nsgy/ Q8H1HxH7yXOCD33chZ06CJkc5J4ssNvn7aEuQbWH4qUQqYEo98lALywkPEwGyiMlRtul uPEi3OoOtR3ZxrYzo77hhfa8jJz5+JtMDp+D7eY4s+2kw8VSfitBM1BWtPVQ3ItIKgKE jqKKJbd9Wmx6XDM9b48x0gmT8tCD0gE9t5b4qK+xseV5TBvglrUEbFnZHg02CLZaOydY sByVp5Sn/qlJ5qHmZ8cZwKSGlAGqGDRZXq2SnseVihrANp9XalSEESE4Cd9iAvRFwotQ hEBg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=USisUSEA; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id yc28si14309135ejb.200.2021.12.12.23.23.42; Sun, 12 Dec 2021 23:23:42 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=USisUSEA; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4865668AF7A; Mon, 13 Dec 2021 09:23:05 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f172.google.com (mail-lj1-f172.google.com [209.85.208.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F28F568AF76 for ; Mon, 13 Dec 2021 09:23:01 +0200 (EET) Received: by mail-lj1-f172.google.com with SMTP id i63so22371007lji.3 for ; Sun, 12 Dec 2021 23:23:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yie/XzRqV5tcCF0aofRHOPPC3ODA4BaeMNA0NuLD30k=; b=USisUSEAaz4aVYrVF0R0GOG9bC0wAkAlOrRqneuOxGUJi7fSCcpHmKgGLtXsfQnqK2 +zPztLSTqmhrQTOvUN19NgN+RF8dT6O8t+HCHL1xu4YUP/y25l8FRsCuMKvnN8YxmIco HshEWwMRH1Q6H+5rkJEeQR3FmsDVp06W639jsf7Vb55jQrpArzg1MkcOYunP9CnJoK+o /Or1749PbnbEsHKQkSGip3XJO4Z/mx2ECOHfnReKIzzY/Kx5yJ+tCcDZBnhK9zVCWSC8 jSEaaBzSVo8faW2GPxGFxO1wRuVUS3ukNuPGtS7a4d/Yb4G9royh+7IxZWwf3Gm9kpqD SKaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yie/XzRqV5tcCF0aofRHOPPC3ODA4BaeMNA0NuLD30k=; b=tVlH3kZP00aqMYY3j4ka9D080rVNqoAboof1yzC1LUeI2XNFobKJX+drrjJg6XXOkh gX2p9CL27yQkMNZgLAeidsLtq3Cl0FAU3gSmgo3prBh4Kvd5wiFAHdUcrrzgZLv3Ex2k WWmscYw2pcLv0KKnTOzSzJazryHkzpjoGxslxASMvNL4k2LL0ejsvV8WsU7x4+hHczzy nJkbTNSHlYBx+8JJ6uneqicSQ43aYP0s1dcWe2pIrldK9YWWFXjSzeRfNh8CKo+pdjAZ vfAUmhEo9AtOf/bdTUgapxxMOd2tGpWK7ds24NGVbetCHBXClbroHhQ8gy2J88AeY7xV gFMw== X-Gm-Message-State: AOAM531yRKV5qRGwWoTfUpAPpO0MIK93eY8LemxkvG44ewNQS2bRRSOA Isb439MERC+GK/ak95OAAOmNDO7YCNSBsw== X-Received: by 2002:a05:651c:1031:: with SMTP id w17mr28730331ljm.112.1639380180773; Sun, 12 Dec 2021 23:23:00 -0800 (PST) Received: from localhost.localdomain (84-217-56-54.customers.ownit.se. [84.217.56.54]) by smtp.gmail.com with ESMTPSA id br7sm1314696lfb.243.2021.12.12.23.23.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Dec 2021 23:23:00 -0800 (PST) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Mon, 13 Dec 2021 08:21:39 +0100 Message-Id: <20211213072143.993-10-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20211213072143.993-1-dcnieho@gmail.com> References: <20211213072143.993-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 09/13] avdevice: add info about media types(s) to AVDeviceInfo X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ldndSNxAbIsz An avdevice, regardless of whether its category says its an audio or video device, may provide access to devices providing different media types, or even single devices providing multiple media types. Also, some devices may provide no media types. dshow is an example encompassing all of these cases. Users should be provided with this information, so AVDeviceInfo is extended to provide it. Bump avdevice version Signed-off-by: Diederick Niehorster --- libavdevice/avdevice.c | 2 ++ libavdevice/avdevice.h | 2 ++ libavdevice/version.h | 2 +- 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/libavdevice/avdevice.c b/libavdevice/avdevice.c index 2ae26ab8e3..712ef1e80c 100644 --- a/libavdevice/avdevice.c +++ b/libavdevice/avdevice.c @@ -157,6 +157,8 @@ void avdevice_free_list_devices(AVDeviceInfoList **device_list) if (dev) { av_freep(&dev->device_name); av_freep(&dev->device_description); + if (dev->media_types) + av_freep(&dev->media_types); av_free(dev); } } diff --git a/libavdevice/avdevice.h b/libavdevice/avdevice.h index 8370bbc7f2..6f24976dcc 100644 --- a/libavdevice/avdevice.h +++ b/libavdevice/avdevice.h @@ -457,6 +457,8 @@ void avdevice_capabilities_free(AVDeviceCapabilitiesQuery **caps, AVFormatContex typedef struct AVDeviceInfo { char *device_name; /**< device name, format depends on device */ char *device_description; /**< human friendly name */ + enum AVMediaType *media_types; /**< array indicating what media types(s), if any, a device can provide. If null, cannot provide any */ + int nb_media_types; /**< length of media_types array, 0 if device cannot provide any media types */ } AVDeviceInfo; /** diff --git a/libavdevice/version.h b/libavdevice/version.h index 914e156ec7..c549768e12 100644 --- a/libavdevice/version.h +++ b/libavdevice/version.h @@ -28,7 +28,7 @@ #include "libavutil/version.h" #define LIBAVDEVICE_VERSION_MAJOR 59 -#define LIBAVDEVICE_VERSION_MINOR 0 +#define LIBAVDEVICE_VERSION_MINOR 1 #define LIBAVDEVICE_VERSION_MICRO 101 #define LIBAVDEVICE_VERSION_INT AV_VERSION_INT(LIBAVDEVICE_VERSION_MAJOR, \ From patchwork Mon Dec 13 07:21:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 32410 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp5073466iog; Sun, 12 Dec 2021 23:23:53 -0800 (PST) X-Google-Smtp-Source: ABdhPJyVd0QCBMpcUM9i7GVfHCUUtU0XUct3d2c7UI8sAu+Kc59crvxscMACcv4oDSdJER2hLyiA X-Received: by 2002:a05:6402:270c:: with SMTP id y12mr62037704edd.258.1639380233184; Sun, 12 Dec 2021 23:23:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639380233; cv=none; d=google.com; s=arc-20160816; b=wl9Aw6YC+pbQSolZtYJkvPGAEx6kt9K7LnDQdtXZKNdmZcnOdn3mvD/R7Qxf/5hhcL qHSkWfxlXKIplLHICmLJTc9jnIivXTdpWmVhIwU/9VBtl5N/3aZA99tnuo/ekOCTEKWf dA2+tGqeAWXxSx5PgUI+txIMrRcCnm7V0Cgjk1VWA8mOX5roZPbFMMciXYDnbsjHpluZ 6s+PIvJk/GH0boJv56P3ArmqfCR9p5GNaSNIzQ3Hae8cFOBtL1frAT05GbqharuKbLBp c60Teyv1g0pvxuUAk0Ientlp5qQoWWYkYq1SPlsxMAG4Z3m1CnFip9DsZEI2aRjXEXSk yHhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=I99bVLkf4f1gGNkFQ7B/WUO2BMay5a+zdoMy9Mk53xU=; b=ns+KoTnzMXykV4DAFE3Tnnp/OMgZDtxikjfOBBlC6OVBMwpdsb14wcUATXfy6rfnX0 3pUkyYI4Nf0yxRe/+vvsAuiTH0dpeH6BU6MBGah7/efPs2c+JpoPgsQ7olsdtnkvP6eO jrn/YwLIq+JPrwOtLTmWDRONAzZh3L60NS78WGCDUgw3t0QEI2PHEqmyzjxgr/lh6lVr lv9vRX7IeJ/rNdYKLTUJQQbJw+7k7zeNEtsNQNKKvShwRswGwkdSeVsR8wug6VP/DsDy RWqgPlpCb/WE037wIhWKlSJ0KEetWk5HYoCnZ2Y8qkgsnvpOAAu6oAdxvBvjGUZ+kCgr voXw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=LxPfrQyi; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id oz6si15892279ejc.728.2021.12.12.23.23.52; Sun, 12 Dec 2021 23:23:53 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=LxPfrQyi; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5A04A68AEEA; Mon, 13 Dec 2021 09:23:15 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f41.google.com (mail-lf1-f41.google.com [209.85.167.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0D7BC68AEEA for ; Mon, 13 Dec 2021 09:23:08 +0200 (EET) Received: by mail-lf1-f41.google.com with SMTP id z7so29010634lfi.11 for ; Sun, 12 Dec 2021 23:23:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BGjJKuN4tAYhDYMwIqjARqL2KwgR7BpzwzoTWujEi0U=; b=LxPfrQyiHH39/9zp+flSjBBUTmw6IS3CEqjPGtSCjKo5s6mIkvK5VE/bqAezse0BlO 1bA56MqDo9k/RdpzqpwvqvD5tFTetrp2Vh91rrY6ukPAzAdg9DK5ETbAOwtdOf6OruJT Pr0wU3mmLSN35qtl6zwe9GUhCaF1c5dFwjJuvIMBjl9yCd8/m7yEyVXbWsK0Hj8cktqy YsA6qOvTLI9R4KvCe/5x08kptnPgUUrsAHbe5GZzMAJGabovLlTUlI2MmANFaMhf4BGV mxaHqy3G64K3Mw3rJqNgnlShxXC36fEZfziwz76KuQb3rzq3cFZzxd/eLxw+kRrU7QNo q4vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BGjJKuN4tAYhDYMwIqjARqL2KwgR7BpzwzoTWujEi0U=; b=N4HOl06gx43meDCQJiJrfmXhFHJN8vsMI3VvEbeq2uBH3KZNY1fvh+IfOGw3xwRKTM xMbRtnW6uwfuMfCW74B9H9BX9Y5ieLF4L1DSlxOV+w09qTF92QVhfk5DWZJdB0Ex5Ybt aJ7o9i85wbSXMfdBqQKQr77DQP3pGFLp9bHdAALQvSl4KkSm+OetQFARudHFS78D2Ku2 kJuju31v+4LOeM+dTIxOCUjoEJ30z6+PFSUk9ssXwco6+ue7ruNiWSni4yWCspkZSq0W T1nwd+2fP0dKkS8ky3SbmA2enUCPwZiYbRgZc3twL2cs89ct5m4LqhMX2qeH20dxsnNw GdrQ== X-Gm-Message-State: AOAM532k979e+vX1mcGtQJE25zw4SyI4Cq2E3q8HPZZ6tq3t0VMYVQl9 AaR6J/zlptyT7JAHl3asJZJxEE5EPiXImA== X-Received: by 2002:ac2:5f74:: with SMTP id c20mr23101624lfc.77.1639380187152; Sun, 12 Dec 2021 23:23:07 -0800 (PST) Received: from localhost.localdomain (84-217-56-54.customers.ownit.se. [84.217.56.54]) by smtp.gmail.com with ESMTPSA id x18sm1334641ljd.110.2021.12.12.23.23.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Dec 2021 23:23:06 -0800 (PST) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Mon, 13 Dec 2021 08:21:40 +0100 Message-Id: <20211213072143.993-11-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20211213072143.993-1-dcnieho@gmail.com> References: <20211213072143.993-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 10/13] avdevice/dshow: add media type info to get_device_list X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9SZHz8QckQcw The list returned by get_device_list now contains info about what media type(s), if any, can be provided by each device. Signed-off-by: Diederick Niehorster --- libavdevice/dshow.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/libavdevice/dshow.c b/libavdevice/dshow.c index f25537db5c..fa3a06c077 100644 --- a/libavdevice/dshow.c +++ b/libavdevice/dshow.c @@ -377,6 +377,11 @@ dshow_cycle_devices(AVFormatContext *avctx, ICreateDevEnum *devenum, if (!device->device_name || !device->device_description) goto fail1; + device->nb_media_types = nb_media_types; + device->media_types = media_types; + nb_media_types = 0; + media_types = NULL; + // store to device_list output if (av_reallocp_array(&(*device_list)->devices, (*device_list)->nb_devices + 1, @@ -412,6 +417,8 @@ dshow_cycle_devices(AVFormatContext *avctx, ICreateDevEnum *devenum, av_freep(&device->device_name); if (device->device_name) av_freep(&device->device_description); + if (device->media_types) + av_freep(&device->media_types); av_free(device); } if (olestr && co_malloc) From patchwork Mon Dec 13 07:21:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 32411 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp5073597iog; Sun, 12 Dec 2021 23:24:04 -0800 (PST) X-Google-Smtp-Source: ABdhPJz/Jqd8DEbaYvo5o+6FTz2wcH7Ympk5KABdyI0WdSufVCrHQnJxzXNvt3UM4DWDxrtln8xr X-Received: by 2002:a17:907:6287:: with SMTP id nd7mr43423405ejc.152.1639380243985; Sun, 12 Dec 2021 23:24:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639380243; cv=none; d=google.com; s=arc-20160816; b=Ersm8LYWVGYfATM1Nimv5mdbgOzkkU2uwh9/IU+he20HZGFUhdBLPnxKoSfVUTThhl mSfrZz9jSO4e4Sl6lKar3SoiH1BipcVQwoHySykND5iW9eULmSUFGxfzAgmrsO+L7wzk P0Y0Bvn8mnI5O2ST4gWQBhNMLZDF4Jn8t9G49FmT4tusRCzMIxBIy/9yxpfu2FAx/0kt +etFjBnBtt2mwzGoJlpYHZf9w7Y1EQvZnAdsoeT6lmBn3A768Ruqqr6owwcDt05WmrKq 4DJv/I5nowCjCTnum/344i1jT+2/hOZoxYbQitekjbCeI9W9aCc4TZQBt3819GL1+jBo /0jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=8mq771mErKkZIbk+UMKQ0hXSxOgfULnRUfk0Kz6cxtA=; b=FZHGPwDHWowex1lxX5x/JYNiJ5bj3isd9TQIzPjRZ5S0XqCv3YykP/8kDOt3reG+1j 4iGrg2FQHgAp6gxjiZVWwhqxeWx3TvyycXtndNIiqxbsFBSlGBJif+YInpy0th1qbURu cGC6ebPLGAT1dZAit0ipIeYcymtg5tlVlLZblZwpTSDyLj64hjqUE2FVgAAALBbBHRQX 1ZJyC6Tz/wdWnLewYD/1lm3FIN8CdZpQ8Z/cJxTjlr2FxgT9DnCr61XhNSyKa7H8X4G1 RWLmwo/dqrRLprTgIRFcxmo8w6W5WgHUQi0x5MfU/gWasvt7x01jTIEDytJSrVKlZeLH uLeQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=jbps7ewh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j7si22292845edw.84.2021.12.12.23.24.03; Sun, 12 Dec 2021 23:24:03 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=jbps7ewh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6FC2068AF94; Mon, 13 Dec 2021 09:23:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f52.google.com (mail-lf1-f52.google.com [209.85.167.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6F9AE68AF2E for ; Mon, 13 Dec 2021 09:23:14 +0200 (EET) Received: by mail-lf1-f52.google.com with SMTP id d10so29071316lfg.6 for ; Sun, 12 Dec 2021 23:23:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fGYrFMIxG8sslSFASAK9bNYbz/ljJHcgjAn2T+n25bQ=; b=jbps7ewhNLXx8pkqKy417voRokBrp8SV8Q54AZMc10XAIF+MwqzEK6VPuFlIGZVrkm Hd0Nya7QPVBbYiEzZyJw47Oto8DPPEUdAkdXhGt+rG9Ol//VvGqH/o/KpwqhIdSJuLhs +fgOzHCGP1UjyAOZvuUJqHt8G1P+OgC3Uui/vd36Or3DL7IIAMvz7+Gfnv63XzqIsrty D7r01PNw4Wm08BFCgS1vDZD1k3gGKzceAeKEhnX7ESejL9pXj+/NHMfNulb2zXJa/TZA sRQ7FxiYk87VxpA2g/h/e6786kXKpIXKaFOr44ceMOnJYQZVyVwH/NXp0B9hwkbw3vsg lDCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fGYrFMIxG8sslSFASAK9bNYbz/ljJHcgjAn2T+n25bQ=; b=ALwT6uZTjiznsS3SbasMxOiGDMv9IvF5gMWZEMfXt7ybhkZ90MouNCLkDLYKAzIsBo ofn04xOZJ3gS8fs7xz+HjONirdB+HnAOhRQ9dRZ+fsiUXyHbQPPeaBpWd0pNJYN0N2S1 /lQ7oPGoA8Ls03EYWAz17B8VwdtOf21TRGNgNfMpY6XfzKk2elCqBlSPl6gBV/Ghl5+M qVdmxEluH6OIUH8oqJUhF5GkEVkeI3iGZzJB1gGd3MRe9ETcdWREvRksuOPslFdFo26T N6/sqb5FKcKc5mWweUUk7GoX+sS87GB23WZ5H14nrca5wZdF7iAZNg2Hq0YQEpWSIhM+ 0SFQ== X-Gm-Message-State: AOAM5317Eone0XLsB/cFAFwZz+a6sbzRUPEc6flh1tRtNWJURexFhDXa yFcieDuKr+zrpns9U18VeicutV4mqBAeCA== X-Received: by 2002:a05:6512:31cd:: with SMTP id j13mr27278357lfe.586.1639380193572; Sun, 12 Dec 2021 23:23:13 -0800 (PST) Received: from localhost.localdomain (84-217-56-54.customers.ownit.se. [84.217.56.54]) by smtp.gmail.com with ESMTPSA id q26sm1316670lfa.203.2021.12.12.23.23.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Dec 2021 23:23:13 -0800 (PST) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Mon, 13 Dec 2021 08:21:41 +0100 Message-Id: <20211213072143.993-12-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20211213072143.993-1-dcnieho@gmail.com> References: <20211213072143.993-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 11/13] fftools: provide media type info for devices X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: hw4DH872+iu6 fftools now print info about what media type(s), if any, are provided by sink and source avdevices. Signed-off-by: Diederick Niehorster --- fftools/cmdutils.c | 34 ++++++++++++++++++++++++---------- 1 file changed, 24 insertions(+), 10 deletions(-) diff --git a/fftools/cmdutils.c b/fftools/cmdutils.c index 3c8e5a82cd..7d7dcce2f9 100644 --- a/fftools/cmdutils.c +++ b/fftools/cmdutils.c @@ -2244,9 +2244,29 @@ double get_rotation(int32_t *displaymatrix) } #if CONFIG_AVDEVICE +static void print_device_list(const AVDeviceInfoList *device_list) +{ + // print devices + for (int i = 0; i < device_list->nb_devices; i++) { + const AVDeviceInfo *device = device_list->devices[i]; + printf("%s %s [%s] (", device_list->default_device == i ? "*" : " ", + device->device_name, device->device_description); + if (device->nb_media_types > 0 && device->media_types) { + for (int j = 0; j < device->nb_media_types; ++j) { + const char* media_type = av_get_media_type_string(device->media_types[j]); + if (j > 0) + printf(", "); + printf("%s", media_type ? media_type : "unknown"); + } + } else { + printf("none"); + } + printf(")\n"); + } +} static int print_device_sources(const AVInputFormat *fmt, AVDictionary *opts) { - int ret, i; + int ret; AVDeviceInfoList *device_list = NULL; if (!fmt || !fmt->priv_class || !AV_IS_INPUT_DEVICE(fmt->priv_class->category)) @@ -2258,10 +2278,7 @@ static int print_device_sources(const AVInputFormat *fmt, AVDictionary *opts) goto fail; } - for (i = 0; i < device_list->nb_devices; i++) { - printf("%c %s [%s]\n", device_list->default_device == i ? '*' : ' ', - device_list->devices[i]->device_name, device_list->devices[i]->device_description); - } + print_device_list(device_list); fail: avdevice_free_list_devices(&device_list); @@ -2270,7 +2287,7 @@ static int print_device_sources(const AVInputFormat *fmt, AVDictionary *opts) static int print_device_sinks(const AVOutputFormat *fmt, AVDictionary *opts) { - int ret, i; + int ret; AVDeviceInfoList *device_list = NULL; if (!fmt || !fmt->priv_class || !AV_IS_OUTPUT_DEVICE(fmt->priv_class->category)) @@ -2282,10 +2299,7 @@ static int print_device_sinks(const AVOutputFormat *fmt, AVDictionary *opts) goto fail; } - for (i = 0; i < device_list->nb_devices; i++) { - printf("%c %s [%s]\n", device_list->default_device == i ? '*' : ' ', - device_list->devices[i]->device_name, device_list->devices[i]->device_description); - } + print_device_list(device_list); fail: avdevice_free_list_devices(&device_list); From patchwork Mon Dec 13 07:21:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 32412 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp5073764iog; Sun, 12 Dec 2021 23:24:15 -0800 (PST) X-Google-Smtp-Source: ABdhPJx2OH5UTAkYj3YIJDcRLL/HBaFqlh+LSso4xs0413Zxe/fukJNWytJ0cc05cjDaDkPrTEUD X-Received: by 2002:a17:907:7d8b:: with SMTP id oz11mr43270580ejc.507.1639380255193; Sun, 12 Dec 2021 23:24:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639380255; cv=none; d=google.com; s=arc-20160816; b=TQRlTqBO0dW4nzjIGx5LTZVKO6Ea0aICz2/Rh1Q33vJfmNSH4z43XRNCW5NwNMDJoZ bwAbOmpVozuQ9kbGH+qyxCjtNwPqw/wC4FCAM3IM3HaV5n2PMpTub3/c9uYQ3gQQU4u9 rupi+/nvY9T4ZlFj5/zEw23jU5mB4W3n739csrczJx9QDrX2dMr0abHqSrmyyiJ2UnC+ JmAwggrlEyotfLn2PVHj1a3JPl4IxSdfA1wwD9eiNIQe0gRpqU/zX28qanYgf48vgbWJ 2U7nTW6Mw49tJ9g+kQDmVtDlRiRJhMH2dOFuU908zeOaNfOoRIQGEnwa271VIMwMhddM MxdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=VmdmT2kBUva7CBaXThWdkA2b6XoyFBX7UqMAzHUhEZs=; b=YhAy0QoWUwKDu94p/Ku6zSdN/rzR/D3DCD0t3riNkfFfoJPMUk6MpnqT3vZJnXR3W5 VUJavv4hB6G60JMe90Tlp4XqRPFbzxUMx+Ci6Mg7cIyHL+FNmjLJ0mH8xGcEaq5lRJma ReZbFnbQ989+W6IsNZCcxOB/ybF++fMzEskBm2LYlCt6RrE+IfGobpg7TCECeE3FgfHK TfC2+Amy1BmdoHkj0v9G4NkRoxO9StF1bidBAPxfOuV/Z8/3w+oFeabb2Q6CzSdvF7FC L9VtPa/7ILOITrO1Aovjk86S5MGfA2F5KPdNZ6Pr/Mv6WeROfsiE61o8WTXhJY/udPlv 2Y3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=pPGaXadk; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h11si17780911edb.296.2021.12.12.23.24.14; Sun, 12 Dec 2021 23:24:15 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=pPGaXadk; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5B75A68AFA1; Mon, 13 Dec 2021 09:23:23 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f41.google.com (mail-lf1-f41.google.com [209.85.167.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 073E868AF2E for ; Mon, 13 Dec 2021 09:23:20 +0200 (EET) Received: by mail-lf1-f41.google.com with SMTP id c32so29138162lfv.4 for ; Sun, 12 Dec 2021 23:23:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SCerCC5q6rwkbt6350mznQN3VDbI7ouEUK822fajwSk=; b=pPGaXadkxF8nRsTPVpILFZufsOrSaNEC5fHXr4EFhosnoBrnjqzjWDUwceVvfdoIKR BLcZpH/P2DHkO9/APOAVzMw/jk7aDsEqs+MLkHMdeU3ePFA08TLcg/wG6+tqiZu8gs0s Bo/+mtcysfGiMVxwx/P7pGc2UDaXDUC9rcJREMjdv0Bg1n0JKOrV1dBMamvzG8nyYG4i XcMlScshQsmfCTn69wxa6odDEQCw5qLr2BPjk+J8vaIEWb+nQfQNxVmkPyDOWLX6AjwG /OcD+eCA5GZ/sqc1fL344r0ODiQvMRv1FunoWLfAqufUP5B89UWcWB3PzGLhRLcHEUov tBzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SCerCC5q6rwkbt6350mznQN3VDbI7ouEUK822fajwSk=; b=e2STa7Lg6C0fyXlsX2HrPPvCTYPsdLryFTvLI6QEJIxjjISGNRdzxnXgNfp055llbc Hi0xm1yXyBbr4aIt7eWIGnsScGFVpQuVEDF9TeupCp8fZ+5XzoVjZzlprsqfs58OGbK4 Us4yoB4bQdaraFg4JXxvwHTZx6bpfX0107IwY0zLg04/bjMU0osW9zt3Ngjt6P0pMC5j vT48djG93W3/pIzRFzQJtRlmapGu590tYYvWFmFk2yPqfvnsyLVZ6oOxYq+uHVqeaKX3 EFldt/31EhqUf6nBVO4ofYQDOK8E47/ppa2ZGN5HUxSMOJQgKSZ4Uy4SXQw5Oe4maSFv GTgg== X-Gm-Message-State: AOAM533Vb1GZFMFRGD3cvpvMsOBR9ho4gOMN88pmh8uQSeQojNW2y/7q Pkaj7XmmYAXtaXTYTi5ztZiXbf/HP2lYUA== X-Received: by 2002:a05:6512:3f7:: with SMTP id n23mr28070043lfq.206.1639380200062; Sun, 12 Dec 2021 23:23:20 -0800 (PST) Received: from localhost.localdomain (84-217-56-54.customers.ownit.se. [84.217.56.54]) by smtp.gmail.com with ESMTPSA id a1sm1317638lfc.92.2021.12.12.23.23.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Dec 2021 23:23:19 -0800 (PST) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Mon, 13 Dec 2021 08:21:42 +0100 Message-Id: <20211213072143.993-13-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20211213072143.993-1-dcnieho@gmail.com> References: <20211213072143.993-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 12/13] avdevice/dshow: discover source color range/space/etc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster , Valerii Zapodovnikov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: VhMAT+oGCjLE Enabled discovering a DirectShow device's color range, space, primaries, transfer characteristics and chroma location, if the device exposes that information. Sets them in the stream's codecpars. Co-authored-by: Valerii Zapodovnikov Signed-off-by: Diederick Niehorster --- libavdevice/dshow.c | 255 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 254 insertions(+), 1 deletion(-) diff --git a/libavdevice/dshow.c b/libavdevice/dshow.c index fa3a06c077..4ad6ae102c 100644 --- a/libavdevice/dshow.c +++ b/libavdevice/dshow.c @@ -29,6 +29,31 @@ #include "libavcodec/raw.h" #include "objidl.h" #include "shlwapi.h" +// NB: technically, we should include dxva.h and use +// DXVA_ExtendedFormat, but that type is not defined in +// the MinGW headers. The DXVA2_ExtendedFormat and the +// contents of its fields is identical to +// DXVA_ExtendedFormat (see https://docs.microsoft.com/en-us/windows/win32/medfound/extended-color-information#color-space-in-media-types) +// and is provided by MinGW as well, so we use that +// instead. NB also that per the Microsoft docs, the +// lowest 8 bits of the structure, i.e. the SampleFormat +// field, contain AMCONTROL_xxx flags instead of sample +// format information, and should thus not be used. +// NB further that various values in the structure's +// fields (e.g. BT.2020 color space) are not provided +// for either of the DXVA structs, but are provided in +// the flags of the corresponding fields of Media Foundation. +// These may be provided by DirectShow devices (e.g. LAVFilters +// does so). So we use those values here too (the equivalence is +// indicated by Microsoft example code: https://docs.microsoft.com/en-us/windows/win32/api/dxva2api/ns-dxva2api-dxva2_videodesc) +typedef DWORD D3DFORMAT; // dxva2api.h header needs these types defined before include apparently in WinSDK (not MinGW). +typedef DWORD D3DPOOL; +#include "dxva2api.h" + +#ifndef AMCONTROL_COLORINFO_PRESENT +// not defined in some versions of MinGW's dvdmedia.h +# define AMCONTROL_COLORINFO_PRESENT 0x00000080 // if set, indicates DXVA color info is present in the upper (24) bits of the dwControlFlags +#endif static enum AVPixelFormat dshow_pixfmt(DWORD biCompression, WORD biBitCount) @@ -54,6 +79,192 @@ static enum AVPixelFormat dshow_pixfmt(DWORD biCompression, WORD biBitCount) return avpriv_find_pix_fmt(avpriv_get_raw_pix_fmt_tags(), biCompression); // all others } +static enum AVColorRange dshow_color_range(DXVA2_ExtendedFormat *fmt_info) +{ + switch (fmt_info->NominalRange) + { + case DXVA2_NominalRange_Unknown: + return AVCOL_RANGE_UNSPECIFIED; + case DXVA2_NominalRange_Normal: // equal to DXVA2_NominalRange_0_255 + return AVCOL_RANGE_JPEG; + case DXVA2_NominalRange_Wide: // equal to DXVA2_NominalRange_16_235 + return AVCOL_RANGE_MPEG; + case DXVA2_NominalRange_48_208: + // not an ffmpeg color range + return AVCOL_RANGE_UNSPECIFIED; + + // values from MediaFoundation SDK (mfobjects.h) + case 4: // MFNominalRange_64_127 + // not an ffmpeg color range + return AVCOL_RANGE_UNSPECIFIED; + + default: + return DXVA2_NominalRange_Unknown; + } +} + +static enum AVColorSpace dshow_color_space(DXVA2_ExtendedFormat *fmt_info) +{ + enum AVColorSpace ret = AVCOL_SPC_UNSPECIFIED; + + switch (fmt_info->VideoTransferMatrix) + { + case DXVA2_VideoTransferMatrix_BT709: + ret = AVCOL_SPC_BT709; + break; + case DXVA2_VideoTransferMatrix_BT601: + ret = AVCOL_SPC_BT470BG; + break; + case DXVA2_VideoTransferMatrix_SMPTE240M: + ret = AVCOL_SPC_SMPTE240M; + break; + + // values from MediaFoundation SDK (mfobjects.h) + case 4: // MFVideoTransferMatrix_BT2020_10 + case 5: // MFVideoTransferMatrix_BT2020_12 + if (fmt_info->VideoTransferFunction==12) // MFVideoTransFunc_2020_const + ret = AVCOL_SPC_BT2020_CL; + else + ret = AVCOL_SPC_BT2020_NCL; + break; + } + + if (ret == AVCOL_SPC_UNSPECIFIED) + { + // if color space not known from transfer matrix, + // fall back to using primaries to guess color space + switch (fmt_info->VideoPrimaries) + { + case DXVA2_VideoPrimaries_BT709: + ret = AVCOL_SPC_BT709; + break; + case DXVA2_VideoPrimaries_BT470_2_SysM: + ret = AVCOL_SPC_FCC; + break; + case DXVA2_VideoPrimaries_BT470_2_SysBG: + case DXVA2_VideoPrimaries_EBU3213: // this is PAL + ret = AVCOL_SPC_BT470BG; + break; + case DXVA2_VideoPrimaries_SMPTE170M: + case DXVA2_VideoPrimaries_SMPTE_C: + ret = AVCOL_SPC_SMPTE170M; + break; + case DXVA2_VideoPrimaries_SMPTE240M: + ret = AVCOL_SPC_SMPTE240M; + break; + } + } + + return ret; +} + +static enum AVColorPrimaries dshow_color_primaries(DXVA2_ExtendedFormat *fmt_info) +{ + switch (fmt_info->VideoPrimaries) + { + case DXVA2_VideoPrimaries_Unknown: + return AVCOL_PRI_UNSPECIFIED; + case DXVA2_VideoPrimaries_reserved: + return AVCOL_PRI_RESERVED; + case DXVA2_VideoPrimaries_BT709: + return AVCOL_PRI_BT709; + case DXVA2_VideoPrimaries_BT470_2_SysM: + return AVCOL_PRI_BT470M; + case DXVA2_VideoPrimaries_BT470_2_SysBG: + case DXVA2_VideoPrimaries_EBU3213: // this is PAL + return AVCOL_PRI_BT470BG; + case DXVA2_VideoPrimaries_SMPTE170M: + case DXVA2_VideoPrimaries_SMPTE_C: + return AVCOL_PRI_SMPTE170M; + case DXVA2_VideoPrimaries_SMPTE240M: + return AVCOL_PRI_SMPTE240M; + + // values from MediaFoundation SDK (mfobjects.h) + case 9: // MFVideoPrimaries_BT2020 + return AVCOL_PRI_BT2020; + case 10: // MFVideoPrimaries_XYZ + return AVCOL_PRI_SMPTE428; + case 11: // MFVideoPrimaries_DCI_P3 + return AVCOL_PRI_SMPTE431; + case 12: // MFVideoPrimaries_ACES (Academy Color Encoding System) + // not an FFmpeg color primary + return AVCOL_PRI_UNSPECIFIED; + + default: + return AVCOL_PRI_UNSPECIFIED; + } +} + +static enum AVColorTransferCharacteristic dshow_color_trc(DXVA2_ExtendedFormat *fmt_info) +{ + switch (fmt_info->VideoTransferFunction) + { + case DXVA2_VideoTransFunc_Unknown: + return AVCOL_TRC_UNSPECIFIED; + case DXVA2_VideoTransFunc_10: + return AVCOL_TRC_LINEAR; + case DXVA2_VideoTransFunc_18: + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case DXVA2_VideoTransFunc_20: + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case DXVA2_VideoTransFunc_22: + return AVCOL_TRC_GAMMA22; + case DXVA2_VideoTransFunc_709: + return AVCOL_TRC_BT709; + case DXVA2_VideoTransFunc_240M: + return AVCOL_TRC_SMPTE240M; + case DXVA2_VideoTransFunc_sRGB: + return AVCOL_TRC_IEC61966_2_1; + case DXVA2_VideoTransFunc_28: + return AVCOL_TRC_GAMMA28; + + // values from MediaFoundation SDK (mfobjects.h) + case 9: // MFVideoTransFunc_Log_100 + return AVCOL_TRC_LOG; + case 10: // MFVideoTransFunc_Log_316 + return AVCOL_TRC_LOG_SQRT; + case 11: // MFVideoTransFunc_709_sym + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case 12: // MFVideoTransFunc_2020_const + case 13: // MFVideoTransFunc_2020 + if (fmt_info->VideoTransferMatrix==5) // MFVideoTransferMatrix_BT2020_12 + return AVCOL_TRC_BT2020_12; + else + return AVCOL_TRC_BT2020_10; + case 14: // MFVideoTransFunc_26 + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case 15: // MFVideoTransFunc_2084 + return AVCOL_TRC_SMPTEST2084; + case 16: // MFVideoTransFunc_HLG + return AVCOL_TRC_ARIB_STD_B67; + case 17: // MFVideoTransFunc_10_rel + // not an FFmpeg transfer characteristic? Undocumented also by MS + return AVCOL_TRC_UNSPECIFIED; + + default: + return AVCOL_TRC_UNSPECIFIED; + } +} + +static enum AVChromaLocation dshow_chroma_loc(DXVA2_ExtendedFormat *fmt_info) +{ + if (fmt_info->VideoChromaSubsampling == DXVA2_VideoChromaSubsampling_Cosited) // that is: (DXVA2_VideoChromaSubsampling_Horizontally_Cosited | DXVA2_VideoChromaSubsampling_Vertically_Cosited | DXVA2_VideoChromaSubsampling_Vertically_AlignedChromaPlanes) + return AVCHROMA_LOC_TOPLEFT; + else if (fmt_info->VideoChromaSubsampling == DXVA2_VideoChromaSubsampling_MPEG1) // that is: DXVA2_VideoChromaSubsampling_Vertically_AlignedChromaPlanes + return AVCHROMA_LOC_CENTER; + else if (fmt_info->VideoChromaSubsampling == DXVA2_VideoChromaSubsampling_MPEG2) // that is: (DXVA2_VideoChromaSubsampling_Horizontally_Cosited | DXVA2_VideoChromaSubsampling_Vertically_AlignedChromaPlanes) + return AVCHROMA_LOC_LEFT; + else if (fmt_info->VideoChromaSubsampling == DXVA2_VideoChromaSubsampling_DV_PAL) // that is: (DXVA2_VideoChromaSubsampling_Horizontally_Cosited | DXVA2_VideoChromaSubsampling_Vertically_Cosited) + return AVCHROMA_LOC_TOPLEFT; + else + // unknown + return AVCHROMA_LOC_UNSPECIFIED; +} + static int dshow_read_close(AVFormatContext *s) { @@ -517,6 +728,7 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, VIDEO_STREAM_CONFIG_CAPS *vcaps = caps; BITMAPINFOHEADER *bih; int64_t *fr; + DXVA2_ExtendedFormat *extended_format_info = NULL; const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; #if DSHOWDEBUG ff_print_VIDEO_STREAM_CONFIG_CAPS(vcaps); @@ -529,6 +741,8 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, VIDEOINFOHEADER2 *v = (void *) type->pbFormat; fr = &v->AvgTimePerFrame; bih = &v->bmiHeader; + if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) + extended_format_info = (DXVA2_ExtendedFormat *) &v->dwControlFlags; } else { goto next; } @@ -545,11 +759,40 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, } else { av_log(avctx, AV_LOG_INFO, " pixel_format=%s", av_get_pix_fmt_name(pix_fmt)); } - av_log(avctx, AV_LOG_INFO, " min s=%ldx%ld fps=%g max s=%ldx%ld fps=%g\n", + av_log(avctx, AV_LOG_INFO, " min s=%ldx%ld fps=%g max s=%ldx%ld fps=%g", vcaps->MinOutputSize.cx, vcaps->MinOutputSize.cy, 1e7 / vcaps->MaxFrameInterval, vcaps->MaxOutputSize.cx, vcaps->MaxOutputSize.cy, 1e7 / vcaps->MinFrameInterval); + if (extended_format_info) { + enum AVColorRange col_range = dshow_color_range(extended_format_info); + enum AVColorSpace col_space = dshow_color_space(extended_format_info); + enum AVColorPrimaries col_prim = dshow_color_primaries(extended_format_info); + enum AVColorTransferCharacteristic col_trc = dshow_color_trc(extended_format_info); + enum AVChromaLocation chroma_loc = dshow_chroma_loc(extended_format_info); + if (col_range != AVCOL_RANGE_UNSPECIFIED || col_space != AVCOL_SPC_UNSPECIFIED || col_prim != AVCOL_PRI_UNSPECIFIED || col_trc != AVCOL_TRC_UNSPECIFIED) { + const char *range = av_color_range_name(col_range); + const char *space = av_color_space_name(col_space); + const char *prim = av_color_primaries_name(col_prim); + const char *trc = av_color_transfer_name(col_trc); + av_log(avctx, AV_LOG_INFO, " (%s, %s/%s/%s", + range ? range : "unknown", + space ? space : "unknown", + prim ? prim : "unknown", + trc ? trc : "unknown"); + if (chroma_loc != AVCHROMA_LOC_UNSPECIFIED) { + const char *chroma = av_chroma_location_name(chroma_loc); + av_log(avctx, AV_LOG_INFO, ", %s", chroma ? chroma : "unknown"); + } + av_log(avctx, AV_LOG_INFO, ")"); + } + else if (chroma_loc != AVCHROMA_LOC_UNSPECIFIED) { + const char *chroma = av_chroma_location_name(chroma_loc); + av_log(avctx, AV_LOG_INFO, "(%s)", chroma ? chroma : "unknown"); + } + } + + av_log(avctx, AV_LOG_INFO, "\n"); continue; } if (ctx->video_codec_id != AV_CODEC_ID_RAWVIDEO) { @@ -1118,6 +1361,7 @@ dshow_add_device(AVFormatContext *avctx, if (devtype == VideoDevice) { BITMAPINFOHEADER *bih = NULL; AVRational time_base; + DXVA2_ExtendedFormat *extended_format_info = NULL; if (IsEqualGUID(&type.formattype, &FORMAT_VideoInfo)) { VIDEOINFOHEADER *v = (void *) type.pbFormat; @@ -1127,6 +1371,8 @@ dshow_add_device(AVFormatContext *avctx, VIDEOINFOHEADER2 *v = (void *) type.pbFormat; time_base = (AVRational) { v->AvgTimePerFrame, 10000000 }; bih = &v->bmiHeader; + if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) + extended_format_info = (DXVA2_ExtendedFormat *) &v->dwControlFlags; } if (!bih) { av_log(avctx, AV_LOG_ERROR, "Could not get media type.\n"); @@ -1145,6 +1391,13 @@ dshow_add_device(AVFormatContext *avctx, av_log(avctx, AV_LOG_DEBUG, "attempt to use full range for HDYC...\n"); par->color_range = AVCOL_RANGE_MPEG; // just in case it needs this... } + if (extended_format_info) { + par->color_range = dshow_color_range(extended_format_info); + par->color_space = dshow_color_space(extended_format_info); + par->color_primaries = dshow_color_primaries(extended_format_info); + par->color_trc = dshow_color_trc(extended_format_info); + par->chroma_location = dshow_chroma_loc(extended_format_info); + } if (par->format == AV_PIX_FMT_NONE) { const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; par->codec_id = av_codec_get_id(tags, bih->biCompression); From patchwork Mon Dec 13 07:21:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 32413 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp5073984iog; Sun, 12 Dec 2021 23:24:28 -0800 (PST) X-Google-Smtp-Source: ABdhPJyQzjQO28TfWSZzq7YhpyuMib+ovsQDUphU7XTdwyHKhvZRsi8RtQGhlgPQNDSFFRjLb3Kp X-Received: by 2002:a17:907:3d88:: with SMTP id he8mr42877202ejc.565.1639380268350; Sun, 12 Dec 2021 23:24:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639380268; cv=none; d=google.com; s=arc-20160816; b=DkOsdra16dUKTlDQKXOLUn8leq3npkwTajV1RvYwrf3NA55O/B/qi4TvSRQX6DP/x0 kb+Zn9PC9mi4pJW6gWsaLiL/suPNhw0yKO6QNOaWuAfnEd6AIiv2Vr5CGtqCsVgQqxqp GOSEEew3dezf7dltlKXikS3iMYtCCOZe48vXGcdsmfVOZ8ddMO+gUCIpNKxBpabUfpsx hgRp2JKNcF+6Ij0Wva3hmi+vLbG2xqozSQS/ssdfCtHkMWEIEAXT44m3R98dpK4Jy6vf 1aL1mTsX3xhOzLUoyH/+VwYXkC3Ni3OCx42J7DY9wacPXidvrK2qarY3y0xnSZDF9Sca noUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=uZH+i2AEjpaYVWllGp+ldXEzxDNvPGEVQkxkRLYeWbk=; b=ZDh5OPbUItiGCUSSqdhAO/GluHwod7Ba26NkXTYhyuDbYGOy6nryaYMhqze7MT3sHy 0yzGD15P8P80FeSKFVANvFkJAh9m0X0NnMWleO3eK5AJTaP03wvcnJVG0fo5kwt8wzXi Ty38xIS2FUsZK5LdqqOz6NH2/N7gKG7dIkWJcDWxcwO8d7SvubVhNMT2ajwfeHJaC69g 1lgJqKPvMAT5TIpl+TnqZSKySXQwpVtMgCunjRhuSnlLeO/eEMYlWNrJ6tqVWkPGoNcf FlP9F7yeUa6iCf+slyXiCvjhK0entsloKtZ/DPuc+Mxr1Yt9ib2i7PjXHbM6ZHM8Nn+k uMtA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=h1GA7Gnu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g23si14263908ejw.632.2021.12.12.23.24.28; Sun, 12 Dec 2021 23:24:28 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=h1GA7Gnu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 670D868AFAB; Mon, 13 Dec 2021 09:23:30 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com [209.85.167.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 028D668AF29 for ; Mon, 13 Dec 2021 09:23:27 +0200 (EET) Received: by mail-lf1-f50.google.com with SMTP id b1so28984320lfs.13 for ; Sun, 12 Dec 2021 23:23:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xW42yQ1pfBONlUZ+SLnj/g9IDS9s2+E43DXg9YJqBUg=; b=h1GA7GnuTmQeeJqPxQd+BAmpvgafscL0qrJL3cRSDKwIOy2G7NmVWF0VriwSr7T4JM AthQvnmqOr75L49zDxnLDUdBg5t0qbVnOU+o3oshLONTrlZNLdEP/0MgR9dPLiZ5qr3X JkDyyXZ6B7E/+N3wBZqFzbcWL/F6izlCBf61nO0QTsJgp/vaq/GoDzYBOLA9e8G54rqI 7XiqWKl5R4yu01QdPbMndWFeUIVSOvVpxul1npoHaJJOxRL8hGEl9y9MDOJj7BwQxgED e6PBV3P1YHyQUm1ft1z958fN901lvQbMTf69Y36mQNMXpG1UQKCSd95MtUeoFpgOsFqc Xx1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xW42yQ1pfBONlUZ+SLnj/g9IDS9s2+E43DXg9YJqBUg=; b=5pgG7bKKMxy7ZvQy3mbXk1oJ0AmYbLRp1gDCrZ6LceKanHFDTm4fFuC2KJvyNcHkNb WKH8aOoz8BQJ8S4nT9gy6obN9dy5jEBWzXuF9ZzWlWNzT9t731+sX24c3Z+TV1xL+CRk nTqg8wCSSsHO6+XsycQusnYz+RnT9TqbyVMSMkdre6YdcVYXCvMGAB7qcqkCkUPUkKic GKVAw46g4NsO+vOMHA3PjKb2voKpuI/BxwkIpOSH39ZesmQgiGjARNsZt6qDc8pjMFNI ISRDvFOiRUcm8tULtxV5/pXwXEdm6hCqGq4UBL7EqMoLxhcGegDPjITrmIVmVi6J62ar 6erg== X-Gm-Message-State: AOAM5304xbrgH8KwFO+0IwagIKzILPpGk+Vhz6zw2FKc276jx5P1S83N xrdWtrIpdtDanmoekleNVwWRBow+AnTZSA== X-Received: by 2002:a05:6512:323c:: with SMTP id f28mr27881409lfe.462.1639380206494; Sun, 12 Dec 2021 23:23:26 -0800 (PST) Received: from localhost.localdomain (84-217-56-54.customers.ownit.se. [84.217.56.54]) by smtp.gmail.com with ESMTPSA id q9sm1315800lfu.232.2021.12.12.23.23.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Dec 2021 23:23:26 -0800 (PST) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Mon, 13 Dec 2021 08:21:43 +0100 Message-Id: <20211213072143.993-14-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20211213072143.993-1-dcnieho@gmail.com> References: <20211213072143.993-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 13/13] avdevice/dshow: select format with extended color info X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: qRvXRGbAFh5P Some DirectShow devices (Logitech C920 webcam) expose each DirectShow format they support twice, once without and once with extended color information. During format selection, both match, this patch ensures that the format with extended color information is selected if it is available, else it falls back to a matching format without such information. This also necessitated a new code path taken for default formats of a device (when user didn't request any specific video size, etc), because the default format may be one without extended color information when a twin with extended color information is also available. Getting the extended color information when available is important as it allows setting the color space, range, primaries, transfer characteristics and chroma location of the stream provided by dshow, enabling users to get more correct color automatically out of their device. Closes: #9271 Signed-off-by: Diederick Niehorster --- libavdevice/dshow.c | 469 +++++++++++++++++++++++++++++++------------- 1 file changed, 338 insertions(+), 131 deletions(-) diff --git a/libavdevice/dshow.c b/libavdevice/dshow.c index 4ad6ae102c..6f2a3ac1ba 100644 --- a/libavdevice/dshow.c +++ b/libavdevice/dshow.c @@ -23,6 +23,7 @@ #include "libavutil/parseutils.h" #include "libavutil/pixdesc.h" #include "libavutil/opt.h" +#include "libavutil/mem.h" #include "libavformat/internal.h" #include "libavformat/riff.h" #include "avdevice.h" @@ -690,9 +691,111 @@ error: return ret; } +static int dshow_should_set_format(AVFormatContext *avctx, enum dshowDeviceType devtype) +{ + struct dshow_ctx *ctx = avctx->priv_data; + + return (devtype == VideoDevice && (ctx->framerate || + (ctx->requested_width && ctx->requested_height) || + ctx->pixel_format != AV_PIX_FMT_NONE || + ctx->video_codec_id != AV_CODEC_ID_RAWVIDEO)) + || (devtype == AudioDevice && (ctx->channels || ctx->sample_size || ctx->sample_rate)); +} + + +struct dshow_format_info { + enum dshowDeviceType devtype; + // video + int64_t framerate; + enum AVPixelFormat pix_fmt; + enum AVCodecID codec_id; + enum AVColorRange col_range; + enum AVColorSpace col_space; + enum AVColorPrimaries col_prim; + enum AVColorTransferCharacteristic col_trc; + enum AVChromaLocation chroma_loc; + int width; + int height; + // audio + int sample_rate; + int sample_size; + int channels; +}; + +// user must av_free the returned pointer +static struct dshow_format_info *dshow_get_format_info(AM_MEDIA_TYPE *type) +{ + struct dshow_format_info *fmt_info = NULL; + BITMAPINFOHEADER *bih; + DXVA2_ExtendedFormat *extended_format_info = NULL; + WAVEFORMATEX *fx; + enum dshowDeviceType devtype; + int64_t framerate; + + if (!type) + return NULL; + + if (IsEqualGUID(&type->formattype, &FORMAT_VideoInfo)) { + VIDEOINFOHEADER *v = (void *) type->pbFormat; + framerate = v->AvgTimePerFrame; + bih = &v->bmiHeader; + devtype = VideoDevice; + } else if (IsEqualGUID(&type->formattype, &FORMAT_VideoInfo2)) { + VIDEOINFOHEADER2 *v = (void *) type->pbFormat; + devtype = VideoDevice; + framerate = v->AvgTimePerFrame; + bih = &v->bmiHeader; + if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) + extended_format_info = (DXVA2_ExtendedFormat *) &v->dwControlFlags; + } else if (IsEqualGUID(&type->formattype, &FORMAT_WaveFormatEx)) { + fx = (void *) type->pbFormat; + devtype = AudioDevice; + } else { + return NULL; + } + + fmt_info = av_mallocz(sizeof(struct dshow_format_info)); + if (!fmt_info) + return NULL; + // initialize fields where unset is not zero + fmt_info->pix_fmt = AV_PIX_FMT_NONE; + fmt_info->col_space = AVCOL_SPC_UNSPECIFIED; + fmt_info->col_prim = AVCOL_PRI_UNSPECIFIED; + fmt_info->col_trc = AVCOL_TRC_UNSPECIFIED; + // now get info about format + fmt_info->devtype = devtype; + if (devtype == VideoDevice) { + fmt_info->width = bih->biWidth; + fmt_info->height = bih->biHeight; + fmt_info->framerate = framerate; + fmt_info->pix_fmt = dshow_pixfmt(bih->biCompression, bih->biBitCount); + if (fmt_info->pix_fmt == AV_PIX_FMT_NONE) { + const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; + fmt_info->codec_id = av_codec_get_id(tags, bih->biCompression); + } + else + fmt_info->codec_id = AV_CODEC_ID_RAWVIDEO; + + if (extended_format_info) { + fmt_info->col_range = dshow_color_range(extended_format_info); + fmt_info->col_space = dshow_color_space(extended_format_info); + fmt_info->col_prim = dshow_color_primaries(extended_format_info); + fmt_info->col_trc = dshow_color_trc(extended_format_info); + fmt_info->chroma_loc = dshow_chroma_loc(extended_format_info); + } + } else { + fmt_info->sample_rate = fx->nSamplesPerSec; + fmt_info->sample_size = fx->wBitsPerSample; + fmt_info->channels = fx->nChannels; + } + + return fmt_info; +} + /** - * Cycle through available formats using the specified pin, - * try to set parameters specified through AVOptions and if successful + * Cycle through available formats available from the specified pin, + * try to set parameters specified through AVOptions, or the pin's + * default format if no such parameters were set. If successful, * return 1 in *pformat_set. * If pformat_set is NULL, list all pin capabilities. */ @@ -703,9 +806,27 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, struct dshow_ctx *ctx = avctx->priv_data; IAMStreamConfig *config = NULL; AM_MEDIA_TYPE *type = NULL; + AM_MEDIA_TYPE *previous_match_type = NULL; int format_set = 0; void *caps = NULL; int i, n, size, r; + int wait_for_better = 0; + int use_default; + + // format parameters requested by user + // if none are requested by user, the values will below be set to + // those of the default format + // video + enum AVCodecID requested_video_codec_id = ctx->video_codec_id; + enum AVPixelFormat requested_pixel_format = ctx->pixel_format; + int64_t requested_framerate = ctx->framerate ? ((int64_t)ctx->requested_framerate.den * 10000000) + / ctx->requested_framerate.num : 0; + int requested_width = ctx->requested_width; + int requested_height = ctx->requested_height; + // audio + int requested_sample_rate = ctx->sample_rate; + int requested_sample_size = ctx->sample_size; + int requested_channels = ctx->channels; if (IPin_QueryInterface(pin, &IID_IAMStreamConfig, (void **) &config) != S_OK) return; @@ -716,7 +837,83 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, if (!caps) goto end; + /** + * If we should open the device with the default format, + * then: + * 1. check what the format of the default device is, and + * 2. below we iterate all formats till we find a matching + * one, with most info exposed (see comment below). + */ + use_default = !dshow_should_set_format(avctx, devtype); + if (use_default && pformat_set) + { + HRESULT hr; + + // get default + if ((hr = IAMStreamConfig_GetFormat(config, &type)) != S_OK) { + if (hr == E_NOTIMPL || !IsEqualGUID(&type->majortype, devtype==VideoDevice ? &MEDIATYPE_Video : &MEDIATYPE_Audio)) { + // default not available or of wrong type, + // fall back to iterating exposed formats + // until one of the right type is found + IEnumMediaTypes *types = NULL; + if (IPin_EnumMediaTypes(pin, &types) != S_OK) + goto end; + IEnumMediaTypes_Reset(types); + while (IEnumMediaTypes_Next(types, 1, &type, NULL) == S_OK) { + if (IsEqualGUID(&type->majortype, devtype==VideoDevice ? &MEDIATYPE_Video : &MEDIATYPE_Audio)) { + break; + } + CoTaskMemFree(type); + type = NULL; + } + IEnumMediaTypes_Release(types); + } + + if (!type) + // this pin does not expose any formats of the expected type + goto end; + } + + if (type) { + // interrogate default format, so we know what to search for below + struct dshow_format_info *fmt_info = dshow_get_format_info(type); + if (fmt_info) { + if (fmt_info->devtype == VideoDevice) { + requested_video_codec_id = fmt_info->codec_id; + requested_pixel_format = fmt_info->pix_fmt; + requested_framerate = fmt_info->framerate; + requested_width = fmt_info->width; + requested_height = fmt_info->height; + } else { + requested_sample_rate = fmt_info->sample_rate; + requested_sample_size = fmt_info->sample_size; + requested_channels = fmt_info->channels; + } + av_free(fmt_info); // free but don't set to NULL to enable below check + } + + if (type && type->pbFormat) + CoTaskMemFree(type->pbFormat); + CoTaskMemFree(type); + type = NULL; + if (!fmt_info) + // default format somehow invalid, can't continue with this pin + goto end; + fmt_info = NULL; + } + } + + // NB: some devices (e.g. Logitech C920) expose each video format twice: + // both a format containing a VIDEOINFOHEADER and a format containing + // a VIDEOINFOHEADER2. We want, if possible, to select a format with a + // VIDEOINFOHEADER2, as this potentially provides more info about the + // format. So, if in the iteration below we have found a matching format, + // but it is a VIDEOINFOHEADER, keep looking for a matching format that + // exposes contains a VIDEOINFOHEADER2. Fall back to the VIDEOINFOHEADER + // format if no corresponding VIDEOINFOHEADER2 is found when we finish + // iterating. for (i = 0; i < n && !format_set; i++) { + struct dshow_format_info *fmt_info = NULL; r = IAMStreamConfig_GetStreamCaps(config, i, &type, (void *) caps); if (r != S_OK) goto next; @@ -724,106 +921,101 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, ff_print_AM_MEDIA_TYPE(type); #endif + fmt_info = dshow_get_format_info(type); + if (!fmt_info) + goto next; + if (devtype == VideoDevice) { VIDEO_STREAM_CONFIG_CAPS *vcaps = caps; BITMAPINFOHEADER *bih; int64_t *fr; - DXVA2_ExtendedFormat *extended_format_info = NULL; - const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; #if DSHOWDEBUG ff_print_VIDEO_STREAM_CONFIG_CAPS(vcaps); #endif + + if (fmt_info->devtype != VideoDevice) + goto next; + if (IsEqualGUID(&type->formattype, &FORMAT_VideoInfo)) { VIDEOINFOHEADER *v = (void *) type->pbFormat; - fr = &v->AvgTimePerFrame; + fr = &v->AvgTimePerFrame; bih = &v->bmiHeader; + wait_for_better = 1; } else if (IsEqualGUID(&type->formattype, &FORMAT_VideoInfo2)) { VIDEOINFOHEADER2 *v = (void *) type->pbFormat; - fr = &v->AvgTimePerFrame; + fr = &v->AvgTimePerFrame; bih = &v->bmiHeader; - if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) - extended_format_info = (DXVA2_ExtendedFormat *) &v->dwControlFlags; - } else { - goto next; + wait_for_better = 0; } + if (!pformat_set) { - enum AVPixelFormat pix_fmt = dshow_pixfmt(bih->biCompression, bih->biBitCount); - if (pix_fmt == AV_PIX_FMT_NONE) { - enum AVCodecID codec_id = av_codec_get_id(tags, bih->biCompression); - const AVCodec *codec = avcodec_find_decoder(codec_id); - if (codec_id == AV_CODEC_ID_NONE || !codec) { + if (fmt_info->pix_fmt == AV_PIX_FMT_NONE) { + const AVCodec *codec = avcodec_find_decoder(fmt_info->codec_id); + if (fmt_info->codec_id == AV_CODEC_ID_NONE || !codec) { av_log(avctx, AV_LOG_INFO, " unknown compression type 0x%X", (int) bih->biCompression); } else { av_log(avctx, AV_LOG_INFO, " vcodec=%s", codec->name); } } else { - av_log(avctx, AV_LOG_INFO, " pixel_format=%s", av_get_pix_fmt_name(pix_fmt)); + av_log(avctx, AV_LOG_INFO, " pixel_format=%s", av_get_pix_fmt_name(fmt_info->pix_fmt)); } av_log(avctx, AV_LOG_INFO, " min s=%ldx%ld fps=%g max s=%ldx%ld fps=%g", vcaps->MinOutputSize.cx, vcaps->MinOutputSize.cy, 1e7 / vcaps->MaxFrameInterval, vcaps->MaxOutputSize.cx, vcaps->MaxOutputSize.cy, 1e7 / vcaps->MinFrameInterval); - if (extended_format_info) { - enum AVColorRange col_range = dshow_color_range(extended_format_info); - enum AVColorSpace col_space = dshow_color_space(extended_format_info); - enum AVColorPrimaries col_prim = dshow_color_primaries(extended_format_info); - enum AVColorTransferCharacteristic col_trc = dshow_color_trc(extended_format_info); - enum AVChromaLocation chroma_loc = dshow_chroma_loc(extended_format_info); - if (col_range != AVCOL_RANGE_UNSPECIFIED || col_space != AVCOL_SPC_UNSPECIFIED || col_prim != AVCOL_PRI_UNSPECIFIED || col_trc != AVCOL_TRC_UNSPECIFIED) { - const char *range = av_color_range_name(col_range); - const char *space = av_color_space_name(col_space); - const char *prim = av_color_primaries_name(col_prim); - const char *trc = av_color_transfer_name(col_trc); - av_log(avctx, AV_LOG_INFO, " (%s, %s/%s/%s", - range ? range : "unknown", - space ? space : "unknown", - prim ? prim : "unknown", - trc ? trc : "unknown"); - if (chroma_loc != AVCHROMA_LOC_UNSPECIFIED) { - const char *chroma = av_chroma_location_name(chroma_loc); - av_log(avctx, AV_LOG_INFO, ", %s", chroma ? chroma : "unknown"); - } - av_log(avctx, AV_LOG_INFO, ")"); - } - else if (chroma_loc != AVCHROMA_LOC_UNSPECIFIED) { - const char *chroma = av_chroma_location_name(chroma_loc); - av_log(avctx, AV_LOG_INFO, "(%s)", chroma ? chroma : "unknown"); - } + + const char *chroma = av_chroma_location_name(fmt_info->chroma_loc); + if (fmt_info->col_range != AVCOL_RANGE_UNSPECIFIED || + fmt_info->col_space != AVCOL_SPC_UNSPECIFIED || + fmt_info->col_prim != AVCOL_PRI_UNSPECIFIED || + fmt_info->col_trc != AVCOL_TRC_UNSPECIFIED) { + const char *range = av_color_range_name(fmt_info->col_range); + const char *space = av_color_space_name(fmt_info->col_space); + const char *prim = av_color_primaries_name(fmt_info->col_prim); + const char *trc = av_color_transfer_name(fmt_info->col_trc); + av_log(avctx, AV_LOG_INFO, " (%s, %s/%s/%s", + range ? range : "unknown", + space ? space : "unknown", + prim ? prim : "unknown", + trc ? trc : "unknown"); + if (fmt_info->chroma_loc != AVCHROMA_LOC_UNSPECIFIED) + av_log(avctx, AV_LOG_INFO, ", %s", chroma ? chroma : "unknown"); + av_log(avctx, AV_LOG_INFO, ")"); } + else if (fmt_info->chroma_loc != AVCHROMA_LOC_UNSPECIFIED) + av_log(avctx, AV_LOG_INFO, "(%s)", chroma ? chroma : "unknown"); av_log(avctx, AV_LOG_INFO, "\n"); continue; } - if (ctx->video_codec_id != AV_CODEC_ID_RAWVIDEO) { - if (ctx->video_codec_id != av_codec_get_id(tags, bih->biCompression)) + if (requested_video_codec_id != AV_CODEC_ID_RAWVIDEO) { + if (requested_video_codec_id != fmt_info->codec_id) goto next; } - if (ctx->pixel_format != AV_PIX_FMT_NONE && - ctx->pixel_format != dshow_pixfmt(bih->biCompression, bih->biBitCount)) { + if (requested_pixel_format != AV_PIX_FMT_NONE && + requested_pixel_format != fmt_info->pix_fmt) { goto next; } - if (ctx->framerate) { - int64_t framerate = ((int64_t) ctx->requested_framerate.den*10000000) - / ctx->requested_framerate.num; - if (framerate > vcaps->MaxFrameInterval || - framerate < vcaps->MinFrameInterval) + if (requested_framerate) { + if (requested_framerate > vcaps->MaxFrameInterval || + requested_framerate < vcaps->MinFrameInterval) goto next; - *fr = framerate; + *fr = requested_framerate; } - if (ctx->requested_width && ctx->requested_height) { - if (ctx->requested_width > vcaps->MaxOutputSize.cx || - ctx->requested_width < vcaps->MinOutputSize.cx || - ctx->requested_height > vcaps->MaxOutputSize.cy || - ctx->requested_height < vcaps->MinOutputSize.cy) + if (requested_width && requested_height) { + if (requested_width > vcaps->MaxOutputSize.cx || + requested_width < vcaps->MinOutputSize.cx || + requested_height > vcaps->MaxOutputSize.cy || + requested_height < vcaps->MinOutputSize.cy) goto next; - bih->biWidth = ctx->requested_width; - bih->biHeight = ctx->requested_height; + bih->biWidth = requested_width; + bih->biHeight = requested_height; } } else { WAVEFORMATEX *fx; -#if DSHOWDEBUG AUDIO_STREAM_CONFIG_CAPS *acaps = caps; +#if DSHOWDEBUG ff_print_AUDIO_STREAM_CONFIG_CAPS(acaps); #endif if (IsEqualGUID(&type->formattype, &FORMAT_WaveFormatEx)) { @@ -840,23 +1032,62 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, ); continue; } - if ( - (ctx->sample_rate && ctx->sample_rate != fx->nSamplesPerSec) || - (ctx->sample_size && ctx->sample_size != fx->wBitsPerSample) || - (ctx->channels && ctx->channels != fx->nChannels ) - ) { - goto next; + if (requested_sample_rate) { + if (requested_sample_rate > acaps->MaximumSampleFrequency || + requested_sample_rate < acaps->MinimumSampleFrequency) + goto next; + fx->nSamplesPerSec = requested_sample_rate; + } + if (requested_sample_size) { + if (requested_sample_size > acaps->MaximumBitsPerSample || + requested_sample_size < acaps->MinimumBitsPerSample) + goto next; + fx->wBitsPerSample = requested_sample_size; + } + if (requested_channels) { + if (requested_channels > acaps->MaximumChannels || + requested_channels < acaps->MinimumChannels) + goto next; + fx->nChannels = requested_channels; } } - if (IAMStreamConfig_SetFormat(config, type) != S_OK) - goto next; - format_set = 1; + + // found a matching format. Either apply or store + // for safekeeping if we might maybe find a better + // format with more info attached to it (see comment + // above loop) + if (!wait_for_better) { + if (IAMStreamConfig_SetFormat(config, type) != S_OK) + goto next; + format_set = 1; + } + else if (!previous_match_type) { + // store this matching format for possible later use. + // If we have already found a matching format, ignore it + previous_match_type = type; + type = NULL; + } next: - if (type->pbFormat) + av_freep(&fmt_info); + if (type && type->pbFormat) CoTaskMemFree(type->pbFormat); CoTaskMemFree(type); } + // previously found a matching VIDEOINFOHEADER format and stored + // it for safe keeping. Searching further for a matching + // VIDEOINFOHEADER2 format yielded nothing. So set the pin's + // format based on the VIDEOINFOHEADER format. + // NB: this never applies to an audio format because + // previous_match_type always NULL in that case + if (!format_set && previous_match_type) { + if (IAMStreamConfig_SetFormat(config, previous_match_type) == S_OK) + format_set = 1; + } + end: + if (previous_match_type && previous_match_type->pbFormat) + CoTaskMemFree(previous_match_type->pbFormat); + CoTaskMemFree(previous_match_type); IAMStreamConfig_Release(config); av_free(caps); if (pformat_set) @@ -975,11 +1206,7 @@ dshow_cycle_pins(AVFormatContext *avctx, enum dshowDeviceType devtype, const char *devtypename = (devtype == VideoDevice) ? "video" : "audio only"; const char *sourcetypename = (sourcetype == VideoSourceDevice) ? "video" : "audio"; - int set_format = (devtype == VideoDevice && (ctx->framerate || - (ctx->requested_width && ctx->requested_height) || - ctx->pixel_format != AV_PIX_FMT_NONE || - ctx->video_codec_id != AV_CODEC_ID_RAWVIDEO)) - || (devtype == AudioDevice && (ctx->channels || ctx->sample_rate || ctx->sample_size)); + int set_format = dshow_should_set_format(avctx, devtype); int format_set = 0; int should_show_properties = (devtype == VideoDevice) ? ctx->show_video_device_dialog : ctx->show_audio_device_dialog; @@ -999,9 +1226,7 @@ dshow_cycle_pins(AVFormatContext *avctx, enum dshowDeviceType devtype, while (!device_pin && IEnumPins_Next(pins, 1, &pin, NULL) == S_OK) { IKsPropertySet *p = NULL; - IEnumMediaTypes *types = NULL; PIN_INFO info = {0}; - AM_MEDIA_TYPE *type; GUID category; DWORD r2; char *name_buf = NULL; @@ -1044,35 +1269,24 @@ dshow_cycle_pins(AVFormatContext *avctx, enum dshowDeviceType devtype, } } - if (set_format) { - dshow_cycle_formats(avctx, devtype, pin, &format_set); - if (!format_set) { - goto next; - } + // will either try to find format matching options supplied by user + // or try to open default format. Successful if returns with format_set==1 + dshow_cycle_formats(avctx, devtype, pin, &format_set); + if (!format_set) { + goto next; } + if (devtype == AudioDevice && ctx->audio_buffer_size) { if (dshow_set_audio_buffer_size(avctx, pin) < 0) { av_log(avctx, AV_LOG_ERROR, "unable to set audio buffer size %d to pin, using pin anyway...", ctx->audio_buffer_size); } } - if (IPin_EnumMediaTypes(pin, &types) != S_OK) - goto next; - - IEnumMediaTypes_Reset(types); - /* in case format_set was not called, just verify the majortype */ - while (!device_pin && IEnumMediaTypes_Next(types, 1, &type, NULL) == S_OK) { - if (IsEqualGUID(&type->majortype, mediatype[devtype])) { - device_pin = pin; - av_log(avctx, AV_LOG_DEBUG, "Selecting pin %s on %s\n", name_buf, devtypename); - goto next; - } - CoTaskMemFree(type); + if (format_set) { + device_pin = pin; + av_log(avctx, AV_LOG_DEBUG, "Selecting pin %s on %s\n", name_buf, devtypename); } - next: - if (types) - IEnumMediaTypes_Release(types); if (p) IKsPropertySet_Release(p); if (device_pin != pin) @@ -1342,6 +1556,7 @@ dshow_add_device(AVFormatContext *avctx, AM_MEDIA_TYPE type; AVCodecParameters *par; AVStream *st; + struct dshow_format_info *fmt_info = NULL; int ret = AVERROR(EIO); type.pbFormat = NULL; @@ -1356,12 +1571,14 @@ dshow_add_device(AVFormatContext *avctx, ctx->capture_filter[devtype]->stream_index = st->index; ff_dshow_pin_ConnectionMediaType(ctx->capture_pin[devtype], &type); + fmt_info = dshow_get_format_info(&type); + if (!fmt_info) + goto error; par = st->codecpar; if (devtype == VideoDevice) { BITMAPINFOHEADER *bih = NULL; AVRational time_base; - DXVA2_ExtendedFormat *extended_format_info = NULL; if (IsEqualGUID(&type.formattype, &FORMAT_VideoInfo)) { VIDEOINFOHEADER *v = (void *) type.pbFormat; @@ -1371,8 +1588,6 @@ dshow_add_device(AVFormatContext *avctx, VIDEOINFOHEADER2 *v = (void *) type.pbFormat; time_base = (AVRational) { v->AvgTimePerFrame, 10000000 }; bih = &v->bmiHeader; - if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) - extended_format_info = (DXVA2_ExtendedFormat *) &v->dwControlFlags; } if (!bih) { av_log(avctx, AV_LOG_ERROR, "Could not get media type.\n"); @@ -1383,33 +1598,21 @@ dshow_add_device(AVFormatContext *avctx, st->r_frame_rate = av_inv_q(time_base); par->codec_type = AVMEDIA_TYPE_VIDEO; - par->width = bih->biWidth; - par->height = bih->biHeight; + par->width = fmt_info->width; + par->height = fmt_info->height; par->codec_tag = bih->biCompression; - par->format = dshow_pixfmt(bih->biCompression, bih->biBitCount); + par->format = fmt_info->pix_fmt; if (bih->biCompression == MKTAG('H', 'D', 'Y', 'C')) { av_log(avctx, AV_LOG_DEBUG, "attempt to use full range for HDYC...\n"); par->color_range = AVCOL_RANGE_MPEG; // just in case it needs this... } - if (extended_format_info) { - par->color_range = dshow_color_range(extended_format_info); - par->color_space = dshow_color_space(extended_format_info); - par->color_primaries = dshow_color_primaries(extended_format_info); - par->color_trc = dshow_color_trc(extended_format_info); - par->chroma_location = dshow_chroma_loc(extended_format_info); - } - if (par->format == AV_PIX_FMT_NONE) { - const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; - par->codec_id = av_codec_get_id(tags, bih->biCompression); - if (par->codec_id == AV_CODEC_ID_NONE) { - av_log(avctx, AV_LOG_ERROR, "Unknown compression type. " - "Please report type 0x%X.\n", (int) bih->biCompression); - ret = AVERROR_PATCHWELCOME; - goto error; - } - par->bits_per_coded_sample = bih->biBitCount; - } else { - par->codec_id = AV_CODEC_ID_RAWVIDEO; + par->color_range = fmt_info->col_range; + par->color_space = fmt_info->col_space; + par->color_primaries = fmt_info->col_prim; + par->color_trc = fmt_info->col_trc; + par->chroma_location = fmt_info->chroma_loc; + par->codec_id = fmt_info->codec_id; + if (par->codec_id == AV_CODEC_ID_RAWVIDEO) { if (bih->biCompression == BI_RGB || bih->biCompression == BI_BITFIELDS) { par->bits_per_coded_sample = bih->biBitCount; if (par->height < 0) { @@ -1422,23 +1625,26 @@ dshow_add_device(AVFormatContext *avctx, } } } + } else { + if (par->codec_id == AV_CODEC_ID_NONE) { + av_log(avctx, AV_LOG_ERROR, "Unknown compression type. " + "Please report type 0x%X.\n", (int) bih->biCompression); + ret = AVERROR_PATCHWELCOME; + goto error; + } + par->bits_per_coded_sample = bih->biBitCount; } } else { - WAVEFORMATEX *fx = NULL; - - if (IsEqualGUID(&type.formattype, &FORMAT_WaveFormatEx)) { - fx = (void *) type.pbFormat; - } - if (!fx) { + if (!IsEqualGUID(&type.formattype, &FORMAT_WaveFormatEx)) { av_log(avctx, AV_LOG_ERROR, "Could not get media type.\n"); goto error; } par->codec_type = AVMEDIA_TYPE_AUDIO; - par->format = sample_fmt_bits_per_sample(fx->wBitsPerSample); + par->format = sample_fmt_bits_per_sample(fmt_info->sample_size); par->codec_id = waveform_codec_id(par->format); - par->sample_rate = fx->nSamplesPerSec; - par->channels = fx->nChannels; + par->sample_rate = fmt_info->sample_rate; + par->channels = fmt_info->channels; } avpriv_set_pts_info(st, 64, 1, 10000000); @@ -1446,6 +1652,7 @@ dshow_add_device(AVFormatContext *avctx, ret = 0; error: + av_freep(&fmt_info); if (type.pbFormat) CoTaskMemFree(type.pbFormat); return ret;