From patchwork Wed Jun 2 06:28:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 28042 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:b214:0:0:0:0:0 with SMTP id b20csp273057iof; Wed, 2 Jun 2021 00:00:36 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx87Un/TS60vzGZ6c02COgmyBUKDwD2Xf45CQLmQJ3QD+aVPiFSfCwj4zaKWQsLtB7K5AR+ X-Received: by 2002:a05:6402:4316:: with SMTP id m22mr2022104edc.316.1622617236226; Wed, 02 Jun 2021 00:00:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1622617236; cv=none; d=google.com; s=arc-20160816; b=SPCa+UGNnpK4b4On53qHhIT/Bt43IH+6ubzDKw5mxDM34xrbaOkuB9eQxvdAGFSMPL lTRR+QcOEmDmHWwxgr/QL6MgzNsz5E57jk5lBS1/LCwENLoKSEGgfmcYYyb6G/E3slUT q85lxLYuIb4vffSFJalVwl1vaVIapqg2C499YIOgh6LAL+ibKj5bV4mQ6QNueuCvCGPV 2WkAZ/ZeTj+wYA4vpSh6YJ1YVH6Li6Itg8Snp9egulx3eEcTvE1ZOpwE6uRUkbYdiL17 CWuvO2TustkMfKXsSaqVIuXZuBlpJ8vJEolkXSRILPKor9urvZWd8eroKvRpmhUhSVS/ ahkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=nAtdlG9eWzyGnEysA3G+epQs0phQX8djfzFLi5HdEJo=; b=ZRJ9MpHDly7wOJUOh5OMdYIpMHy6r9gyN40kOTFwqaSed3Bb0m8om7u3Sk5YwkmU5H xcO+ueWEEOMpB+EGVJsbACNdoSqbMgajBW6nAxom1qZvwnNE0451R7lQRC8w20re+J98 HtieHCyy/IcQTF4GmP+HTCAOSlLMNJaCSyvKf4ncOwS4KaBWu5hP1K+fVp5QOOpwh/sF 31Mpbh3v63b0opohLqAgvJN/uoqmIy3cRxxkhWP2sStWNsl0SKUaAzkwH+OIUq6RxASw lvbzUeo6E8w51YUvLoQ5/XU1O4Jd1/jHfdjE9setQVY1ivh8bEDLic9GR5hXdodQuAMK 0JwA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=Q5JH97hG; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id w7si17865483ejy.722.2021.06.02.00.00.35; Wed, 02 Jun 2021 00:00:36 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=Q5JH97hG; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 55AA9689F86; Wed, 2 Jun 2021 10:00:33 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 73701680998 for ; Wed, 2 Jun 2021 10:00:27 +0300 (EEST) Received: by mail-lf1-f42.google.com with SMTP id v8so1727975lft.8 for ; Wed, 02 Jun 2021 00:00:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KN6yGgfmMG2RXOylGaDbAtJ5zpM3FTO7q7fd7zO7IgM=; b=Q5JH97hGDbA6u2IeAVDjCOQMPMjSvnp95syZNUSF5KMAMwBPcrPQM/kbPa6ERw1J8t lZdtW7KviSnLQEEcx1BjYRQ1iApqGJ+8JmS5EHYbu02WjkQdsaMMVYomZT4+afkdynEn Ij8PDmg2uVpVG3D/KMR5bI5ygyLSC3/zLtBaznqVydIuxeWhkFjb+MTrXjK9cTWpkPdp kpdpwbBdgYeIQv0ezYfdNwwuJUWJqwBmyhS9vjJ3q4sterQ5/1YHzaZsf5c8HAGGvZMe DkaJaF5fYciY/sh4uaeugfuQjHxL/9EdcVMiZZsFI1UobGVZqJdXc5gcsv9LA8DgkIvT M7AA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KN6yGgfmMG2RXOylGaDbAtJ5zpM3FTO7q7fd7zO7IgM=; b=ew5M/YAnqypyI1iC53GOZvRLqPJ1DDM6h2grPgI2Bzg6K5uGXRONgYE6qjebYhNSjY t8+1osPuf15+iK0191qKuRMT95NFJyQf69YmY7MujPZsaFJ2KFgnGnJGfmVS8KQ8VoJL T7N/eDNozLT/q65jS+wBYB7NsD7rh7c8W6ssS9iw7LguY2fSdb+8nXkciRcamCPsZjIV Foq14cRC2gj2YAZk+ZjudHzXcpgGFat3OJVYmb8N48/28kCMgsVikPxo3cqkudit0Huo B3lTi1TSQzOJGPYlye/Yk+afoBzWW8gdMigcofUzC/7FImahzbJvl11pIPlAJ4EmltVg /YJA== X-Gm-Message-State: AOAM533dJkEFSp1Z072eccin55dLjn9Umg+AZR4BgPWUrFkBYoCpgQe0 LsEwqnY0uQfjAkxRr8liIlMwsqt3T/3+kg== X-Received: by 2002:a2e:808f:: with SMTP id i15mr23817604ljg.376.1622615357376; Tue, 01 Jun 2021 23:29:17 -0700 (PDT) Received: from localhost.localdomain (deedock.humlab.lu.se. [130.235.135.183]) by smtp.gmail.com with ESMTPSA id p6sm1901985lfr.163.2021.06.01.23.29.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Jun 2021 23:29:16 -0700 (PDT) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Wed, 2 Jun 2021 08:28:56 +0200 Message-Id: <20210602062857.1767-2-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20210602062857.1767-1-dcnieho@gmail.com> References: <20210602062857.1767-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] avdevice/dshow: discover source color range/space/etc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: tBs5SZ89ltmH Enabled discovering a DirectShow device's color range, space, primaries, transfer characteristics and chroma location, if the device exposes that information. Sets them in the stream's codecpars. Signed-off-by: Diederick Niehorster --- libavdevice/dshow.c | 231 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 230 insertions(+), 1 deletion(-) diff --git a/libavdevice/dshow.c b/libavdevice/dshow.c index 8d0a6fcc09..c59cd30f30 100644 --- a/libavdevice/dshow.c +++ b/libavdevice/dshow.c @@ -29,6 +29,7 @@ #include "libavcodec/raw.h" #include "objidl.h" #include "shlwapi.h" +#include "dxva.h" static enum AVPixelFormat dshow_pixfmt(DWORD biCompression, WORD biBitCount) @@ -54,6 +55,192 @@ static enum AVPixelFormat dshow_pixfmt(DWORD biCompression, WORD biBitCount) return avpriv_find_pix_fmt(avpriv_get_raw_pix_fmt_tags(), biCompression); // all others } +static enum AVColorRange dshow_color_range(DXVA_ExtendedFormat* fmt_info) +{ + switch (fmt_info->NominalRange) + { + case DXVA_NominalRange_Unknown: + return AVCOL_RANGE_UNSPECIFIED; + case DXVA_NominalRange_Normal: // equal to DXVA_NominalRange_0_255 + return AVCOL_RANGE_JPEG; + case DXVA_NominalRange_Wide: // equal to DXVA_NominalRange_16_235 + return AVCOL_RANGE_MPEG; + case DXVA_NominalRange_48_208: + // not an ffmpeg color range + return AVCOL_RANGE_UNSPECIFIED; + + // values from MediaFoundation SDK (mfobjects.h) + case 4: // MFNominalRange_64_127 + // not an ffmpeg color range + return AVCOL_RANGE_UNSPECIFIED; + + default: + return DXVA_NominalRange_Unknown; + } +} + +static enum AVColorSpace dshow_color_space(DXVA_ExtendedFormat* fmt_info) +{ + enum AVColorSpace ret = AVCOL_SPC_UNSPECIFIED; + + switch (fmt_info->VideoTransferMatrix) + { + case DXVA_VideoTransferMatrix_BT709: + ret = AVCOL_SPC_BT709; + break; + case DXVA_VideoTransferMatrix_BT601: + ret = AVCOL_SPC_BT470BG; + break; + case DXVA_VideoTransferMatrix_SMPTE240M: + ret = AVCOL_SPC_SMPTE240M; + break; + + // values from MediaFoundation SDK (mfobjects.h) + case 4: // MFVideoTransferMatrix_BT2020_10 + case 5: // MFVideoTransferMatrix_BT2020_12 + if (fmt_info->VideoTransferFunction==12) // MFVideoTransFunc_2020_const + ret = AVCOL_SPC_BT2020_CL; + else + ret = AVCOL_SPC_BT2020_NCL; + break; + } + + if (ret == AVCOL_SPC_UNSPECIFIED) + { + // if color space not known from transfer matrix, + // fall back to using primaries to guess color space + switch (fmt_info->VideoPrimaries) + { + case DXVA_VideoPrimaries_BT709: + ret = AVCOL_SPC_BT709; + break; + case DXVA_VideoPrimaries_BT470_2_SysM: + ret = AVCOL_SPC_FCC; + break; + case DXVA_VideoPrimaries_BT470_2_SysBG: + case DXVA_VideoPrimaries_EBU3213: // this is PAL + ret = AVCOL_SPC_BT470BG; + break; + case DXVA_VideoPrimaries_SMPTE170M: + case DXVA_VideoPrimaries_SMPTE_C: + ret = AVCOL_SPC_SMPTE170M; + break; + case DXVA_VideoPrimaries_SMPTE240M: + ret = AVCOL_SPC_SMPTE240M; + break; + } + } + + return ret; +} + +static enum AVColorPrimaries dshow_color_primaries(DXVA_ExtendedFormat* fmt_info) +{ + switch (fmt_info->VideoPrimaries) + { + case DXVA_VideoPrimaries_Unknown: + return AVCOL_PRI_UNSPECIFIED; + case DXVA_VideoPrimaries_reserved: + return AVCOL_PRI_RESERVED; + case DXVA_VideoPrimaries_BT709: + return AVCOL_PRI_BT709; + case DXVA_VideoPrimaries_BT470_2_SysM: + return AVCOL_PRI_BT470M; + case DXVA_VideoPrimaries_BT470_2_SysBG: + case DXVA_VideoPrimaries_EBU3213: // this is PAL + return AVCOL_PRI_BT470BG; + case DXVA_VideoPrimaries_SMPTE170M: + case DXVA_VideoPrimaries_SMPTE_C: + return AVCOL_PRI_SMPTE170M; + case DXVA_VideoPrimaries_SMPTE240M: + return AVCOL_PRI_SMPTE240M; + + // values from MediaFoundation SDK (mfobjects.h) + case 9: // MFVideoPrimaries_BT2020 + return AVCOL_PRI_BT2020; + case 10: // MFVideoPrimaries_XYZ + return AVCOL_PRI_SMPTE428; + case 11: // MFVideoPrimaries_DCI_P3 + return AVCOL_PRI_SMPTE431; + case 12: // MFVideoPrimaries_ACES (Academy Color Encoding System) + // not an FFmpeg color primary + return AVCOL_PRI_UNSPECIFIED; + + default: + return AVCOL_PRI_UNSPECIFIED; + } +} + +static enum AVColorTransferCharacteristic dshow_color_trc(DXVA_ExtendedFormat* fmt_info) +{ + switch (fmt_info->VideoTransferFunction) + { + case DXVA_VideoTransFunc_Unknown: + return AVCOL_TRC_UNSPECIFIED; + case DXVA_VideoTransFunc_10: + return AVCOL_TRC_LINEAR; + case DXVA_VideoTransFunc_18: + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case DXVA_VideoTransFunc_20: + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case DXVA_VideoTransFunc_22: + return AVCOL_TRC_GAMMA22; + case DXVA_VideoTransFunc_22_709: + return AVCOL_TRC_BT709; + case DXVA_VideoTransFunc_22_240M: + return AVCOL_TRC_SMPTE240M; + case DXVA_VideoTransFunc_22_8bit_sRGB: + return AVCOL_TRC_IEC61966_2_1; + case DXVA_VideoTransFunc_28: + return AVCOL_TRC_GAMMA28; + + // values from MediaFoundation SDK (mfobjects.h) + case 9: // MFVideoTransFunc_Log_100 + return AVCOL_TRC_LOG; + case 10: // MFVideoTransFunc_Log_316 + return AVCOL_TRC_LOG_SQRT; + case 11: // MFVideoTransFunc_709_sym + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case 12: // MFVideoTransFunc_2020_const + case 13: // MFVideoTransFunc_2020 + if (fmt_info->VideoTransferMatrix==5) // MFVideoTransferMatrix_BT2020_12 + return AVCOL_TRC_BT2020_12; + else + return AVCOL_TRC_BT2020_10; + case 14: // MFVideoTransFunc_26 + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case 15: // MFVideoTransFunc_2084 + return AVCOL_TRC_SMPTEST2084; + case 16: // MFVideoTransFunc_HLG + return AVCOL_TRC_ARIB_STD_B67; + case 17: // MFVideoTransFunc_10_rel + // not an FFmpeg transfer characteristic? Undocumented also by MS + return AVCOL_TRC_UNSPECIFIED; + + default: + return AVCOL_TRC_UNSPECIFIED; + } +} + +static enum AVChromaLocation dshow_chroma_loc(DXVA_ExtendedFormat* fmt_info) +{ + if (fmt_info->VideoChromaSubsampling == DXVA_VideoChromaSubsampling_Cosited) // that is: (DXVA_VideoChromaSubsampling_Horizontally_Cosited | DXVA_VideoChromaSubsampling_Vertically_Cosited | DXVA_VideoChromaSubsampling_Vertically_AlignedChromaPlanes) + return AVCHROMA_LOC_TOPLEFT; + else if (fmt_info->VideoChromaSubsampling == DXVA_VideoChromaSubsampling_MPEG1) // that is: DXVA_VideoChromaSubsampling_Vertically_AlignedChromaPlanes + return AVCHROMA_LOC_CENTER; + else if (fmt_info->VideoChromaSubsampling == DXVA_VideoChromaSubsampling_MPEG2) // that is: (DXVA_VideoChromaSubsampling_Horizontally_Cosited | DXVA_VideoChromaSubsampling_Vertically_AlignedChromaPlanes) + return AVCHROMA_LOC_LEFT; + else if (fmt_info->VideoChromaSubsampling == DXVA_VideoChromaSubsampling_DV_PAL)// that is: (DXVA_VideoChromaSubsampling_Horizontally_Cosited | DXVA_VideoChromaSubsampling_Vertically_Cosited) + return AVCHROMA_LOC_TOPLEFT; + else + // unknown + return AVCHROMA_LOC_UNSPECIFIED; +} + static int dshow_read_close(AVFormatContext *s) { @@ -350,6 +537,7 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, VIDEO_STREAM_CONFIG_CAPS *vcaps = caps; BITMAPINFOHEADER *bih; int64_t *fr; + DXVA_ExtendedFormat *extended_format_info = NULL; const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; #if DSHOWDEBUG ff_print_VIDEO_STREAM_CONFIG_CAPS(vcaps); @@ -362,6 +550,8 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, VIDEOINFOHEADER2 *v = (void *) type->pbFormat; fr = &v->AvgTimePerFrame; bih = &v->bmiHeader; + if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) + extended_format_info = (DXVA_ExtendedFormat*)&v->dwControlFlags; } else { goto next; } @@ -378,11 +568,40 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, } else { av_log(avctx, AV_LOG_INFO, " pixel_format=%s", av_get_pix_fmt_name(pix_fmt)); } - av_log(avctx, AV_LOG_INFO, " min s=%ldx%ld fps=%g max s=%ldx%ld fps=%g\n", + av_log(avctx, AV_LOG_INFO, " min s=%ldx%ld fps=%g max s=%ldx%ld fps=%g", vcaps->MinOutputSize.cx, vcaps->MinOutputSize.cy, 1e7 / vcaps->MaxFrameInterval, vcaps->MaxOutputSize.cx, vcaps->MaxOutputSize.cy, 1e7 / vcaps->MinFrameInterval); + if (extended_format_info) { + enum AVColorRange col_range = dshow_color_range(extended_format_info); + enum AVColorSpace col_space = dshow_color_space(extended_format_info); + enum AVColorPrimaries col_prim = dshow_color_primaries(extended_format_info); + enum AVColorTransferCharacteristic col_trc = dshow_color_trc(extended_format_info); + enum AVChromaLocation chroma_loc = dshow_chroma_loc(extended_format_info); + if (col_range != AVCOL_RANGE_UNSPECIFIED || col_space != AVCOL_SPC_UNSPECIFIED || col_prim != AVCOL_PRI_UNSPECIFIED || col_trc != AVCOL_TRC_UNSPECIFIED) { + const char* range = av_color_range_name(col_range); + const char* space = av_color_space_name(col_space); + const char* prim = av_color_primaries_name(col_prim); + const char* trc = av_color_transfer_name(col_trc); + av_log(avctx, AV_LOG_INFO, " (%s, %s/%s/%s", + range ? range : "unknown", + space ? space : "unknown", + prim ? prim : "unknown", + trc ? trc : "unknown"); + if (chroma_loc != AVCHROMA_LOC_UNSPECIFIED) { + const char* chroma = av_chroma_location_name(chroma_loc); + av_log(avctx, AV_LOG_INFO, ", %s", chroma ? chroma : "unknown"); + } + av_log(avctx, AV_LOG_INFO, ")"); + } + else if (chroma_loc != AVCHROMA_LOC_UNSPECIFIED) { + const char* chroma = av_chroma_location_name(chroma_loc); + av_log(avctx, AV_LOG_INFO, "(%s)", chroma ? chroma : "unknown"); + } + } + + av_log(avctx, AV_LOG_INFO, "\n"); continue; } if (ctx->video_codec_id != AV_CODEC_ID_RAWVIDEO) { @@ -959,6 +1178,7 @@ dshow_add_device(AVFormatContext *avctx, if (devtype == VideoDevice) { BITMAPINFOHEADER *bih = NULL; AVRational time_base; + DXVA_ExtendedFormat* extended_format_info = NULL; if (IsEqualGUID(&type.formattype, &FORMAT_VideoInfo)) { VIDEOINFOHEADER *v = (void *) type.pbFormat; @@ -968,6 +1188,8 @@ dshow_add_device(AVFormatContext *avctx, VIDEOINFOHEADER2 *v = (void *) type.pbFormat; time_base = (AVRational) { v->AvgTimePerFrame, 10000000 }; bih = &v->bmiHeader; + if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) + extended_format_info = (DXVA_ExtendedFormat*)&v->dwControlFlags; } if (!bih) { av_log(avctx, AV_LOG_ERROR, "Could not get media type.\n"); @@ -986,6 +1208,13 @@ dshow_add_device(AVFormatContext *avctx, av_log(avctx, AV_LOG_DEBUG, "attempt to use full range for HDYC...\n"); par->color_range = AVCOL_RANGE_MPEG; // just in case it needs this... } + if (extended_format_info) { + par->color_range = dshow_color_range(extended_format_info); + par->color_space = dshow_color_space(extended_format_info); + par->color_primaries = dshow_color_primaries(extended_format_info); + par->color_trc = dshow_color_trc(extended_format_info); + par->chroma_location = dshow_chroma_loc(extended_format_info); + } if (par->format == AV_PIX_FMT_NONE) { const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; par->codec_id = av_codec_get_id(tags, bih->biCompression); From patchwork Wed Jun 2 06:28:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 28043 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:b214:0:0:0:0:0 with SMTP id b20csp272253iof; Tue, 1 Jun 2021 23:59:22 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxqiy6BYVJqam0Sqi/xyqvki2vYCBHFwjRvT5wLln5J+KHz6ZVnmmY963087AzMiLEne7/q X-Received: by 2002:aa7:db16:: with SMTP id t22mr21842072eds.49.1622617162544; Tue, 01 Jun 2021 23:59:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1622617162; cv=none; d=google.com; s=arc-20160816; b=d2v4rjMYeOzqNnDpLx2sWml57VXaJzEzRNuhgVWf+9wXue8bZ1HcLIDM1TKr4vKy5J cQHHp1euaRcsityuBzgsbPuwxWOBfsvy5Mr/PfgwluCpB44nAp8FTv0QnB/oN0k1LfC4 g3pTha85Q5mgw75LOogUhcLxYuOu/ird1LOnSWXPFLzVcjirpIFmLCs2GZGQ1gaJtmNZ 4cquD93Eeh0LRGZSZ0dzkolaT5If0s/CUzJE1hefrWVp6l4H8JmaX7jxdIvnHYM2Z76d NO2iyzUkG5Ni1Qz/rBGYn3877vtri6I7LAEexFgSwkag8JVbOUtfZT0PKasxY4djq0b7 4OzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=HiiTpbtOoJa3N2fSSf30ScgW7C26Fccd14upWLPBus4=; b=lLcGUi1BqqK/m/BPsp7kAawrDwWH6cheCQHKgopRmTRMSmYxTqZnwdsg0xh+Ns98R6 UnWhn1wXXMB22uyxeP/n3gOAwrH2xdTiUQJAC35C+nl3x5qcQRlicR4zgWYqiMs7zY5w Hj4plTC2Q1B8lqZ9Q8sIsNgTfZzEINxbfXP8hDlISCt8d3wM4NYfkonl0WK215kvwmaq e9edQD6CZoge638CntFxG6ef3AfpZIHx6N5U1JDbl9jwj6MX7zIxVPZFZD/TJXdb2Hqs bcXcjqAYZuRKen0rVZanZ5L6T/aIsMylAMAcyQqt5TDIwALSnyioFVIMiuVT6YcEdCAE StWQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b="UN0/9WtK"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b15si1229170ede.378.2021.06.01.23.59.22; Tue, 01 Jun 2021 23:59:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b="UN0/9WtK"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1E3E6689F14; Wed, 2 Jun 2021 09:59:19 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 80AE968819D for ; Wed, 2 Jun 2021 09:59:11 +0300 (EEST) Received: by mail-ej1-f51.google.com with SMTP id c10so2205607eja.11 for ; Tue, 01 Jun 2021 23:59:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=V1XPuf0vhVhVwAkNfSfN/J+IPRd3idtcWVOP1QvSFlw=; b=UN0/9WtKLmSToARu4XiEx3JMZAAvS+A1Z8S/5B0IkAGTm/7yyLRvel0IUaU9QAOj+l 4y5ZE6nA3Od03DepHkwucsrl2MweZQhIT5CKFsp1t7Ywl4LEt0RHwUkIwQamN2s277Wn t2ySPrA73G8ojIQdh+OaCO4Eg8n0CkXgZMZO2FCmBbs+APinSBjcrw2NLfZqJaJCPugu z+kbCppurHXD3gzlrPqjXUiQB1edZa3CkNtL31elED1cU9GggzsUP8Q+AJRO1EaQWKb3 yvnzOs41Yi1IcYaZUOs/XO8T2nOadU5f7azTj1m/1uz8/OhYStVq0H5zBSvc3U9oIlUv ri+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=V1XPuf0vhVhVwAkNfSfN/J+IPRd3idtcWVOP1QvSFlw=; b=XdTPabJh32S8+f9w6wHHtmihyjr0uybBByhsN1jz7vv4kIRw1mJd9XA5oia6PMOSIL 5RnS5ujG53pxibFNn22PRFK6f1lL2gLwb8+tPhPu5aTJzmM3YDndSIlhAimTv9q4N4hK FTp/7kTZCAEXzblEno9PdgtC41FUfSdMYUHkb5ibcnwW/c/RrY3ha9ut7dG6bSYHd2kb 0sMoipICDqT0B7KOlrO+7tY+AtD//zQcxt/rmkDZT0P+S6vgTYG4SdypLvAyNkGfVjzH w+OiyjCf29aM7OozJJIImFuhFLuzhHfUefH04DhwZZJHvJI/ge1tgwmugSPNEYcSNIEU b4Bw== X-Gm-Message-State: AOAM531wx3Jhu0rw0K5oDbjO1sOtG/7FJy+2w7Vj03u5dHF8tYhVvpL2 dLuS17pFTFIZQn4alc9/3/WMV89x2QG2eQ== X-Received: by 2002:a2e:99cd:: with SMTP id l13mr24307039ljj.89.1622615358167; Tue, 01 Jun 2021 23:29:18 -0700 (PDT) Received: from localhost.localdomain (deedock.humlab.lu.se. [130.235.135.183]) by smtp.gmail.com with ESMTPSA id p6sm1901985lfr.163.2021.06.01.23.29.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Jun 2021 23:29:17 -0700 (PDT) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Wed, 2 Jun 2021 08:28:57 +0200 Message-Id: <20210602062857.1767-3-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20210602062857.1767-1-dcnieho@gmail.com> References: <20210602062857.1767-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] avdevice/dshow: select format with extended color info X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: IxnnPVMW5b5c Some DirectShow devices (Logitech C920 webcam) expose each DirectShow format they support twice, once without and once with extended color information. During format selection, both match, this patch ensures that the format with extended color information is selected if it is available, else it falls back to a matching format without such information. This also necessitated a new code path taken for default formats of a device (when user didn't request any specific video size, etc), because the default format may be one without extended color information when a twin with extended color information is also available. Getting the extended color information when available is important as it allows setting the color space, range, primaries, transfer characteristics and chroma location of the stream provided by dshow, enabling users to get more correct color automatically out of their device. Signed-off-by: Diederick Niehorster --- libavdevice/dshow.c | 448 ++++++++++++++++++++++++++++++++------------ 1 file changed, 323 insertions(+), 125 deletions(-) diff --git a/libavdevice/dshow.c b/libavdevice/dshow.c index c59cd30f30..7467fae1fc 100644 --- a/libavdevice/dshow.c +++ b/libavdevice/dshow.c @@ -23,6 +23,7 @@ #include "libavutil/parseutils.h" #include "libavutil/pixdesc.h" #include "libavutil/opt.h" +#include "libavutil/mem.h" #include "libavformat/internal.h" #include "libavformat/riff.h" #include "avdevice.h" @@ -499,6 +500,107 @@ fail1: return 0; } +static int dshow_should_set_format(AVFormatContext* avctx, enum dshowDeviceType devtype) +{ + struct dshow_ctx* ctx = avctx->priv_data; + + return (devtype == VideoDevice && (ctx->framerate || + (ctx->requested_width && ctx->requested_height) || + ctx->pixel_format != AV_PIX_FMT_NONE || + ctx->video_codec_id != AV_CODEC_ID_RAWVIDEO)) + || (devtype == AudioDevice && (ctx->channels || ctx->sample_size || ctx->sample_rate)); +} + + +struct dshow_format_info { + enum dshowDeviceType devtype; + // video + int64_t framerate; + enum AVPixelFormat pix_fmt; + enum AVCodecID codec_id; + enum AVColorRange col_range; + enum AVColorSpace col_space; + enum AVColorPrimaries col_prim; + enum AVColorTransferCharacteristic col_trc; + enum AVChromaLocation chroma_loc; + int width; + int height; + // audio + int sample_rate; + int sample_size; + int channels; +}; + +// user must av_free the returned pointer +static struct dshow_format_info* dshow_get_format_info(AM_MEDIA_TYPE* type) +{ + struct dshow_format_info *fmt_info = NULL; + BITMAPINFOHEADER *bih; + DXVA_ExtendedFormat *extended_format_info = NULL; + WAVEFORMATEX *fx; + enum dshowDeviceType devtype; + int64_t framerate; + + if (!type) + return NULL; + + if (IsEqualGUID(&type->formattype, &FORMAT_VideoInfo)) { + VIDEOINFOHEADER *v = (void *) type->pbFormat; + framerate = v->AvgTimePerFrame; + bih = &v->bmiHeader; + devtype = VideoDevice; + } else if (IsEqualGUID(&type->formattype, &FORMAT_VideoInfo2)) { + VIDEOINFOHEADER2 *v = (void *) type->pbFormat; + devtype = VideoDevice; + framerate = v->AvgTimePerFrame; + bih = &v->bmiHeader; + if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) + extended_format_info = (DXVA_ExtendedFormat *) &v->dwControlFlags; + } else if (IsEqualGUID(&type->formattype, &FORMAT_WaveFormatEx)) { + fx = (void *) type->pbFormat; + devtype = AudioDevice; + } else { + return NULL; + } + + fmt_info = av_mallocz(sizeof(struct dshow_format_info)); + if (!fmt_info) + return NULL; + // initialize fields where unset is not zero + fmt_info->pix_fmt = AV_PIX_FMT_NONE; + fmt_info->col_space = AVCOL_SPC_UNSPECIFIED; + fmt_info->col_prim = AVCOL_PRI_UNSPECIFIED; + fmt_info->col_trc = AVCOL_TRC_UNSPECIFIED; + // now get info about format + fmt_info->devtype = devtype; + if (devtype == VideoDevice) { + fmt_info->width = bih->biWidth; + fmt_info->height = bih->biHeight; + fmt_info->framerate = framerate; + fmt_info->pix_fmt = dshow_pixfmt(bih->biCompression, bih->biBitCount); + if (fmt_info->pix_fmt == AV_PIX_FMT_NONE) { + const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; + fmt_info->codec_id = av_codec_get_id(tags, bih->biCompression); + } + else + fmt_info->codec_id = AV_CODEC_ID_RAWVIDEO; + + if (extended_format_info) { + fmt_info->col_range = dshow_color_range(extended_format_info); + fmt_info->col_space = dshow_color_space(extended_format_info); + fmt_info->col_prim = dshow_color_primaries(extended_format_info); + fmt_info->col_trc = dshow_color_trc(extended_format_info); + fmt_info->chroma_loc = dshow_chroma_loc(extended_format_info); + } + } else { + fmt_info->sample_rate = fx->nSamplesPerSec; + fmt_info->sample_size = fx->wBitsPerSample; + fmt_info->channels = fx->nChannels; + } + + return fmt_info; +} + /** * Cycle through available formats using the specified pin, * try to set parameters specified through AVOptions and if successful @@ -512,9 +614,26 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, struct dshow_ctx *ctx = avctx->priv_data; IAMStreamConfig *config = NULL; AM_MEDIA_TYPE *type = NULL; + AM_MEDIA_TYPE *previous_match_type = NULL; int format_set = 0; void *caps = NULL; int i, n, size, r; + int wait_for_better = 0; + + // format parameters requested by user + // if none are requested by user, the values will below be set to + // those of the default format + // video + enum AVCodecID requested_video_codec_id = ctx->video_codec_id; + enum AVPixelFormat requested_pixel_format = ctx->pixel_format; + int64_t requested_framerate = ctx->framerate ? ((int64_t)ctx->requested_framerate.den * 10000000) + / ctx->requested_framerate.num : 0; + int requested_width = ctx->requested_width; + int requested_height = ctx->requested_height; + // audio + int requested_sample_rate = ctx->sample_rate; + int requested_sample_size = ctx->sample_size; + int requested_channels = ctx->channels; if (IPin_QueryInterface(pin, &IID_IAMStreamConfig, (void **) &config) != S_OK) return; @@ -525,7 +644,82 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, if (!caps) goto end; + /** + * If we should open the default device, then: + * 1. check what the format of the default device is, and + * 2. below we iterate all formats till we find a matching + * one, with most info exposed (see comment below). + */ + int use_default = !dshow_should_set_format(avctx, devtype); + if (use_default && pformat_set) + { + HRESULT hr; + + // get default + if ((hr = IAMStreamConfig_GetFormat(config, &type)) != S_OK) { + if (hr == E_NOTIMPL || !IsEqualGUID(&type->majortype, devtype==VideoDevice ? &MEDIATYPE_Video : &MEDIATYPE_Audio)) { + // default not available or of wrong type, + // fall back to iterating exposed formats + // until one of the right type is found + IEnumMediaTypes *types = NULL; + if (IPin_EnumMediaTypes(pin, &types) != S_OK) + goto end; + IEnumMediaTypes_Reset(types); + while (IEnumMediaTypes_Next(types, 1, &type, NULL) == S_OK) { + if (IsEqualGUID(&type->majortype, devtype==VideoDevice ? &MEDIATYPE_Video : &MEDIATYPE_Audio)) { + break; + } + CoTaskMemFree(type); + type = NULL; + } + IEnumMediaTypes_Release(types); + } + + if (!type) + // this pin does not expose any video formats + goto end; + } + + if (type) { + // interrogate default format, so we know what to search for below + struct dshow_format_info *fmt_info = dshow_get_format_info(type); + if (fmt_info) { + if (fmt_info->devtype == VideoDevice) + { + requested_video_codec_id = fmt_info->codec_id; + requested_pixel_format = fmt_info->pix_fmt; + requested_framerate = fmt_info->framerate; + requested_width = fmt_info->width; + requested_height = fmt_info->height; + } else { + requested_sample_rate = fmt_info->sample_rate; + requested_sample_size = fmt_info->sample_size; + requested_channels = fmt_info->channels; + } + av_free(fmt_info); + } + + if (type && type->pbFormat) + CoTaskMemFree(type->pbFormat); + CoTaskMemFree(type); + type = NULL; + if (!fmt_info) + // default format somehow invalid, can't continue with this pin + goto end; + } + } + + // NB: some devices (e.g. Logitech C920) expose each video format twice: + // both a format containing a VIDEOINFOHEADER and a format containing + // a VIDEOINFOHEADER2. We want, if possible, to select a format with a + // VIDEOINFOHEADER2, as this potentially provides more info about the + // format. So, if in the iteration below we have found a matching format, + // but it is a VIDEOINFOHEADER, keep looking for a matching format that + // exposes contains a VIDEOINFOHEADER2. Fall back to the VIDEOINFOHEADER + // format if no corresponding VIDEOINFOHEADER2 is found when we finish + // iterating. for (i = 0; i < n && !format_set; i++) { + struct dshow_format_info *fmt_info = NULL; r = IAMStreamConfig_GetStreamCaps(config, i, &type, (void *) caps); if (r != S_OK) goto next; @@ -533,101 +727,96 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, ff_print_AM_MEDIA_TYPE(type); #endif + fmt_info = dshow_get_format_info(type); + if (!fmt_info) + goto next; + if (devtype == VideoDevice) { VIDEO_STREAM_CONFIG_CAPS *vcaps = caps; BITMAPINFOHEADER *bih; int64_t *fr; - DXVA_ExtendedFormat *extended_format_info = NULL; - const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; #if DSHOWDEBUG ff_print_VIDEO_STREAM_CONFIG_CAPS(vcaps); #endif + + if (fmt_info->devtype != VideoDevice) + goto next; + if (IsEqualGUID(&type->formattype, &FORMAT_VideoInfo)) { VIDEOINFOHEADER *v = (void *) type->pbFormat; - fr = &v->AvgTimePerFrame; + fr = &v->AvgTimePerFrame; bih = &v->bmiHeader; + wait_for_better = 1; } else if (IsEqualGUID(&type->formattype, &FORMAT_VideoInfo2)) { VIDEOINFOHEADER2 *v = (void *) type->pbFormat; - fr = &v->AvgTimePerFrame; + fr = &v->AvgTimePerFrame; bih = &v->bmiHeader; - if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) - extended_format_info = (DXVA_ExtendedFormat*)&v->dwControlFlags; - } else { - goto next; + wait_for_better = 0; } + if (!pformat_set) { - enum AVPixelFormat pix_fmt = dshow_pixfmt(bih->biCompression, bih->biBitCount); - if (pix_fmt == AV_PIX_FMT_NONE) { - enum AVCodecID codec_id = av_codec_get_id(tags, bih->biCompression); - const AVCodec *codec = avcodec_find_decoder(codec_id); - if (codec_id == AV_CODEC_ID_NONE || !codec) { + if (fmt_info->pix_fmt == AV_PIX_FMT_NONE) { + const AVCodec *codec = avcodec_find_decoder(fmt_info->codec_id); + if (fmt_info->codec_id == AV_CODEC_ID_NONE || !codec) { av_log(avctx, AV_LOG_INFO, " unknown compression type 0x%X", (int) bih->biCompression); } else { av_log(avctx, AV_LOG_INFO, " vcodec=%s", codec->name); } } else { - av_log(avctx, AV_LOG_INFO, " pixel_format=%s", av_get_pix_fmt_name(pix_fmt)); + av_log(avctx, AV_LOG_INFO, " pixel_format=%s", av_get_pix_fmt_name(fmt_info->pix_fmt)); } av_log(avctx, AV_LOG_INFO, " min s=%ldx%ld fps=%g max s=%ldx%ld fps=%g", vcaps->MinOutputSize.cx, vcaps->MinOutputSize.cy, 1e7 / vcaps->MaxFrameInterval, vcaps->MaxOutputSize.cx, vcaps->MaxOutputSize.cy, 1e7 / vcaps->MinFrameInterval); - if (extended_format_info) { - enum AVColorRange col_range = dshow_color_range(extended_format_info); - enum AVColorSpace col_space = dshow_color_space(extended_format_info); - enum AVColorPrimaries col_prim = dshow_color_primaries(extended_format_info); - enum AVColorTransferCharacteristic col_trc = dshow_color_trc(extended_format_info); - enum AVChromaLocation chroma_loc = dshow_chroma_loc(extended_format_info); - if (col_range != AVCOL_RANGE_UNSPECIFIED || col_space != AVCOL_SPC_UNSPECIFIED || col_prim != AVCOL_PRI_UNSPECIFIED || col_trc != AVCOL_TRC_UNSPECIFIED) { - const char* range = av_color_range_name(col_range); - const char* space = av_color_space_name(col_space); - const char* prim = av_color_primaries_name(col_prim); - const char* trc = av_color_transfer_name(col_trc); - av_log(avctx, AV_LOG_INFO, " (%s, %s/%s/%s", - range ? range : "unknown", - space ? space : "unknown", - prim ? prim : "unknown", - trc ? trc : "unknown"); - if (chroma_loc != AVCHROMA_LOC_UNSPECIFIED) { - const char* chroma = av_chroma_location_name(chroma_loc); - av_log(avctx, AV_LOG_INFO, ", %s", chroma ? chroma : "unknown"); - } - av_log(avctx, AV_LOG_INFO, ")"); - } - else if (chroma_loc != AVCHROMA_LOC_UNSPECIFIED) { - const char* chroma = av_chroma_location_name(chroma_loc); - av_log(avctx, AV_LOG_INFO, "(%s)", chroma ? chroma : "unknown"); - } + + const char *chroma = av_chroma_location_name(fmt_info->chroma_loc); + if (fmt_info->col_range != AVCOL_RANGE_UNSPECIFIED || + fmt_info->col_space != AVCOL_SPC_UNSPECIFIED || + fmt_info->col_prim != AVCOL_PRI_UNSPECIFIED || + fmt_info->col_trc != AVCOL_TRC_UNSPECIFIED) { + const char *range = av_color_range_name(fmt_info->col_range); + const char *space = av_color_space_name(fmt_info->col_space); + const char *prim = av_color_primaries_name(fmt_info->col_prim); + const char *trc = av_color_transfer_name(fmt_info->col_trc); + av_log(avctx, AV_LOG_INFO, " (%s, %s/%s/%s", + range ? range : "unknown", + space ? space : "unknown", + prim ? prim : "unknown", + trc ? trc : "unknown"); + if (fmt_info->chroma_loc != AVCHROMA_LOC_UNSPECIFIED) + av_log(avctx, AV_LOG_INFO, ", %s", chroma ? chroma : "unknown"); + av_log(avctx, AV_LOG_INFO, ")"); } + else if (fmt_info->chroma_loc != AVCHROMA_LOC_UNSPECIFIED) + av_log(avctx, AV_LOG_INFO, "(%s)", chroma ? chroma : "unknown"); av_log(avctx, AV_LOG_INFO, "\n"); continue; } - if (ctx->video_codec_id != AV_CODEC_ID_RAWVIDEO) { - if (ctx->video_codec_id != av_codec_get_id(tags, bih->biCompression)) + if (requested_video_codec_id != AV_CODEC_ID_RAWVIDEO) { + if (requested_video_codec_id != fmt_info->codec_id) goto next; } - if (ctx->pixel_format != AV_PIX_FMT_NONE && - ctx->pixel_format != dshow_pixfmt(bih->biCompression, bih->biBitCount)) { + if (requested_pixel_format != AV_PIX_FMT_NONE && + requested_pixel_format != fmt_info->pix_fmt) { goto next; } - if (ctx->framerate) { - int64_t framerate = ((int64_t) ctx->requested_framerate.den*10000000) - / ctx->requested_framerate.num; - if (framerate > vcaps->MaxFrameInterval || - framerate < vcaps->MinFrameInterval) + if (requested_framerate) { + if (requested_framerate > vcaps->MaxFrameInterval || + requested_framerate < vcaps->MinFrameInterval) goto next; - *fr = framerate; + *fr = requested_framerate; } - if (ctx->requested_width && ctx->requested_height) { - if (ctx->requested_width > vcaps->MaxOutputSize.cx || - ctx->requested_width < vcaps->MinOutputSize.cx || - ctx->requested_height > vcaps->MaxOutputSize.cy || - ctx->requested_height < vcaps->MinOutputSize.cy) + if (requested_width && requested_height) { + if (requested_width > vcaps->MaxOutputSize.cx || + requested_width < vcaps->MinOutputSize.cx || + requested_height > vcaps->MaxOutputSize.cy || + requested_height < vcaps->MinOutputSize.cy) goto next; - bih->biWidth = ctx->requested_width; - bih->biHeight = ctx->requested_height; + bih->biWidth = requested_width; + bih->biHeight = requested_height; } } else { AUDIO_STREAM_CONFIG_CAPS *acaps = caps; @@ -646,34 +835,63 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, acaps->MaximumChannels, acaps->MaximumBitsPerSample, acaps->MaximumSampleFrequency); continue; } - if (ctx->sample_rate) { - if (ctx->sample_rate > acaps->MaximumSampleFrequency || - ctx->sample_rate < acaps->MinimumSampleFrequency) + if (requested_sample_rate) { + if (requested_sample_rate > acaps->MaximumSampleFrequency || + requested_sample_rate < acaps->MinimumSampleFrequency) goto next; - fx->nSamplesPerSec = ctx->sample_rate; + fx->nSamplesPerSec = requested_sample_rate; } - if (ctx->sample_size) { - if (ctx->sample_size > acaps->MaximumBitsPerSample || - ctx->sample_size < acaps->MinimumBitsPerSample) + if (requested_sample_size) { + if (requested_sample_size > acaps->MaximumBitsPerSample || + requested_sample_size < acaps->MinimumBitsPerSample) goto next; - fx->wBitsPerSample = ctx->sample_size; + fx->wBitsPerSample = requested_sample_size; } - if (ctx->channels) { - if (ctx->channels > acaps->MaximumChannels || - ctx->channels < acaps->MinimumChannels) + if (requested_channels) { + if (requested_channels > acaps->MaximumChannels || + requested_channels < acaps->MinimumChannels) goto next; - fx->nChannels = ctx->channels; + fx->nChannels = requested_channels; } } - if (IAMStreamConfig_SetFormat(config, type) != S_OK) - goto next; - format_set = 1; + + // found a matching format. Either apply or store + // for safekeeping if we might maybe find a better + // format with more info attached to it (see comment + // above loop) + if (!wait_for_better) { + if (IAMStreamConfig_SetFormat(config, type) != S_OK) + goto next; + format_set = 1; + } + else if (!previous_match_type) { + // store this matching format for possible later use. + // If we have already found a matching format, ignore it + previous_match_type = type; + type = NULL; + } next: - if (type->pbFormat) + if (fmt_info) + av_free(fmt_info); + if (type && type->pbFormat) CoTaskMemFree(type->pbFormat); CoTaskMemFree(type); } + // previously found a matching VIDEOINFOHEADER format and stored + // it for safe keeping. Searching further for a matching + // VIDEOINFOHEADER2 format yielded nothing. So set the pin's + // format based on the VIDEOINFOHEADER format. + // NB: this never applies to an audio format because + // previous_match_type always NULL in that case + if (!format_set && previous_match_type) { + if (IAMStreamConfig_SetFormat(config, previous_match_type) == S_OK) + format_set = 1; + } + end: + if (previous_match_type && previous_match_type->pbFormat) + CoTaskMemFree(previous_match_type->pbFormat); + CoTaskMemFree(previous_match_type); IAMStreamConfig_Release(config); av_free(caps); if (pformat_set) @@ -792,11 +1010,7 @@ dshow_cycle_pins(AVFormatContext *avctx, enum dshowDeviceType devtype, const char *devtypename = (devtype == VideoDevice) ? "video" : "audio only"; const char *sourcetypename = (sourcetype == VideoSourceDevice) ? "video" : "audio"; - int set_format = (devtype == VideoDevice && (ctx->framerate || - (ctx->requested_width && ctx->requested_height) || - ctx->pixel_format != AV_PIX_FMT_NONE || - ctx->video_codec_id != AV_CODEC_ID_RAWVIDEO)) - || (devtype == AudioDevice && (ctx->channels || ctx->sample_rate)); + int set_format = dshow_should_set_format(avctx, devtype); int format_set = 0; int should_show_properties = (devtype == VideoDevice) ? ctx->show_video_device_dialog : ctx->show_audio_device_dialog; @@ -816,9 +1030,7 @@ dshow_cycle_pins(AVFormatContext *avctx, enum dshowDeviceType devtype, while (!device_pin && IEnumPins_Next(pins, 1, &pin, NULL) == S_OK) { IKsPropertySet *p = NULL; - IEnumMediaTypes *types = NULL; PIN_INFO info = {0}; - AM_MEDIA_TYPE *type; GUID category; DWORD r2; char *name_buf = NULL; @@ -861,35 +1073,24 @@ dshow_cycle_pins(AVFormatContext *avctx, enum dshowDeviceType devtype, } } - if (set_format) { - dshow_cycle_formats(avctx, devtype, pin, &format_set); - if (!format_set) { - goto next; - } + // will either try to find format matching options supplied by user + // or try to open default format. Successful if returns with format_set==1 + dshow_cycle_formats(avctx, devtype, pin, &format_set); + if (!format_set) { + goto next; } + if (devtype == AudioDevice && ctx->audio_buffer_size) { if (dshow_set_audio_buffer_size(avctx, pin) < 0) { av_log(avctx, AV_LOG_ERROR, "unable to set audio buffer size %d to pin, using pin anyway...", ctx->audio_buffer_size); } } - if (IPin_EnumMediaTypes(pin, &types) != S_OK) - goto next; - - IEnumMediaTypes_Reset(types); - /* in case format_set was not called, just verify the majortype */ - while (!device_pin && IEnumMediaTypes_Next(types, 1, &type, NULL) == S_OK) { - if (IsEqualGUID(&type->majortype, mediatype[devtype])) { - device_pin = pin; - av_log(avctx, AV_LOG_DEBUG, "Selecting pin %s on %s\n", name_buf, devtypename); - goto next; - } - CoTaskMemFree(type); + if (format_set) { + device_pin = pin; + av_log(avctx, AV_LOG_DEBUG, "Selecting pin %s on %s\n", name_buf, devtypename); } - next: - if (types) - IEnumMediaTypes_Release(types); if (p) IKsPropertySet_Release(p); if (device_pin != pin) @@ -1159,6 +1360,7 @@ dshow_add_device(AVFormatContext *avctx, AM_MEDIA_TYPE type; AVCodecParameters *par; AVStream *st; + struct dshow_format_info *fmt_info = NULL; int ret = AVERROR(EIO); type.pbFormat = NULL; @@ -1173,12 +1375,14 @@ dshow_add_device(AVFormatContext *avctx, ctx->capture_filter[devtype]->stream_index = st->index; ff_dshow_pin_ConnectionMediaType(ctx->capture_pin[devtype], &type); + fmt_info = dshow_get_format_info(&type); + if (!fmt_info) + goto error; par = st->codecpar; if (devtype == VideoDevice) { BITMAPINFOHEADER *bih = NULL; AVRational time_base; - DXVA_ExtendedFormat* extended_format_info = NULL; if (IsEqualGUID(&type.formattype, &FORMAT_VideoInfo)) { VIDEOINFOHEADER *v = (void *) type.pbFormat; @@ -1188,8 +1392,6 @@ dshow_add_device(AVFormatContext *avctx, VIDEOINFOHEADER2 *v = (void *) type.pbFormat; time_base = (AVRational) { v->AvgTimePerFrame, 10000000 }; bih = &v->bmiHeader; - if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) - extended_format_info = (DXVA_ExtendedFormat*)&v->dwControlFlags; } if (!bih) { av_log(avctx, AV_LOG_ERROR, "Could not get media type.\n"); @@ -1200,33 +1402,21 @@ dshow_add_device(AVFormatContext *avctx, st->r_frame_rate = av_inv_q(time_base); par->codec_type = AVMEDIA_TYPE_VIDEO; - par->width = bih->biWidth; - par->height = bih->biHeight; + par->width = fmt_info->width; + par->height = fmt_info->height; par->codec_tag = bih->biCompression; - par->format = dshow_pixfmt(bih->biCompression, bih->biBitCount); + par->format = fmt_info->pix_fmt; if (bih->biCompression == MKTAG('H', 'D', 'Y', 'C')) { av_log(avctx, AV_LOG_DEBUG, "attempt to use full range for HDYC...\n"); par->color_range = AVCOL_RANGE_MPEG; // just in case it needs this... } - if (extended_format_info) { - par->color_range = dshow_color_range(extended_format_info); - par->color_space = dshow_color_space(extended_format_info); - par->color_primaries = dshow_color_primaries(extended_format_info); - par->color_trc = dshow_color_trc(extended_format_info); - par->chroma_location = dshow_chroma_loc(extended_format_info); - } - if (par->format == AV_PIX_FMT_NONE) { - const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; - par->codec_id = av_codec_get_id(tags, bih->biCompression); - if (par->codec_id == AV_CODEC_ID_NONE) { - av_log(avctx, AV_LOG_ERROR, "Unknown compression type. " - "Please report type 0x%X.\n", (int) bih->biCompression); - ret = AVERROR_PATCHWELCOME; - goto error; - } - par->bits_per_coded_sample = bih->biBitCount; - } else { - par->codec_id = AV_CODEC_ID_RAWVIDEO; + par->color_range = fmt_info->col_range; + par->color_space = fmt_info->col_space; + par->color_primaries = fmt_info->col_prim; + par->color_trc = fmt_info->col_trc; + par->chroma_location = fmt_info->chroma_loc; + par->codec_id = fmt_info->codec_id; + if (par->codec_id == AV_CODEC_ID_RAWVIDEO) { if (bih->biCompression == BI_RGB || bih->biCompression == BI_BITFIELDS) { par->bits_per_coded_sample = bih->biBitCount; if (par->height < 0) { @@ -1239,6 +1429,14 @@ dshow_add_device(AVFormatContext *avctx, } } } + } else { + if (par->codec_id == AV_CODEC_ID_NONE) { + av_log(avctx, AV_LOG_ERROR, "Unknown compression type. " + "Please report type 0x%X.\n", (int) bih->biCompression); + ret = AVERROR_PATCHWELCOME; + goto error; + } + par->bits_per_coded_sample = bih->biBitCount; } } else { WAVEFORMATEX *fx = NULL;