From patchwork Wed Jun 2 06:28:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 28042 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:b214:0:0:0:0:0 with SMTP id b20csp273057iof; Wed, 2 Jun 2021 00:00:36 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx87Un/TS60vzGZ6c02COgmyBUKDwD2Xf45CQLmQJ3QD+aVPiFSfCwj4zaKWQsLtB7K5AR+ X-Received: by 2002:a05:6402:4316:: with SMTP id m22mr2022104edc.316.1622617236226; Wed, 02 Jun 2021 00:00:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1622617236; cv=none; d=google.com; s=arc-20160816; b=SPCa+UGNnpK4b4On53qHhIT/Bt43IH+6ubzDKw5mxDM34xrbaOkuB9eQxvdAGFSMPL lTRR+QcOEmDmHWwxgr/QL6MgzNsz5E57jk5lBS1/LCwENLoKSEGgfmcYYyb6G/E3slUT q85lxLYuIb4vffSFJalVwl1vaVIapqg2C499YIOgh6LAL+ibKj5bV4mQ6QNueuCvCGPV 2WkAZ/ZeTj+wYA4vpSh6YJ1YVH6Li6Itg8Snp9egulx3eEcTvE1ZOpwE6uRUkbYdiL17 CWuvO2TustkMfKXsSaqVIuXZuBlpJ8vJEolkXSRILPKor9urvZWd8eroKvRpmhUhSVS/ ahkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=nAtdlG9eWzyGnEysA3G+epQs0phQX8djfzFLi5HdEJo=; b=ZRJ9MpHDly7wOJUOh5OMdYIpMHy6r9gyN40kOTFwqaSed3Bb0m8om7u3Sk5YwkmU5H xcO+ueWEEOMpB+EGVJsbACNdoSqbMgajBW6nAxom1qZvwnNE0451R7lQRC8w20re+J98 HtieHCyy/IcQTF4GmP+HTCAOSlLMNJaCSyvKf4ncOwS4KaBWu5hP1K+fVp5QOOpwh/sF 31Mpbh3v63b0opohLqAgvJN/uoqmIy3cRxxkhWP2sStWNsl0SKUaAzkwH+OIUq6RxASw lvbzUeo6E8w51YUvLoQ5/XU1O4Jd1/jHfdjE9setQVY1ivh8bEDLic9GR5hXdodQuAMK 0JwA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=Q5JH97hG; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id w7si17865483ejy.722.2021.06.02.00.00.35; Wed, 02 Jun 2021 00:00:36 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=Q5JH97hG; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 55AA9689F86; Wed, 2 Jun 2021 10:00:33 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 73701680998 for ; Wed, 2 Jun 2021 10:00:27 +0300 (EEST) Received: by mail-lf1-f42.google.com with SMTP id v8so1727975lft.8 for ; Wed, 02 Jun 2021 00:00:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KN6yGgfmMG2RXOylGaDbAtJ5zpM3FTO7q7fd7zO7IgM=; b=Q5JH97hGDbA6u2IeAVDjCOQMPMjSvnp95syZNUSF5KMAMwBPcrPQM/kbPa6ERw1J8t lZdtW7KviSnLQEEcx1BjYRQ1iApqGJ+8JmS5EHYbu02WjkQdsaMMVYomZT4+afkdynEn Ij8PDmg2uVpVG3D/KMR5bI5ygyLSC3/zLtBaznqVydIuxeWhkFjb+MTrXjK9cTWpkPdp kpdpwbBdgYeIQv0ezYfdNwwuJUWJqwBmyhS9vjJ3q4sterQ5/1YHzaZsf5c8HAGGvZMe DkaJaF5fYciY/sh4uaeugfuQjHxL/9EdcVMiZZsFI1UobGVZqJdXc5gcsv9LA8DgkIvT M7AA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KN6yGgfmMG2RXOylGaDbAtJ5zpM3FTO7q7fd7zO7IgM=; b=ew5M/YAnqypyI1iC53GOZvRLqPJ1DDM6h2grPgI2Bzg6K5uGXRONgYE6qjebYhNSjY t8+1osPuf15+iK0191qKuRMT95NFJyQf69YmY7MujPZsaFJ2KFgnGnJGfmVS8KQ8VoJL T7N/eDNozLT/q65jS+wBYB7NsD7rh7c8W6ssS9iw7LguY2fSdb+8nXkciRcamCPsZjIV Foq14cRC2gj2YAZk+ZjudHzXcpgGFat3OJVYmb8N48/28kCMgsVikPxo3cqkudit0Huo B3lTi1TSQzOJGPYlye/Yk+afoBzWW8gdMigcofUzC/7FImahzbJvl11pIPlAJ4EmltVg /YJA== X-Gm-Message-State: AOAM533dJkEFSp1Z072eccin55dLjn9Umg+AZR4BgPWUrFkBYoCpgQe0 LsEwqnY0uQfjAkxRr8liIlMwsqt3T/3+kg== X-Received: by 2002:a2e:808f:: with SMTP id i15mr23817604ljg.376.1622615357376; Tue, 01 Jun 2021 23:29:17 -0700 (PDT) Received: from localhost.localdomain (deedock.humlab.lu.se. [130.235.135.183]) by smtp.gmail.com with ESMTPSA id p6sm1901985lfr.163.2021.06.01.23.29.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Jun 2021 23:29:16 -0700 (PDT) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Wed, 2 Jun 2021 08:28:56 +0200 Message-Id: <20210602062857.1767-2-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20210602062857.1767-1-dcnieho@gmail.com> References: <20210602062857.1767-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] avdevice/dshow: discover source color range/space/etc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: tBs5SZ89ltmH Enabled discovering a DirectShow device's color range, space, primaries, transfer characteristics and chroma location, if the device exposes that information. Sets them in the stream's codecpars. Signed-off-by: Diederick Niehorster --- libavdevice/dshow.c | 231 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 230 insertions(+), 1 deletion(-) diff --git a/libavdevice/dshow.c b/libavdevice/dshow.c index 8d0a6fcc09..c59cd30f30 100644 --- a/libavdevice/dshow.c +++ b/libavdevice/dshow.c @@ -29,6 +29,7 @@ #include "libavcodec/raw.h" #include "objidl.h" #include "shlwapi.h" +#include "dxva.h" static enum AVPixelFormat dshow_pixfmt(DWORD biCompression, WORD biBitCount) @@ -54,6 +55,192 @@ static enum AVPixelFormat dshow_pixfmt(DWORD biCompression, WORD biBitCount) return avpriv_find_pix_fmt(avpriv_get_raw_pix_fmt_tags(), biCompression); // all others } +static enum AVColorRange dshow_color_range(DXVA_ExtendedFormat* fmt_info) +{ + switch (fmt_info->NominalRange) + { + case DXVA_NominalRange_Unknown: + return AVCOL_RANGE_UNSPECIFIED; + case DXVA_NominalRange_Normal: // equal to DXVA_NominalRange_0_255 + return AVCOL_RANGE_JPEG; + case DXVA_NominalRange_Wide: // equal to DXVA_NominalRange_16_235 + return AVCOL_RANGE_MPEG; + case DXVA_NominalRange_48_208: + // not an ffmpeg color range + return AVCOL_RANGE_UNSPECIFIED; + + // values from MediaFoundation SDK (mfobjects.h) + case 4: // MFNominalRange_64_127 + // not an ffmpeg color range + return AVCOL_RANGE_UNSPECIFIED; + + default: + return DXVA_NominalRange_Unknown; + } +} + +static enum AVColorSpace dshow_color_space(DXVA_ExtendedFormat* fmt_info) +{ + enum AVColorSpace ret = AVCOL_SPC_UNSPECIFIED; + + switch (fmt_info->VideoTransferMatrix) + { + case DXVA_VideoTransferMatrix_BT709: + ret = AVCOL_SPC_BT709; + break; + case DXVA_VideoTransferMatrix_BT601: + ret = AVCOL_SPC_BT470BG; + break; + case DXVA_VideoTransferMatrix_SMPTE240M: + ret = AVCOL_SPC_SMPTE240M; + break; + + // values from MediaFoundation SDK (mfobjects.h) + case 4: // MFVideoTransferMatrix_BT2020_10 + case 5: // MFVideoTransferMatrix_BT2020_12 + if (fmt_info->VideoTransferFunction==12) // MFVideoTransFunc_2020_const + ret = AVCOL_SPC_BT2020_CL; + else + ret = AVCOL_SPC_BT2020_NCL; + break; + } + + if (ret == AVCOL_SPC_UNSPECIFIED) + { + // if color space not known from transfer matrix, + // fall back to using primaries to guess color space + switch (fmt_info->VideoPrimaries) + { + case DXVA_VideoPrimaries_BT709: + ret = AVCOL_SPC_BT709; + break; + case DXVA_VideoPrimaries_BT470_2_SysM: + ret = AVCOL_SPC_FCC; + break; + case DXVA_VideoPrimaries_BT470_2_SysBG: + case DXVA_VideoPrimaries_EBU3213: // this is PAL + ret = AVCOL_SPC_BT470BG; + break; + case DXVA_VideoPrimaries_SMPTE170M: + case DXVA_VideoPrimaries_SMPTE_C: + ret = AVCOL_SPC_SMPTE170M; + break; + case DXVA_VideoPrimaries_SMPTE240M: + ret = AVCOL_SPC_SMPTE240M; + break; + } + } + + return ret; +} + +static enum AVColorPrimaries dshow_color_primaries(DXVA_ExtendedFormat* fmt_info) +{ + switch (fmt_info->VideoPrimaries) + { + case DXVA_VideoPrimaries_Unknown: + return AVCOL_PRI_UNSPECIFIED; + case DXVA_VideoPrimaries_reserved: + return AVCOL_PRI_RESERVED; + case DXVA_VideoPrimaries_BT709: + return AVCOL_PRI_BT709; + case DXVA_VideoPrimaries_BT470_2_SysM: + return AVCOL_PRI_BT470M; + case DXVA_VideoPrimaries_BT470_2_SysBG: + case DXVA_VideoPrimaries_EBU3213: // this is PAL + return AVCOL_PRI_BT470BG; + case DXVA_VideoPrimaries_SMPTE170M: + case DXVA_VideoPrimaries_SMPTE_C: + return AVCOL_PRI_SMPTE170M; + case DXVA_VideoPrimaries_SMPTE240M: + return AVCOL_PRI_SMPTE240M; + + // values from MediaFoundation SDK (mfobjects.h) + case 9: // MFVideoPrimaries_BT2020 + return AVCOL_PRI_BT2020; + case 10: // MFVideoPrimaries_XYZ + return AVCOL_PRI_SMPTE428; + case 11: // MFVideoPrimaries_DCI_P3 + return AVCOL_PRI_SMPTE431; + case 12: // MFVideoPrimaries_ACES (Academy Color Encoding System) + // not an FFmpeg color primary + return AVCOL_PRI_UNSPECIFIED; + + default: + return AVCOL_PRI_UNSPECIFIED; + } +} + +static enum AVColorTransferCharacteristic dshow_color_trc(DXVA_ExtendedFormat* fmt_info) +{ + switch (fmt_info->VideoTransferFunction) + { + case DXVA_VideoTransFunc_Unknown: + return AVCOL_TRC_UNSPECIFIED; + case DXVA_VideoTransFunc_10: + return AVCOL_TRC_LINEAR; + case DXVA_VideoTransFunc_18: + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case DXVA_VideoTransFunc_20: + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case DXVA_VideoTransFunc_22: + return AVCOL_TRC_GAMMA22; + case DXVA_VideoTransFunc_22_709: + return AVCOL_TRC_BT709; + case DXVA_VideoTransFunc_22_240M: + return AVCOL_TRC_SMPTE240M; + case DXVA_VideoTransFunc_22_8bit_sRGB: + return AVCOL_TRC_IEC61966_2_1; + case DXVA_VideoTransFunc_28: + return AVCOL_TRC_GAMMA28; + + // values from MediaFoundation SDK (mfobjects.h) + case 9: // MFVideoTransFunc_Log_100 + return AVCOL_TRC_LOG; + case 10: // MFVideoTransFunc_Log_316 + return AVCOL_TRC_LOG_SQRT; + case 11: // MFVideoTransFunc_709_sym + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case 12: // MFVideoTransFunc_2020_const + case 13: // MFVideoTransFunc_2020 + if (fmt_info->VideoTransferMatrix==5) // MFVideoTransferMatrix_BT2020_12 + return AVCOL_TRC_BT2020_12; + else + return AVCOL_TRC_BT2020_10; + case 14: // MFVideoTransFunc_26 + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case 15: // MFVideoTransFunc_2084 + return AVCOL_TRC_SMPTEST2084; + case 16: // MFVideoTransFunc_HLG + return AVCOL_TRC_ARIB_STD_B67; + case 17: // MFVideoTransFunc_10_rel + // not an FFmpeg transfer characteristic? Undocumented also by MS + return AVCOL_TRC_UNSPECIFIED; + + default: + return AVCOL_TRC_UNSPECIFIED; + } +} + +static enum AVChromaLocation dshow_chroma_loc(DXVA_ExtendedFormat* fmt_info) +{ + if (fmt_info->VideoChromaSubsampling == DXVA_VideoChromaSubsampling_Cosited) // that is: (DXVA_VideoChromaSubsampling_Horizontally_Cosited | DXVA_VideoChromaSubsampling_Vertically_Cosited | DXVA_VideoChromaSubsampling_Vertically_AlignedChromaPlanes) + return AVCHROMA_LOC_TOPLEFT; + else if (fmt_info->VideoChromaSubsampling == DXVA_VideoChromaSubsampling_MPEG1) // that is: DXVA_VideoChromaSubsampling_Vertically_AlignedChromaPlanes + return AVCHROMA_LOC_CENTER; + else if (fmt_info->VideoChromaSubsampling == DXVA_VideoChromaSubsampling_MPEG2) // that is: (DXVA_VideoChromaSubsampling_Horizontally_Cosited | DXVA_VideoChromaSubsampling_Vertically_AlignedChromaPlanes) + return AVCHROMA_LOC_LEFT; + else if (fmt_info->VideoChromaSubsampling == DXVA_VideoChromaSubsampling_DV_PAL)// that is: (DXVA_VideoChromaSubsampling_Horizontally_Cosited | DXVA_VideoChromaSubsampling_Vertically_Cosited) + return AVCHROMA_LOC_TOPLEFT; + else + // unknown + return AVCHROMA_LOC_UNSPECIFIED; +} + static int dshow_read_close(AVFormatContext *s) { @@ -350,6 +537,7 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, VIDEO_STREAM_CONFIG_CAPS *vcaps = caps; BITMAPINFOHEADER *bih; int64_t *fr; + DXVA_ExtendedFormat *extended_format_info = NULL; const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; #if DSHOWDEBUG ff_print_VIDEO_STREAM_CONFIG_CAPS(vcaps); @@ -362,6 +550,8 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, VIDEOINFOHEADER2 *v = (void *) type->pbFormat; fr = &v->AvgTimePerFrame; bih = &v->bmiHeader; + if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) + extended_format_info = (DXVA_ExtendedFormat*)&v->dwControlFlags; } else { goto next; } @@ -378,11 +568,40 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, } else { av_log(avctx, AV_LOG_INFO, " pixel_format=%s", av_get_pix_fmt_name(pix_fmt)); } - av_log(avctx, AV_LOG_INFO, " min s=%ldx%ld fps=%g max s=%ldx%ld fps=%g\n", + av_log(avctx, AV_LOG_INFO, " min s=%ldx%ld fps=%g max s=%ldx%ld fps=%g", vcaps->MinOutputSize.cx, vcaps->MinOutputSize.cy, 1e7 / vcaps->MaxFrameInterval, vcaps->MaxOutputSize.cx, vcaps->MaxOutputSize.cy, 1e7 / vcaps->MinFrameInterval); + if (extended_format_info) { + enum AVColorRange col_range = dshow_color_range(extended_format_info); + enum AVColorSpace col_space = dshow_color_space(extended_format_info); + enum AVColorPrimaries col_prim = dshow_color_primaries(extended_format_info); + enum AVColorTransferCharacteristic col_trc = dshow_color_trc(extended_format_info); + enum AVChromaLocation chroma_loc = dshow_chroma_loc(extended_format_info); + if (col_range != AVCOL_RANGE_UNSPECIFIED || col_space != AVCOL_SPC_UNSPECIFIED || col_prim != AVCOL_PRI_UNSPECIFIED || col_trc != AVCOL_TRC_UNSPECIFIED) { + const char* range = av_color_range_name(col_range); + const char* space = av_color_space_name(col_space); + const char* prim = av_color_primaries_name(col_prim); + const char* trc = av_color_transfer_name(col_trc); + av_log(avctx, AV_LOG_INFO, " (%s, %s/%s/%s", + range ? range : "unknown", + space ? space : "unknown", + prim ? prim : "unknown", + trc ? trc : "unknown"); + if (chroma_loc != AVCHROMA_LOC_UNSPECIFIED) { + const char* chroma = av_chroma_location_name(chroma_loc); + av_log(avctx, AV_LOG_INFO, ", %s", chroma ? chroma : "unknown"); + } + av_log(avctx, AV_LOG_INFO, ")"); + } + else if (chroma_loc != AVCHROMA_LOC_UNSPECIFIED) { + const char* chroma = av_chroma_location_name(chroma_loc); + av_log(avctx, AV_LOG_INFO, "(%s)", chroma ? chroma : "unknown"); + } + } + + av_log(avctx, AV_LOG_INFO, "\n"); continue; } if (ctx->video_codec_id != AV_CODEC_ID_RAWVIDEO) { @@ -959,6 +1178,7 @@ dshow_add_device(AVFormatContext *avctx, if (devtype == VideoDevice) { BITMAPINFOHEADER *bih = NULL; AVRational time_base; + DXVA_ExtendedFormat* extended_format_info = NULL; if (IsEqualGUID(&type.formattype, &FORMAT_VideoInfo)) { VIDEOINFOHEADER *v = (void *) type.pbFormat; @@ -968,6 +1188,8 @@ dshow_add_device(AVFormatContext *avctx, VIDEOINFOHEADER2 *v = (void *) type.pbFormat; time_base = (AVRational) { v->AvgTimePerFrame, 10000000 }; bih = &v->bmiHeader; + if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) + extended_format_info = (DXVA_ExtendedFormat*)&v->dwControlFlags; } if (!bih) { av_log(avctx, AV_LOG_ERROR, "Could not get media type.\n"); @@ -986,6 +1208,13 @@ dshow_add_device(AVFormatContext *avctx, av_log(avctx, AV_LOG_DEBUG, "attempt to use full range for HDYC...\n"); par->color_range = AVCOL_RANGE_MPEG; // just in case it needs this... } + if (extended_format_info) { + par->color_range = dshow_color_range(extended_format_info); + par->color_space = dshow_color_space(extended_format_info); + par->color_primaries = dshow_color_primaries(extended_format_info); + par->color_trc = dshow_color_trc(extended_format_info); + par->chroma_location = dshow_chroma_loc(extended_format_info); + } if (par->format == AV_PIX_FMT_NONE) { const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; par->codec_id = av_codec_get_id(tags, bih->biCompression);