From patchwork Mon Jun 7 23:03:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 28131 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:b214:0:0:0:0:0 with SMTP id b20csp3885749iof; Mon, 7 Jun 2021 16:04:48 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyKr4FKFJgeykBQYqiNMS1+abvcRPGk9vrRGFJ2Y+lmxJYuZ5NZkO+RrB0q4QvbOZnLZ6Cm X-Received: by 2002:aa7:c4d0:: with SMTP id p16mr23038693edr.150.1623107088683; Mon, 07 Jun 2021 16:04:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1623107088; cv=none; d=google.com; s=arc-20160816; b=BrtSb4Lf7ilQjQH402BrynDru9U/Mk5HElnL9ad7TDhfnVo05ZJ+yXZO09W8fpLUNR w/Efj9T3xL+i6JEQ/gy/8EBpa+19XpRYdfXW530IrCJEW9QPnzZOBvnyxrum4KkqdO+T DF4CT/BkfSaXnYPsMQsB1DIH5YZLNpXXxdo4JNXDCq1RdQeoEBDHgrN4OoLZxRPyEmyl 6NrbE9ZjGaVJZjcsXstQY+grUH+qYumBsHTN33/bbugoCPMKbYhraSWPXYs3Up2V2Ndz A4aNM3jfGIcqonFvkxmeYX5FGu6nZ1G0+uu/i+yIbJaD9HlyF0AHQtp2nssUZMotDQIu mPUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=ZLt/rt4xdHB+vBqnMSwbD9hg0pCPouK/I8kzuUQJ1Eo=; b=a7KKIVOs+/U/4uhjpE9m9i4paL4Lfvpnl71JAvI/fe5LhVA4kNonuWYr4FcQEnfOTm mQKV/ij5Fb06SCmM3VzZnXt4EFNSRL/xPIecNvLV32uCvTSBeHydKIgG3KyrKbHWc9yg IE1bt+3nTOhn1dAbshoN1CDGA0VT13YNnBy9PWGqv1rQBcCLilpi90Ue+4kA42Wzb/K0 WNxyVDLt0BiD4CuQb4yUPqifqaLBHPsTxJP1rvBSUlBk/JPbBs3OuAzftemLYOt65+h4 bH1ZYnaTGvVB+lUjC6d6JRx6KnRkFKp9r3DQIvI4/tHzDh1XWXr2NSOEJoHS8d8fVXj8 a5XQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=JoO91yxw; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e7si11182939eds.607.2021.06.07.16.04.48; Mon, 07 Jun 2021 16:04:48 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=JoO91yxw; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0C6EB6808D9; Tue, 8 Jun 2021 02:04:45 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f181.google.com (mail-lj1-f181.google.com [209.85.208.181]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 32F866802B6 for ; Tue, 8 Jun 2021 02:04:43 +0300 (EEST) Received: by mail-lj1-f181.google.com with SMTP id r16so5242946ljk.9 for ; Mon, 07 Jun 2021 16:04:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iHdAuhOAnpZYrrkQnkinXR2KoaQk82/x/x4nyoZNn0I=; b=JoO91yxwUFNb7B/0flXY6oOkNSwQwe81VyQKBikRNGpsI9s/7GElWzHCkaXZHUPX2D 1BGfUjSOdsTrIOT+QeAVZWDu7jpEVYHim01dGdOd1CmsFHkuDKkWvb+RNJ0YYHJ1RG4+ 3n9Nkgk7OfwwwfcYA17GUmktgRTIbFOwA7bXjHLAN4ivq2Waftfcvx8uErJqtj7kAI56 JjVewFsolFU5KepczHcklYsk5FDzwq6ly31n0nfiq1R1NiTtM6uBXf+kAWx5oJpf1RuY Sd/St0qGVLBJg9uqDPNEY2exfPECfY+zOdpjoTh8mpMO5tu+CNNqU8kGokFXOFpXjT7d MJEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iHdAuhOAnpZYrrkQnkinXR2KoaQk82/x/x4nyoZNn0I=; b=sd8iLDgC7ZIX3NgL9ciuqBD3tHRhdcSWYH5H+xCf3L37Q91u60wRhAgo7vf6K/FNpK iWeemM5GCwnSlHxwTaZkYLKcGSx9WKlVOuWifBWZ88UH+KgiOyXJDBZZptfRWeUmE5kz MirayhTDOdj0L/sZW80K/CkiHcco6YQcl9/3jFN5CqsWZ/eU57ChetXIkQEBhr8YMmnu hruwFyotaYP14Km09T+vn2TYPkXWcomVv1k0JH4s9E+HqsNNDd0RQEp3tMNjp6gUiJCg Bp+XGzZYXQwL43GD1p5KY9dh9Jv5QA1MHO6++6yw3aOc/VIZaIMPCQ5s8bc+y2QE9I46 ioWg== X-Gm-Message-State: AOAM532oEBxeuGmF02k68h0ufXD4aWMtiyr0tpCkhkbXpsQNO84UWeSf K8q8LTr0+66LwovRy3VP0U4P+QACcLrpEQ== X-Received: by 2002:a2e:155e:: with SMTP id 30mr59858ljv.316.1623107082950; Mon, 07 Jun 2021 16:04:42 -0700 (PDT) Received: from localhost.localdomain (84-217-56-54.customers.ownit.se. [84.217.56.54]) by smtp.gmail.com with ESMTPSA id v9sm1999563ljv.131.2021.06.07.16.04.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Jun 2021 16:04:42 -0700 (PDT) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Tue, 8 Jun 2021 01:03:56 +0200 Message-Id: <20210607230414.612-18-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20210607230414.612-1-dcnieho@gmail.com> References: <20210607230414.612-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 17/35] avdevice/dshow: discover source color range/space/etc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster , Valerii Zapodovnikov Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: HbELDXK5hSCr Enabled discovering a DirectShow device's color range, space, primaries, transfer characteristics and chroma location, if the device exposes that information. Sets them in the stream's codecpars. Signed-off-by: Diederick Niehorster Co-authored-by: Valerii Zapodovnikov --- libavdevice/dshow.c | 231 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 230 insertions(+), 1 deletion(-) diff --git a/libavdevice/dshow.c b/libavdevice/dshow.c index d8a97797cc..6a7bac579a 100644 --- a/libavdevice/dshow.c +++ b/libavdevice/dshow.c @@ -29,6 +29,7 @@ #include "libavcodec/raw.h" #include "objidl.h" #include "shlwapi.h" +#include "dxva.h" static enum AVPixelFormat dshow_pixfmt(DWORD biCompression, WORD biBitCount) @@ -54,6 +55,192 @@ static enum AVPixelFormat dshow_pixfmt(DWORD biCompression, WORD biBitCount) return avpriv_find_pix_fmt(avpriv_get_raw_pix_fmt_tags(), biCompression); // all others } +static enum AVColorRange dshow_color_range(DXVA_ExtendedFormat* fmt_info) +{ + switch (fmt_info->NominalRange) + { + case DXVA_NominalRange_Unknown: + return AVCOL_RANGE_UNSPECIFIED; + case DXVA_NominalRange_Normal: // equal to DXVA_NominalRange_0_255 + return AVCOL_RANGE_JPEG; + case DXVA_NominalRange_Wide: // equal to DXVA_NominalRange_16_235 + return AVCOL_RANGE_MPEG; + case DXVA_NominalRange_48_208: + // not an ffmpeg color range + return AVCOL_RANGE_UNSPECIFIED; + + // values from MediaFoundation SDK (mfobjects.h) + case 4: // MFNominalRange_64_127 + // not an ffmpeg color range + return AVCOL_RANGE_UNSPECIFIED; + + default: + return DXVA_NominalRange_Unknown; + } +} + +static enum AVColorSpace dshow_color_space(DXVA_ExtendedFormat* fmt_info) +{ + enum AVColorSpace ret = AVCOL_SPC_UNSPECIFIED; + + switch (fmt_info->VideoTransferMatrix) + { + case DXVA_VideoTransferMatrix_BT709: + ret = AVCOL_SPC_BT709; + break; + case DXVA_VideoTransferMatrix_BT601: + ret = AVCOL_SPC_BT470BG; + break; + case DXVA_VideoTransferMatrix_SMPTE240M: + ret = AVCOL_SPC_SMPTE240M; + break; + + // values from MediaFoundation SDK (mfobjects.h) + case 4: // MFVideoTransferMatrix_BT2020_10 + case 5: // MFVideoTransferMatrix_BT2020_12 + if (fmt_info->VideoTransferFunction==12) // MFVideoTransFunc_2020_const + ret = AVCOL_SPC_BT2020_CL; + else + ret = AVCOL_SPC_BT2020_NCL; + break; + } + + if (ret == AVCOL_SPC_UNSPECIFIED) + { + // if color space not known from transfer matrix, + // fall back to using primaries to guess color space + switch (fmt_info->VideoPrimaries) + { + case DXVA_VideoPrimaries_BT709: + ret = AVCOL_SPC_BT709; + break; + case DXVA_VideoPrimaries_BT470_2_SysM: + ret = AVCOL_SPC_FCC; + break; + case DXVA_VideoPrimaries_BT470_2_SysBG: + case DXVA_VideoPrimaries_EBU3213: // this is PAL + ret = AVCOL_SPC_BT470BG; + break; + case DXVA_VideoPrimaries_SMPTE170M: + case DXVA_VideoPrimaries_SMPTE_C: + ret = AVCOL_SPC_SMPTE170M; + break; + case DXVA_VideoPrimaries_SMPTE240M: + ret = AVCOL_SPC_SMPTE240M; + break; + } + } + + return ret; +} + +static enum AVColorPrimaries dshow_color_primaries(DXVA_ExtendedFormat* fmt_info) +{ + switch (fmt_info->VideoPrimaries) + { + case DXVA_VideoPrimaries_Unknown: + return AVCOL_PRI_UNSPECIFIED; + case DXVA_VideoPrimaries_reserved: + return AVCOL_PRI_RESERVED; + case DXVA_VideoPrimaries_BT709: + return AVCOL_PRI_BT709; + case DXVA_VideoPrimaries_BT470_2_SysM: + return AVCOL_PRI_BT470M; + case DXVA_VideoPrimaries_BT470_2_SysBG: + case DXVA_VideoPrimaries_EBU3213: // this is PAL + return AVCOL_PRI_BT470BG; + case DXVA_VideoPrimaries_SMPTE170M: + case DXVA_VideoPrimaries_SMPTE_C: + return AVCOL_PRI_SMPTE170M; + case DXVA_VideoPrimaries_SMPTE240M: + return AVCOL_PRI_SMPTE240M; + + // values from MediaFoundation SDK (mfobjects.h) + case 9: // MFVideoPrimaries_BT2020 + return AVCOL_PRI_BT2020; + case 10: // MFVideoPrimaries_XYZ + return AVCOL_PRI_SMPTE428; + case 11: // MFVideoPrimaries_DCI_P3 + return AVCOL_PRI_SMPTE431; + case 12: // MFVideoPrimaries_ACES (Academy Color Encoding System) + // not an FFmpeg color primary + return AVCOL_PRI_UNSPECIFIED; + + default: + return AVCOL_PRI_UNSPECIFIED; + } +} + +static enum AVColorTransferCharacteristic dshow_color_trc(DXVA_ExtendedFormat* fmt_info) +{ + switch (fmt_info->VideoTransferFunction) + { + case DXVA_VideoTransFunc_Unknown: + return AVCOL_TRC_UNSPECIFIED; + case DXVA_VideoTransFunc_10: + return AVCOL_TRC_LINEAR; + case DXVA_VideoTransFunc_18: + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case DXVA_VideoTransFunc_20: + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case DXVA_VideoTransFunc_22: + return AVCOL_TRC_GAMMA22; + case DXVA_VideoTransFunc_22_709: + return AVCOL_TRC_BT709; + case DXVA_VideoTransFunc_22_240M: + return AVCOL_TRC_SMPTE240M; + case DXVA_VideoTransFunc_22_8bit_sRGB: + return AVCOL_TRC_IEC61966_2_1; + case DXVA_VideoTransFunc_28: + return AVCOL_TRC_GAMMA28; + + // values from MediaFoundation SDK (mfobjects.h) + case 9: // MFVideoTransFunc_Log_100 + return AVCOL_TRC_LOG; + case 10: // MFVideoTransFunc_Log_316 + return AVCOL_TRC_LOG_SQRT; + case 11: // MFVideoTransFunc_709_sym + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case 12: // MFVideoTransFunc_2020_const + case 13: // MFVideoTransFunc_2020 + if (fmt_info->VideoTransferMatrix==5) // MFVideoTransferMatrix_BT2020_12 + return AVCOL_TRC_BT2020_12; + else + return AVCOL_TRC_BT2020_10; + case 14: // MFVideoTransFunc_26 + // not an FFmpeg transfer characteristic + return AVCOL_TRC_UNSPECIFIED; + case 15: // MFVideoTransFunc_2084 + return AVCOL_TRC_SMPTEST2084; + case 16: // MFVideoTransFunc_HLG + return AVCOL_TRC_ARIB_STD_B67; + case 17: // MFVideoTransFunc_10_rel + // not an FFmpeg transfer characteristic? Undocumented also by MS + return AVCOL_TRC_UNSPECIFIED; + + default: + return AVCOL_TRC_UNSPECIFIED; + } +} + +static enum AVChromaLocation dshow_chroma_loc(DXVA_ExtendedFormat* fmt_info) +{ + if (fmt_info->VideoChromaSubsampling == DXVA_VideoChromaSubsampling_Cosited) // that is: (DXVA_VideoChromaSubsampling_Horizontally_Cosited | DXVA_VideoChromaSubsampling_Vertically_Cosited | DXVA_VideoChromaSubsampling_Vertically_AlignedChromaPlanes) + return AVCHROMA_LOC_TOPLEFT; + else if (fmt_info->VideoChromaSubsampling == DXVA_VideoChromaSubsampling_MPEG1) // that is: DXVA_VideoChromaSubsampling_Vertically_AlignedChromaPlanes + return AVCHROMA_LOC_CENTER; + else if (fmt_info->VideoChromaSubsampling == DXVA_VideoChromaSubsampling_MPEG2) // that is: (DXVA_VideoChromaSubsampling_Horizontally_Cosited | DXVA_VideoChromaSubsampling_Vertically_AlignedChromaPlanes) + return AVCHROMA_LOC_LEFT; + else if (fmt_info->VideoChromaSubsampling == DXVA_VideoChromaSubsampling_DV_PAL)// that is: (DXVA_VideoChromaSubsampling_Horizontally_Cosited | DXVA_VideoChromaSubsampling_Vertically_Cosited) + return AVCHROMA_LOC_TOPLEFT; + else + // unknown + return AVCHROMA_LOC_UNSPECIFIED; +} + static int dshow_read_close(AVFormatContext *s) { @@ -522,6 +709,7 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, VIDEO_STREAM_CONFIG_CAPS *vcaps = caps; BITMAPINFOHEADER *bih; int64_t *fr; + DXVA_ExtendedFormat *extended_format_info = NULL; const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; #if DSHOWDEBUG ff_print_VIDEO_STREAM_CONFIG_CAPS(vcaps); @@ -534,6 +722,8 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, VIDEOINFOHEADER2 *v = (void *) type->pbFormat; fr = &v->AvgTimePerFrame; bih = &v->bmiHeader; + if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) + extended_format_info = (DXVA_ExtendedFormat*)&v->dwControlFlags; } else { goto next; } @@ -550,11 +740,40 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, } else { av_log(avctx, AV_LOG_INFO, " pixel_format=%s", av_get_pix_fmt_name(pix_fmt)); } - av_log(avctx, AV_LOG_INFO, " min s=%ldx%ld fps=%g max s=%ldx%ld fps=%g\n", + av_log(avctx, AV_LOG_INFO, " min s=%ldx%ld fps=%g max s=%ldx%ld fps=%g", vcaps->MinOutputSize.cx, vcaps->MinOutputSize.cy, 1e7 / vcaps->MaxFrameInterval, vcaps->MaxOutputSize.cx, vcaps->MaxOutputSize.cy, 1e7 / vcaps->MinFrameInterval); + if (extended_format_info) { + enum AVColorRange col_range = dshow_color_range(extended_format_info); + enum AVColorSpace col_space = dshow_color_space(extended_format_info); + enum AVColorPrimaries col_prim = dshow_color_primaries(extended_format_info); + enum AVColorTransferCharacteristic col_trc = dshow_color_trc(extended_format_info); + enum AVChromaLocation chroma_loc = dshow_chroma_loc(extended_format_info); + if (col_range != AVCOL_RANGE_UNSPECIFIED || col_space != AVCOL_SPC_UNSPECIFIED || col_prim != AVCOL_PRI_UNSPECIFIED || col_trc != AVCOL_TRC_UNSPECIFIED) { + const char* range = av_color_range_name(col_range); + const char* space = av_color_space_name(col_space); + const char* prim = av_color_primaries_name(col_prim); + const char* trc = av_color_transfer_name(col_trc); + av_log(avctx, AV_LOG_INFO, " (%s, %s/%s/%s", + range ? range : "unknown", + space ? space : "unknown", + prim ? prim : "unknown", + trc ? trc : "unknown"); + if (chroma_loc != AVCHROMA_LOC_UNSPECIFIED) { + const char* chroma = av_chroma_location_name(chroma_loc); + av_log(avctx, AV_LOG_INFO, ", %s", chroma ? chroma : "unknown"); + } + av_log(avctx, AV_LOG_INFO, ")"); + } + else if (chroma_loc != AVCHROMA_LOC_UNSPECIFIED) { + const char* chroma = av_chroma_location_name(chroma_loc); + av_log(avctx, AV_LOG_INFO, "(%s)", chroma ? chroma : "unknown"); + } + } + + av_log(avctx, AV_LOG_INFO, "\n"); continue; } if (ctx->video_codec_id != AV_CODEC_ID_RAWVIDEO) { @@ -1233,6 +1452,7 @@ dshow_add_device(AVFormatContext *avctx, if (devtype == VideoDevice) { BITMAPINFOHEADER *bih = NULL; AVRational time_base; + DXVA_ExtendedFormat* extended_format_info = NULL; if (IsEqualGUID(&type.formattype, &FORMAT_VideoInfo)) { VIDEOINFOHEADER *v = (void *) type.pbFormat; @@ -1242,6 +1462,8 @@ dshow_add_device(AVFormatContext *avctx, VIDEOINFOHEADER2 *v = (void *) type.pbFormat; time_base = (AVRational) { v->AvgTimePerFrame, 10000000 }; bih = &v->bmiHeader; + if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) + extended_format_info = (DXVA_ExtendedFormat*)&v->dwControlFlags; } if (!bih) { av_log(avctx, AV_LOG_ERROR, "Could not get media type.\n"); @@ -1260,6 +1482,13 @@ dshow_add_device(AVFormatContext *avctx, av_log(avctx, AV_LOG_DEBUG, "attempt to use full range for HDYC...\n"); par->color_range = AVCOL_RANGE_MPEG; // just in case it needs this... } + if (extended_format_info) { + par->color_range = dshow_color_range(extended_format_info); + par->color_space = dshow_color_space(extended_format_info); + par->color_primaries = dshow_color_primaries(extended_format_info); + par->color_trc = dshow_color_trc(extended_format_info); + par->chroma_location = dshow_chroma_loc(extended_format_info); + } if (par->format == AV_PIX_FMT_NONE) { const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; par->codec_id = av_codec_get_id(tags, bih->biCompression);