From patchwork Fri Jun 11 20:30:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Diederick C. Niehorster" X-Patchwork-Id: 28225 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5e:c91a:0:0:0:0:0 with SMTP id z26csp765133iol; Fri, 11 Jun 2021 13:36:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxmaaMv26xxOYsEVxf7JyrwVq1nn02OsikZ6Ifb494aTIz6TojTIYe8DX/9E54oPIs+464F X-Received: by 2002:a17:906:7c49:: with SMTP id g9mr5321872ejp.84.1623443814711; Fri, 11 Jun 2021 13:36:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1623443814; cv=none; d=google.com; s=arc-20160816; b=OoPkzGLYm4kgszXv6gSctf5E3fMRyqYjsUjLVEm5ivSjYJ/RiRcg1Bc6TyrsIwAuQ6 M35lxAleS0jz4IZTw1sTPsbcjTLOQY5Wm4zftCSV72cNpsHbqpg+TmSgIW724nGzf1tG N+5xps/o6zSYzcIIrEQ5rNNpaDLAWj7YNzLo6OpyVquo0JvINAAYLAQKLSyZ81qYiq+G zZKMZ1w/oEPJRD1oJZvrpHVSx25akwvNBedRGPgd6o4FjYXtN8NgrjvpNLFaEV4j9k96 VMdb31V124yXtcs+6WF1wet97HO/IpGzdB8fmE7VVqnFAYJLuQpaPT0dx2O0+nTmqaB2 ZIMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=P65C2ImregTsFGkfdUpKnEpqyTiE1dSi3DCGEN4NT4Y=; b=vktpivrbbI9K8WjsM0nZdfK2s+nzTpkQD25qDT4hJefZGKyl6d/ra5svO2VoJR97V1 u07HAwj+uUNVZEaVArD1Ojbh/ky978Ee2EhDirxmqoSspMEv/tBcnNuTHM+du9h4Zq2o qq8hsDJa8VQ+iO1nyda/nweCp4+cTw2KOxxh++B9/Nu4WITK0ijbmHHqv4NSyAxyyQsW dZ60jiB2wHVLGMs12cj9cBy3tQQfKE/kBLkwPF6HuJoAYLF3C78E9firsyKm0M+7iEGm KV9D91wBSey/0VL5gVfy+NFZoqWhP2VZr5wTbjFJkaFHOMMOFjh1htCzsWx5buaq+zdZ F5QA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=nQxaMOYe; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id yc20si5475915ejb.387.2021.06.11.13.36.54; Fri, 11 Jun 2021 13:36:54 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=nQxaMOYe; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5F794689884; Fri, 11 Jun 2021 23:36:51 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f179.google.com (mail-lj1-f179.google.com [209.85.208.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 026E3688388 for ; Fri, 11 Jun 2021 23:36:45 +0300 (EEST) Received: by mail-lj1-f179.google.com with SMTP id r16so11288459ljk.9 for ; Fri, 11 Jun 2021 13:36:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oFvt3auICRkG8MFKLrLyih0PjQJVmaYb9ShJQetAS5I=; b=nQxaMOYeoc3VvWm4yRNLH37eek14we63eZj6TGGH9R1lNosQTbqUKL4YMaJZcwMsxz LfyR7Ozqn0dNidDdRNuf4K+LWfr4UVigr+qkKlvy3jBLntT+fOqzU2vA6QKp26Gs3zcx EEfkP2ir8BBxK2L2Js2yoC2jVTty8YRfwRjPG+ohKJF/352x5WeKYvIYHU7buA0cee74 KZy/lUhOJB0QHBft172SK9fiDY+6GV6wwcbvl68uV6hsiD9d+PUWvkKWfqSMsLGVBL6Z w6gnQd6MZ8TKKqT9g733NNwT9lVhyndJCqFaX/gPRj1rlN8V07GOvlvvkQ4I1a4Bf9LL 0sNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oFvt3auICRkG8MFKLrLyih0PjQJVmaYb9ShJQetAS5I=; b=m0Eg1dK1YhEDMn+7ko40KnpYZErO6BOcNzUAizcwa9Za5PJD1MsdyQvOzDo5iBLelu dhEK7HrjDpEn8ZzXpZxP1BOBAxrIuMnN/VLUpzte8HwjJnU/YgEYo8rcjy2CIt2IuGAV AkzbgFRXs4oVt3WF7FQ8ef6kwXq+sxQ8wSAXu0da1sfdhaXya39t8BH8L1uETK1bEUa4 wFAO74B0GN0o6XjL9RM7kvyABNzYLs3t4xrkjz9/kx6HCvvx5vOOEbMBbG37KWDGceUl FbNGFW2qEsuaIy3r1LvecaC2uWZRPEKQkALn+FFdi7neu568sUC4N66A/Lc2zjl41vdr q6ew== X-Gm-Message-State: AOAM530nBXMp0aRUj3GkPfMuSLt1VRaiM8nR86stsBQ3hh0cM03/GFBk /33vF4KZonPlLvcj4El+vRQiaEnDlOe1Tw== X-Received: by 2002:a2e:b60d:: with SMTP id r13mr4466304ljn.314.1623443803729; Fri, 11 Jun 2021 13:36:43 -0700 (PDT) Received: from localhost.localdomain ([196.196.203.214]) by smtp.gmail.com with ESMTPSA id x19sm745216lfe.270.2021.06.11.13.36.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Jun 2021 13:36:43 -0700 (PDT) From: Diederick Niehorster To: ffmpeg-devel@ffmpeg.org Date: Fri, 11 Jun 2021 22:30:49 +0200 Message-Id: <20210611203104.1692-19-dcnieho@gmail.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20210611203104.1692-1-dcnieho@gmail.com> References: <20210611203104.1692-1-dcnieho@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 18/33] avdevice/dshow: select format with extended color info X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Diederick Niehorster Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: HicXypt151xv Some DirectShow devices (Logitech C920 webcam) expose each DirectShow format they support twice, once without and once with extended color information. During format selection, both match, this patch ensures that the format with extended color information is selected if it is available, else it falls back to a matching format without such information. This also necessitated a new code path taken for default formats of a device (when user didn't request any specific video size, etc), because the default format may be one without extended color information when a twin with extended color information is also available. Getting the extended color information when available is important as it allows setting the color space, range, primaries, transfer characteristics and chroma location of the stream provided by dshow, enabling users to get more correct color automatically out of their device. Closes: #9271 Signed-off-by: Diederick Niehorster --- libavdevice/dshow.c | 460 ++++++++++++++++++++++++++++++++------------ 1 file changed, 332 insertions(+), 128 deletions(-) diff --git a/libavdevice/dshow.c b/libavdevice/dshow.c index b3fcd9444e..34c7b63220 100644 --- a/libavdevice/dshow.c +++ b/libavdevice/dshow.c @@ -23,6 +23,7 @@ #include "libavutil/parseutils.h" #include "libavutil/pixdesc.h" #include "libavutil/opt.h" +#include "libavutil/mem.h" #include "libavformat/internal.h" #include "libavformat/riff.h" #include "avdevice.h" @@ -690,9 +691,111 @@ error: return ret; } +static int dshow_should_set_format(AVFormatContext* avctx, enum dshowDeviceType devtype) +{ + struct dshow_ctx* ctx = avctx->priv_data; + + return (devtype == VideoDevice && (ctx->framerate || + (ctx->requested_width && ctx->requested_height) || + ctx->pixel_format != AV_PIX_FMT_NONE || + ctx->video_codec_id != AV_CODEC_ID_RAWVIDEO)) + || (devtype == AudioDevice && (ctx->channels || ctx->sample_size || ctx->sample_rate)); +} + + +struct dshow_format_info { + enum dshowDeviceType devtype; + // video + int64_t framerate; + enum AVPixelFormat pix_fmt; + enum AVCodecID codec_id; + enum AVColorRange col_range; + enum AVColorSpace col_space; + enum AVColorPrimaries col_prim; + enum AVColorTransferCharacteristic col_trc; + enum AVChromaLocation chroma_loc; + int width; + int height; + // audio + int sample_rate; + int sample_size; + int channels; +}; + +// user must av_free the returned pointer +static struct dshow_format_info* dshow_get_format_info(AM_MEDIA_TYPE* type) +{ + struct dshow_format_info *fmt_info = NULL; + BITMAPINFOHEADER *bih; + DXVA2_ExtendedFormat *extended_format_info = NULL; + WAVEFORMATEX *fx; + enum dshowDeviceType devtype; + int64_t framerate; + + if (!type) + return NULL; + + if (IsEqualGUID(&type->formattype, &FORMAT_VideoInfo)) { + VIDEOINFOHEADER *v = (void *) type->pbFormat; + framerate = v->AvgTimePerFrame; + bih = &v->bmiHeader; + devtype = VideoDevice; + } else if (IsEqualGUID(&type->formattype, &FORMAT_VideoInfo2)) { + VIDEOINFOHEADER2 *v = (void *) type->pbFormat; + devtype = VideoDevice; + framerate = v->AvgTimePerFrame; + bih = &v->bmiHeader; + if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) + extended_format_info = (DXVA2_ExtendedFormat *) &v->dwControlFlags; + } else if (IsEqualGUID(&type->formattype, &FORMAT_WaveFormatEx)) { + fx = (void *) type->pbFormat; + devtype = AudioDevice; + } else { + return NULL; + } + + fmt_info = av_mallocz(sizeof(struct dshow_format_info)); + if (!fmt_info) + return NULL; + // initialize fields where unset is not zero + fmt_info->pix_fmt = AV_PIX_FMT_NONE; + fmt_info->col_space = AVCOL_SPC_UNSPECIFIED; + fmt_info->col_prim = AVCOL_PRI_UNSPECIFIED; + fmt_info->col_trc = AVCOL_TRC_UNSPECIFIED; + // now get info about format + fmt_info->devtype = devtype; + if (devtype == VideoDevice) { + fmt_info->width = bih->biWidth; + fmt_info->height = bih->biHeight; + fmt_info->framerate = framerate; + fmt_info->pix_fmt = dshow_pixfmt(bih->biCompression, bih->biBitCount); + if (fmt_info->pix_fmt == AV_PIX_FMT_NONE) { + const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; + fmt_info->codec_id = av_codec_get_id(tags, bih->biCompression); + } + else + fmt_info->codec_id = AV_CODEC_ID_RAWVIDEO; + + if (extended_format_info) { + fmt_info->col_range = dshow_color_range(extended_format_info); + fmt_info->col_space = dshow_color_space(extended_format_info); + fmt_info->col_prim = dshow_color_primaries(extended_format_info); + fmt_info->col_trc = dshow_color_trc(extended_format_info); + fmt_info->chroma_loc = dshow_chroma_loc(extended_format_info); + } + } else { + fmt_info->sample_rate = fx->nSamplesPerSec; + fmt_info->sample_size = fx->wBitsPerSample; + fmt_info->channels = fx->nChannels; + } + + return fmt_info; +} + /** - * Cycle through available formats using the specified pin, - * try to set parameters specified through AVOptions and if successful + * Cycle through available formats available from the specified pin, + * try to set parameters specified through AVOptions, or the pin's + * default format if no such parameters were set. If successful, * return 1 in *pformat_set. * If pformat_set is NULL, list all pin capabilities. */ @@ -703,9 +806,27 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, struct dshow_ctx *ctx = avctx->priv_data; IAMStreamConfig *config = NULL; AM_MEDIA_TYPE *type = NULL; + AM_MEDIA_TYPE *previous_match_type = NULL; int format_set = 0; void *caps = NULL; int i, n, size, r; + int wait_for_better = 0; + int use_default; + + // format parameters requested by user + // if none are requested by user, the values will below be set to + // those of the default format + // video + enum AVCodecID requested_video_codec_id = ctx->video_codec_id; + enum AVPixelFormat requested_pixel_format = ctx->pixel_format; + int64_t requested_framerate = ctx->framerate ? ((int64_t)ctx->requested_framerate.den * 10000000) + / ctx->requested_framerate.num : 0; + int requested_width = ctx->requested_width; + int requested_height = ctx->requested_height; + // audio + int requested_sample_rate = ctx->sample_rate; + int requested_sample_size = ctx->sample_size; + int requested_channels = ctx->channels; if (IPin_QueryInterface(pin, &IID_IAMStreamConfig, (void **) &config) != S_OK) return; @@ -716,7 +837,84 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, if (!caps) goto end; + /** + * If we should open the device with the default format, + * then: + * 1. check what the format of the default device is, and + * 2. below we iterate all formats till we find a matching + * one, with most info exposed (see comment below). + */ + use_default = !dshow_should_set_format(avctx, devtype); + if (use_default && pformat_set) + { + HRESULT hr; + + // get default + if ((hr = IAMStreamConfig_GetFormat(config, &type)) != S_OK) { + if (hr == E_NOTIMPL || !IsEqualGUID(&type->majortype, devtype==VideoDevice ? &MEDIATYPE_Video : &MEDIATYPE_Audio)) { + // default not available or of wrong type, + // fall back to iterating exposed formats + // until one of the right type is found + IEnumMediaTypes *types = NULL; + if (IPin_EnumMediaTypes(pin, &types) != S_OK) + goto end; + IEnumMediaTypes_Reset(types); + while (IEnumMediaTypes_Next(types, 1, &type, NULL) == S_OK) { + if (IsEqualGUID(&type->majortype, devtype==VideoDevice ? &MEDIATYPE_Video : &MEDIATYPE_Audio)) { + break; + } + CoTaskMemFree(type); + type = NULL; + } + IEnumMediaTypes_Release(types); + } + + if (!type) + // this pin does not expose any video formats + goto end; + } + + if (type) { + // interrogate default format, so we know what to search for below + struct dshow_format_info *fmt_info = dshow_get_format_info(type); + if (fmt_info) { + if (fmt_info->devtype == VideoDevice) + { + requested_video_codec_id = fmt_info->codec_id; + requested_pixel_format = fmt_info->pix_fmt; + requested_framerate = fmt_info->framerate; + requested_width = fmt_info->width; + requested_height = fmt_info->height; + } else { + requested_sample_rate = fmt_info->sample_rate; + requested_sample_size = fmt_info->sample_size; + requested_channels = fmt_info->channels; + } + av_free(fmt_info); // free but don't set to NULL to enable below check + } + + if (type && type->pbFormat) + CoTaskMemFree(type->pbFormat); + CoTaskMemFree(type); + type = NULL; + if (!fmt_info) + // default format somehow invalid, can't continue with this pin + goto end; + fmt_info = NULL; + } + } + + // NB: some devices (e.g. Logitech C920) expose each video format twice: + // both a format containing a VIDEOINFOHEADER and a format containing + // a VIDEOINFOHEADER2. We want, if possible, to select a format with a + // VIDEOINFOHEADER2, as this potentially provides more info about the + // format. So, if in the iteration below we have found a matching format, + // but it is a VIDEOINFOHEADER, keep looking for a matching format that + // exposes contains a VIDEOINFOHEADER2. Fall back to the VIDEOINFOHEADER + // format if no corresponding VIDEOINFOHEADER2 is found when we finish + // iterating. for (i = 0; i < n && !format_set; i++) { + struct dshow_format_info *fmt_info = NULL; r = IAMStreamConfig_GetStreamCaps(config, i, &type, (void *) caps); if (r != S_OK) goto next; @@ -724,101 +922,96 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, ff_print_AM_MEDIA_TYPE(type); #endif + fmt_info = dshow_get_format_info(type); + if (!fmt_info) + goto next; + if (devtype == VideoDevice) { VIDEO_STREAM_CONFIG_CAPS *vcaps = caps; BITMAPINFOHEADER *bih; int64_t *fr; - DXVA_ExtendedFormat *extended_format_info = NULL; - const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; #if DSHOWDEBUG ff_print_VIDEO_STREAM_CONFIG_CAPS(vcaps); #endif + + if (fmt_info->devtype != VideoDevice) + goto next; + if (IsEqualGUID(&type->formattype, &FORMAT_VideoInfo)) { VIDEOINFOHEADER *v = (void *) type->pbFormat; - fr = &v->AvgTimePerFrame; + fr = &v->AvgTimePerFrame; bih = &v->bmiHeader; + wait_for_better = 1; } else if (IsEqualGUID(&type->formattype, &FORMAT_VideoInfo2)) { VIDEOINFOHEADER2 *v = (void *) type->pbFormat; - fr = &v->AvgTimePerFrame; + fr = &v->AvgTimePerFrame; bih = &v->bmiHeader; - if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) - extended_format_info = (DXVA_ExtendedFormat*)&v->dwControlFlags; - } else { - goto next; + wait_for_better = 0; } + if (!pformat_set) { - enum AVPixelFormat pix_fmt = dshow_pixfmt(bih->biCompression, bih->biBitCount); - if (pix_fmt == AV_PIX_FMT_NONE) { - enum AVCodecID codec_id = av_codec_get_id(tags, bih->biCompression); - const AVCodec *codec = avcodec_find_decoder(codec_id); - if (codec_id == AV_CODEC_ID_NONE || !codec) { + if (fmt_info->pix_fmt == AV_PIX_FMT_NONE) { + const AVCodec *codec = avcodec_find_decoder(fmt_info->codec_id); + if (fmt_info->codec_id == AV_CODEC_ID_NONE || !codec) { av_log(avctx, AV_LOG_INFO, " unknown compression type 0x%X", (int) bih->biCompression); } else { av_log(avctx, AV_LOG_INFO, " vcodec=%s", codec->name); } } else { - av_log(avctx, AV_LOG_INFO, " pixel_format=%s", av_get_pix_fmt_name(pix_fmt)); + av_log(avctx, AV_LOG_INFO, " pixel_format=%s", av_get_pix_fmt_name(fmt_info->pix_fmt)); } av_log(avctx, AV_LOG_INFO, " min s=%ldx%ld fps=%g max s=%ldx%ld fps=%g", vcaps->MinOutputSize.cx, vcaps->MinOutputSize.cy, 1e7 / vcaps->MaxFrameInterval, vcaps->MaxOutputSize.cx, vcaps->MaxOutputSize.cy, 1e7 / vcaps->MinFrameInterval); - if (extended_format_info) { - enum AVColorRange col_range = dshow_color_range(extended_format_info); - enum AVColorSpace col_space = dshow_color_space(extended_format_info); - enum AVColorPrimaries col_prim = dshow_color_primaries(extended_format_info); - enum AVColorTransferCharacteristic col_trc = dshow_color_trc(extended_format_info); - enum AVChromaLocation chroma_loc = dshow_chroma_loc(extended_format_info); - if (col_range != AVCOL_RANGE_UNSPECIFIED || col_space != AVCOL_SPC_UNSPECIFIED || col_prim != AVCOL_PRI_UNSPECIFIED || col_trc != AVCOL_TRC_UNSPECIFIED) { - const char* range = av_color_range_name(col_range); - const char* space = av_color_space_name(col_space); - const char* prim = av_color_primaries_name(col_prim); - const char* trc = av_color_transfer_name(col_trc); - av_log(avctx, AV_LOG_INFO, " (%s, %s/%s/%s", - range ? range : "unknown", - space ? space : "unknown", - prim ? prim : "unknown", - trc ? trc : "unknown"); - if (chroma_loc != AVCHROMA_LOC_UNSPECIFIED) { - const char* chroma = av_chroma_location_name(chroma_loc); - av_log(avctx, AV_LOG_INFO, ", %s", chroma ? chroma : "unknown"); - } - av_log(avctx, AV_LOG_INFO, ")"); - } - else if (chroma_loc != AVCHROMA_LOC_UNSPECIFIED) { - const char* chroma = av_chroma_location_name(chroma_loc); - av_log(avctx, AV_LOG_INFO, "(%s)", chroma ? chroma : "unknown"); - } + + const char *chroma = av_chroma_location_name(fmt_info->chroma_loc); + if (fmt_info->col_range != AVCOL_RANGE_UNSPECIFIED || + fmt_info->col_space != AVCOL_SPC_UNSPECIFIED || + fmt_info->col_prim != AVCOL_PRI_UNSPECIFIED || + fmt_info->col_trc != AVCOL_TRC_UNSPECIFIED) { + const char *range = av_color_range_name(fmt_info->col_range); + const char *space = av_color_space_name(fmt_info->col_space); + const char *prim = av_color_primaries_name(fmt_info->col_prim); + const char *trc = av_color_transfer_name(fmt_info->col_trc); + av_log(avctx, AV_LOG_INFO, " (%s, %s/%s/%s", + range ? range : "unknown", + space ? space : "unknown", + prim ? prim : "unknown", + trc ? trc : "unknown"); + if (fmt_info->chroma_loc != AVCHROMA_LOC_UNSPECIFIED) + av_log(avctx, AV_LOG_INFO, ", %s", chroma ? chroma : "unknown"); + av_log(avctx, AV_LOG_INFO, ")"); } + else if (fmt_info->chroma_loc != AVCHROMA_LOC_UNSPECIFIED) + av_log(avctx, AV_LOG_INFO, "(%s)", chroma ? chroma : "unknown"); av_log(avctx, AV_LOG_INFO, "\n"); continue; } - if (ctx->video_codec_id != AV_CODEC_ID_RAWVIDEO) { - if (ctx->video_codec_id != av_codec_get_id(tags, bih->biCompression)) + if (requested_video_codec_id != AV_CODEC_ID_RAWVIDEO) { + if (requested_video_codec_id != fmt_info->codec_id) goto next; } - if (ctx->pixel_format != AV_PIX_FMT_NONE && - ctx->pixel_format != dshow_pixfmt(bih->biCompression, bih->biBitCount)) { + if (requested_pixel_format != AV_PIX_FMT_NONE && + requested_pixel_format != fmt_info->pix_fmt) { goto next; } - if (ctx->framerate) { - int64_t framerate = ((int64_t) ctx->requested_framerate.den*10000000) - / ctx->requested_framerate.num; - if (framerate > vcaps->MaxFrameInterval || - framerate < vcaps->MinFrameInterval) + if (requested_framerate) { + if (requested_framerate > vcaps->MaxFrameInterval || + requested_framerate < vcaps->MinFrameInterval) goto next; - *fr = framerate; + *fr = requested_framerate; } - if (ctx->requested_width && ctx->requested_height) { - if (ctx->requested_width > vcaps->MaxOutputSize.cx || - ctx->requested_width < vcaps->MinOutputSize.cx || - ctx->requested_height > vcaps->MaxOutputSize.cy || - ctx->requested_height < vcaps->MinOutputSize.cy) + if (requested_width && requested_height) { + if (requested_width > vcaps->MaxOutputSize.cx || + requested_width < vcaps->MinOutputSize.cx || + requested_height > vcaps->MaxOutputSize.cy || + requested_height < vcaps->MinOutputSize.cy) goto next; - bih->biWidth = ctx->requested_width; - bih->biHeight = ctx->requested_height; + bih->biWidth = requested_width; + bih->biHeight = requested_height; } } else { AUDIO_STREAM_CONFIG_CAPS *acaps = caps; @@ -837,34 +1030,63 @@ dshow_cycle_formats(AVFormatContext *avctx, enum dshowDeviceType devtype, acaps->MaximumChannels, acaps->MaximumBitsPerSample, acaps->MaximumSampleFrequency); continue; } - if (ctx->sample_rate) { - if (ctx->sample_rate > acaps->MaximumSampleFrequency || - ctx->sample_rate < acaps->MinimumSampleFrequency) + if (requested_sample_rate) { + if (requested_sample_rate > acaps->MaximumSampleFrequency || + requested_sample_rate < acaps->MinimumSampleFrequency) goto next; - fx->nSamplesPerSec = ctx->sample_rate; + fx->nSamplesPerSec = requested_sample_rate; } - if (ctx->sample_size) { - if (ctx->sample_size > acaps->MaximumBitsPerSample || - ctx->sample_size < acaps->MinimumBitsPerSample) + if (requested_sample_size) { + if (requested_sample_size > acaps->MaximumBitsPerSample || + requested_sample_size < acaps->MinimumBitsPerSample) goto next; - fx->wBitsPerSample = ctx->sample_size; + fx->wBitsPerSample = requested_sample_size; } - if (ctx->channels) { - if (ctx->channels > acaps->MaximumChannels || - ctx->channels < acaps->MinimumChannels) + if (requested_channels) { + if (requested_channels > acaps->MaximumChannels || + requested_channels < acaps->MinimumChannels) goto next; - fx->nChannels = ctx->channels; + fx->nChannels = requested_channels; } } - if (IAMStreamConfig_SetFormat(config, type) != S_OK) - goto next; - format_set = 1; + + // found a matching format. Either apply or store + // for safekeeping if we might maybe find a better + // format with more info attached to it (see comment + // above loop) + if (!wait_for_better) { + if (IAMStreamConfig_SetFormat(config, type) != S_OK) + goto next; + format_set = 1; + } + else if (!previous_match_type) { + // store this matching format for possible later use. + // If we have already found a matching format, ignore it + previous_match_type = type; + type = NULL; + } next: - if (type->pbFormat) + if (fmt_info) + av_free(fmt_info); + if (type && type->pbFormat) CoTaskMemFree(type->pbFormat); CoTaskMemFree(type); } + // previously found a matching VIDEOINFOHEADER format and stored + // it for safe keeping. Searching further for a matching + // VIDEOINFOHEADER2 format yielded nothing. So set the pin's + // format based on the VIDEOINFOHEADER format. + // NB: this never applies to an audio format because + // previous_match_type always NULL in that case + if (!format_set && previous_match_type) { + if (IAMStreamConfig_SetFormat(config, previous_match_type) == S_OK) + format_set = 1; + } + end: + if (previous_match_type && previous_match_type->pbFormat) + CoTaskMemFree(previous_match_type->pbFormat); + CoTaskMemFree(previous_match_type); IAMStreamConfig_Release(config); av_free(caps); if (pformat_set) @@ -983,11 +1205,7 @@ dshow_cycle_pins(AVFormatContext *avctx, enum dshowDeviceType devtype, const char *devtypename = (devtype == VideoDevice) ? "video" : "audio only"; const char *sourcetypename = (sourcetype == VideoSourceDevice) ? "video" : "audio"; - int set_format = (devtype == VideoDevice && (ctx->framerate || - (ctx->requested_width && ctx->requested_height) || - ctx->pixel_format != AV_PIX_FMT_NONE || - ctx->video_codec_id != AV_CODEC_ID_RAWVIDEO)) - || (devtype == AudioDevice && (ctx->channels || ctx->sample_rate)); + int set_format = dshow_should_set_format(avctx, devtype); int format_set = 0; int should_show_properties = (devtype == VideoDevice) ? ctx->show_video_device_dialog : ctx->show_audio_device_dialog; @@ -1007,9 +1225,7 @@ dshow_cycle_pins(AVFormatContext *avctx, enum dshowDeviceType devtype, while (!device_pin && IEnumPins_Next(pins, 1, &pin, NULL) == S_OK) { IKsPropertySet *p = NULL; - IEnumMediaTypes *types = NULL; PIN_INFO info = {0}; - AM_MEDIA_TYPE *type; GUID category; DWORD r2; char *name_buf = NULL; @@ -1052,35 +1268,24 @@ dshow_cycle_pins(AVFormatContext *avctx, enum dshowDeviceType devtype, } } - if (set_format) { - dshow_cycle_formats(avctx, devtype, pin, &format_set); - if (!format_set) { - goto next; - } + // will either try to find format matching options supplied by user + // or try to open default format. Successful if returns with format_set==1 + dshow_cycle_formats(avctx, devtype, pin, &format_set); + if (!format_set) { + goto next; } + if (devtype == AudioDevice && ctx->audio_buffer_size) { if (dshow_set_audio_buffer_size(avctx, pin) < 0) { av_log(avctx, AV_LOG_ERROR, "unable to set audio buffer size %d to pin, using pin anyway...", ctx->audio_buffer_size); } } - if (IPin_EnumMediaTypes(pin, &types) != S_OK) - goto next; - - IEnumMediaTypes_Reset(types); - /* in case format_set was not called, just verify the majortype */ - while (!device_pin && IEnumMediaTypes_Next(types, 1, &type, NULL) == S_OK) { - if (IsEqualGUID(&type->majortype, mediatype[devtype])) { - device_pin = pin; - av_log(avctx, AV_LOG_DEBUG, "Selecting pin %s on %s\n", name_buf, devtypename); - goto next; - } - CoTaskMemFree(type); + if (format_set) { + device_pin = pin; + av_log(avctx, AV_LOG_DEBUG, "Selecting pin %s on %s\n", name_buf, devtypename); } - next: - if (types) - IEnumMediaTypes_Release(types); if (p) IKsPropertySet_Release(p); if (device_pin != pin) @@ -1452,6 +1657,7 @@ dshow_add_device(AVFormatContext *avctx, AM_MEDIA_TYPE type; AVCodecParameters *par; AVStream *st; + struct dshow_format_info *fmt_info = NULL; int ret = AVERROR(EIO); type.pbFormat = NULL; @@ -1466,12 +1672,14 @@ dshow_add_device(AVFormatContext *avctx, ctx->capture_filter[devtype]->stream_index = st->index; ff_dshow_pin_ConnectionMediaType(ctx->capture_pin[devtype], &type); + fmt_info = dshow_get_format_info(&type); + if (!fmt_info) + goto error; par = st->codecpar; if (devtype == VideoDevice) { BITMAPINFOHEADER *bih = NULL; AVRational time_base; - DXVA_ExtendedFormat* extended_format_info = NULL; if (IsEqualGUID(&type.formattype, &FORMAT_VideoInfo)) { VIDEOINFOHEADER *v = (void *) type.pbFormat; @@ -1481,8 +1689,6 @@ dshow_add_device(AVFormatContext *avctx, VIDEOINFOHEADER2 *v = (void *) type.pbFormat; time_base = (AVRational) { v->AvgTimePerFrame, 10000000 }; bih = &v->bmiHeader; - if (v->dwControlFlags & AMCONTROL_COLORINFO_PRESENT) - extended_format_info = (DXVA_ExtendedFormat*)&v->dwControlFlags; } if (!bih) { av_log(avctx, AV_LOG_ERROR, "Could not get media type.\n"); @@ -1493,33 +1699,21 @@ dshow_add_device(AVFormatContext *avctx, st->r_frame_rate = av_inv_q(time_base); par->codec_type = AVMEDIA_TYPE_VIDEO; - par->width = bih->biWidth; - par->height = bih->biHeight; + par->width = fmt_info->width; + par->height = fmt_info->height; par->codec_tag = bih->biCompression; - par->format = dshow_pixfmt(bih->biCompression, bih->biBitCount); + par->format = fmt_info->pix_fmt; if (bih->biCompression == MKTAG('H', 'D', 'Y', 'C')) { av_log(avctx, AV_LOG_DEBUG, "attempt to use full range for HDYC...\n"); par->color_range = AVCOL_RANGE_MPEG; // just in case it needs this... } - if (extended_format_info) { - par->color_range = dshow_color_range(extended_format_info); - par->color_space = dshow_color_space(extended_format_info); - par->color_primaries = dshow_color_primaries(extended_format_info); - par->color_trc = dshow_color_trc(extended_format_info); - par->chroma_location = dshow_chroma_loc(extended_format_info); - } - if (par->format == AV_PIX_FMT_NONE) { - const AVCodecTag *const tags[] = { avformat_get_riff_video_tags(), NULL }; - par->codec_id = av_codec_get_id(tags, bih->biCompression); - if (par->codec_id == AV_CODEC_ID_NONE) { - av_log(avctx, AV_LOG_ERROR, "Unknown compression type. " - "Please report type 0x%X.\n", (int) bih->biCompression); - ret = AVERROR_PATCHWELCOME; - goto error; - } - par->bits_per_coded_sample = bih->biBitCount; - } else { - par->codec_id = AV_CODEC_ID_RAWVIDEO; + par->color_range = fmt_info->col_range; + par->color_space = fmt_info->col_space; + par->color_primaries = fmt_info->col_prim; + par->color_trc = fmt_info->col_trc; + par->chroma_location = fmt_info->chroma_loc; + par->codec_id = fmt_info->codec_id; + if (par->codec_id == AV_CODEC_ID_RAWVIDEO) { if (bih->biCompression == BI_RGB || bih->biCompression == BI_BITFIELDS) { par->bits_per_coded_sample = bih->biBitCount; if (par->height < 0) { @@ -1532,6 +1726,14 @@ dshow_add_device(AVFormatContext *avctx, } } } + } else { + if (par->codec_id == AV_CODEC_ID_NONE) { + av_log(avctx, AV_LOG_ERROR, "Unknown compression type. " + "Please report type 0x%X.\n", (int) bih->biCompression); + ret = AVERROR_PATCHWELCOME; + goto error; + } + par->bits_per_coded_sample = bih->biBitCount; } } else { WAVEFORMATEX *fx = NULL; @@ -1556,6 +1758,8 @@ dshow_add_device(AVFormatContext *avctx, ret = 0; error: + if (fmt_info) + av_free(fmt_info); if (type.pbFormat) CoTaskMemFree(type.pbFormat); return ret; @@ -1858,8 +2062,8 @@ const AVInputFormat ff_dshow_demuxer = { .read_header = dshow_read_header, .read_packet = dshow_read_packet, .read_close = dshow_read_close, - .get_device_list= dshow_get_device_list, .control_message = dshow_control_message, + .get_device_list= dshow_get_device_list, .flags = AVFMT_NOFILE | AVFMT_NOBINSEARCH | AVFMT_NOGENSEARCH | AVFMT_NO_BYTE_SEEK, .priv_class = &dshow_class, };