From patchwork Fri Apr 29 10:45:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 35505 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a885:b0:7f:4be2:bd17 with SMTP id ca5csp212204pzb; Fri, 29 Apr 2022 03:45:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzuGg0EXStqoSfm0dMoiJSyJPwqnD6GplIHPldv2smpyS6VMWYNxNXhPo544cfNRZRX0xO4 X-Received: by 2002:a50:fa94:0:b0:425:e8ac:3fb4 with SMTP id w20-20020a50fa94000000b00425e8ac3fb4mr26401279edr.43.1651229152433; Fri, 29 Apr 2022 03:45:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1651229152; cv=none; d=google.com; s=arc-20160816; b=xlPwVueC5WK2PLQ/PKoUCpbACYcMYLzBswPV8aGSUJP/arC2dZycogfk7WR1gDgqv1 oS/5u2HL5jtan0+jRg5680XBm6MKKmrJ6k8Ay2Q3ANOATie6Tr5zUuaag847z+E0SMzK MLvyOzCRX7J/teMlZ0yAuXRLOLjuhWvTK6dB7/8uZoTcGNufPA3Q2D2apXS4v6CPB8UW HU4x6dF+AnAobxIdSxQUiMQT2DYx+dKru9+3ajN/Sqz4Gf+mIGwKAhoOEEc/HMYcB6zV YqmgAWyubnCketi6fQfdxhvamd14G1KT4iaV/vp+A2ha/zm2im2Zc1rHXTq7wxDcWXW+ knKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=aCPxCnxEeF37YteMzEx3V64C6TM+ZT7Ds2NeYosbAxQ=; b=djR1NPclVHvXN8UadsKz8CAcbmoRqfQwjaCPm1H9w0hzQMLaKbWdpYp2zvfu9XPeVK ze7TM76VbYxNZVDL4zNPV6E1OPAzyU5skQpYk/LdlQ4FkZLy4WjHZTqcBQ0f6RIcpinL jF9WxIPfxQM9gCSafjMUKb2yVSW3U0duXUkskUmAEISMD+CS9awrUoYatmt3TV3cBUsV w49yTQCcV2I2GVkyP7DqhhQ8p1t5HwZzeb08dXDpAINVKLmbma9p8nhd/7wHmCVqX056 2eHorBfXfjLQizbyd6xbcE7ogeLzQRCmtR2Tmnwn7U0OhDLlolLHsTDjON9Y7/w9151/ DFXQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=aC64jdn5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sa31-20020a1709076d1f00b006f405295484si375321ejc.947.2022.04.29.03.45.51; Fri, 29 Apr 2022 03:45:52 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=aC64jdn5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 06F9A68B2D5; Fri, 29 Apr 2022 13:45:32 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B5B5368B267 for ; Fri, 29 Apr 2022 13:45:23 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1651229128; x=1682765128; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=j5J4pwC3Zw42mRsmtyI6LzLhC4Fhi8bZoqCJXGG5Vxk=; b=aC64jdn5Fone+ehltLUib2FfNcXhO8elN9HJ+xLQ50XLYo+Jr6CrvS1W bCUkrXIbGw+H6VQrrhoxZp7Ktoz/xsflpXbLCwqXGqd/dxvtkMDYjut1I xbim5UEyC0ZFgutrAf1FxP4Hm3GgQSy5VPdFUYNJM6JgJPckpRScnbfj6 rNt8UnJlD8DXuoXXq7OsaWWnyLslqUkuNWqbd9AQUupnakqX0yaJhsnZe 719XV5T5RCRo11VyDifajdhHDXXthvyuCUiVZVqRRCJ0umxSzdW4gnjp8 jY0Q8upIMA1bxJ7t6VqFCVJBQsbWZJJfuKPYnpysIRgUHI5+YBqnBKON4 w==; X-IronPort-AV: E=McAfee;i="6400,9594,10331"; a="329548328" X-IronPort-AV: E=Sophos;i="5.91,298,1647327600"; d="scan'208";a="329548328" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Apr 2022 03:45:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,298,1647327600"; d="scan'208";a="566080497" Received: from gtapc.iind.intel.com (HELO localhost.localdomain) ([10.190.235.42]) by fmsmga007.fm.intel.com with ESMTP; 29 Apr 2022 03:45:18 -0700 From: Tong Wu To: ffmpeg-devel@ffmpeg.org Date: Fri, 29 Apr 2022 10:45:03 +0000 Message-Id: <20220429104505.1747-3-tong1.wu@intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220429104505.1747-1-tong1.wu@intel.com> References: <20220429104505.1747-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 3/5] avutil/hwcontext_d3d11va: add a format check for staging texture X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 7dAuydJvoigl The texDesc.Format needs to be filled in with a corresponding format. I add a format check to initialize the format in case sometimes the ctx->internal->priv is not initialized, such as during the hwmap process. $ ffmpeg.exe -y -hwaccel qsv -init_hw_device d3d11va=d3d11 \ -init_hw_device qsv=qsv@d3d11 -c:v h264_qsv \ -i input.h264 -vf "hwmap=derive_device=d3d11va,format=d3d11,hwdownload,format=nv12" \ -f null - Signed-off-by: Tong Wu --- libavutil/hwcontext_d3d11va.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c index db529acbb4..0ec0e07d3a 100644 --- a/libavutil/hwcontext_d3d11va.c +++ b/libavutil/hwcontext_d3d11va.c @@ -349,6 +349,8 @@ static int d3d11va_create_staging_texture(AVHWFramesContext *ctx) AVD3D11VADeviceContext *device_hwctx = ctx->device_ctx->hwctx; D3D11VAFramesContext *s = ctx->internal->priv; HRESULT hr; + int i; + D3D11_TEXTURE2D_DESC texDesc = { .Width = ctx->width, .Height = ctx->height, @@ -360,6 +362,20 @@ static int d3d11va_create_staging_texture(AVHWFramesContext *ctx) .CPUAccessFlags = D3D11_CPU_ACCESS_READ | D3D11_CPU_ACCESS_WRITE, }; + if (!texDesc.Format) { + for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++) { + if (ctx->sw_format == supported_formats[i].pix_fmt) { + texDesc.Format = supported_formats[i].d3d_format; + break; + } + } + if (i == FF_ARRAY_ELEMS(supported_formats)) { + av_log(ctx, AV_LOG_ERROR, "Unsupported pixel format: %s\n", + av_get_pix_fmt_name(ctx->sw_format)); + return AVERROR(EINVAL); + } + } + hr = ID3D11Device_CreateTexture2D(device_hwctx->device, &texDesc, NULL, &s->staging_texture); if (FAILED(hr)) { av_log(ctx, AV_LOG_ERROR, "Could not create the staging texture (%lx)\n", (long)hr);