From patchwork Tue Mar 27 00:22:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 8185 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp4144477jad; Mon, 26 Mar 2018 17:22:44 -0700 (PDT) X-Google-Smtp-Source: AG47ELsLf6+Npzng79XN2jzolsViOLEB4LCwMgWaECHtBkh6eT+ybY4YFw54cm0NNrBKB+1hl6Ns X-Received: by 10.28.194.8 with SMTP id s8mr12178319wmf.134.1522110164560; Mon, 26 Mar 2018 17:22:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1522110164; cv=none; d=google.com; s=arc-20160816; b=DbLGdOw6gc3oqU3ZXL9Ldzw1zQnm2n8JHXxUnX1aqt90YXCIOAdX2Z17VqPFfP1gvu Iq/2GnwNuitaTYQ9hVivQHEFzswotazQsU6rZXrwPVUPiGWGij3etSxiF3bBVMAHyNuu ZfnHzmbxNNnVyQjxmyQunB3iu1oikKBLJFBNgBWXUQ65pLRFOZSWMH3HPL5qnhz0wUz/ i0afA7fJ9sZ6uJ3xdzSig0i2gTIasUIWBqPY0V0juXzVdAdA2TvgzS2w7mrSUkgIHB/u 0LESR4sRm9Qf0ObE3M3KzvjcLod5B0+NgaOYhvU9WW55avp532feinXhf7j5i4hx0enG oWEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:content-language:mime-version:user-agent:date :message-id:from:to:dkim-signature:delivered-to :arc-authentication-results; bh=X/aITQBRRh67GVQb6Z3PRwzbf/RL9xIq3Lfxdbc545w=; b=NzfaBmCCb7fDHc1K9tc9CVSZU0CgZW3DQYn7u6tACn5mlB3+LamW7VbP1QdJznWKAD JjtgEbMy8F9XBDTwvKfptGPZO5LIaVHwnvTCtvwOfDuwO0qRupqXuTvVgYFCeWMg0k+s riSq2X0VjY9/eS28nOosEmmpYRH9906meh5I7d/ixn62aVLo1k8+tdKxJuwk16S6UkLg /9o8y0NPmzIDhAcdgwGoNhJ5A5Vlbn58dAWAnb/JCfeDmZyd+rbsJlio+i5hUqkIqZAy uj4gZP4yiD5orBPPjwvi8akpRi2yhkqyxUWv8ZBQUk3rqeH6cLRQyQtvOC4FnSDjhnoU ypGA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=nwb/2cYU; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h63si152621wmh.118.2018.03.26.17.22.44; Mon, 26 Mar 2018 17:22:44 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=nwb/2cYU; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 93EBB689A0C; Tue, 27 Mar 2018 03:22:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f66.google.com (mail-wm0-f66.google.com [74.125.82.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DF5DE689951 for ; Tue, 27 Mar 2018 03:22:18 +0300 (EEST) Received: by mail-wm0-f66.google.com with SMTP id r131so11684136wmb.2 for ; Mon, 26 Mar 2018 17:22:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=to:from:subject:message-id:date:user-agent:mime-version :content-language:content-transfer-encoding; bh=bH+d+G/cFem9qZrOUTY1n7yiXOAPwa0mv9zCh478EdU=; b=nwb/2cYUmB9ffiP1QsHE8PwyHdOMpsWEyAwuWtFHIpiySGS3GHftzFvCHH01onk/oI iUw70bgR6qJg/Puphy8Mo4pJkSpIQGwv5Fuese1+BjBNUwwgtqNcB6rU/0UsigG9iGeI 95VNX6RI9N/1ro7SJyG/NjRQ3qY5XF54bx8KlzRhRrQNYKk1nuqKso4K+byAf6+9oTj+ d2aBT+ug68+jnVgcNzBCpEwluadVjYKFziikB/XcY+EGUHDz+VHiT0/4EGTKReLaqOM/ Iz5npbzIh3rClJwOYlow3800uvTAIhmUXNmr/2WL/HCK35azLd99siHORX2wGSibPgZo 9/+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-language:content-transfer-encoding; bh=bH+d+G/cFem9qZrOUTY1n7yiXOAPwa0mv9zCh478EdU=; b=BhLWwV4gZ370rmyUfe5tRm3MsRIeQPCVmyY4B1EhIZHatzqthg+4ijV9trrUySW8C2 /XOsiU6LL9bK4APND6lZD58P2O+NBjKjFgMat21fYor1hkMEwbwQ1ADOkNchA4Ldn3Ho mTSUQtHdsBofyxBKS2WQ9oO0LSnpkfSBFYbKDCwboO6zNxs6z83LBVT/sBY2iiefEJ+9 ftT/K12M+K5lgEXXsAYScGCXZV8XUPsjW2MN9hydNGYF0dhqApV+dk697ihnI7StyGz9 27e5pCGrHHNEFBjrHcfRHZ8XqV/MDOQesl8dB/18D/JnrHXM9S2JBhrVGgQH74BiGE5B 3VdA== X-Gm-Message-State: AElRT7GtdINFYyBMgvSZZ+uwk4J0oO9hOK18tCDWzP7EGwU4UluyiD9M ScCEXHpgY0ijHLKvT88AdEHMRsuK X-Received: by 10.28.164.68 with SMTP id n65mr736213wme.123.1522110155703; Mon, 26 Mar 2018 17:22:35 -0700 (PDT) Received: from [192.168.0.3] (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id k35sm6591171wre.55.2018.03.26.17.22.34 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Mar 2018 17:22:35 -0700 (PDT) To: FFmpeg development discussions and patches From: Mark Thompson Message-ID: Date: Tue, 27 Mar 2018 01:22:34 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 Content-Language: en-US Subject: [FFmpeg-devel] [PATCH] hwcontext_d3d11: Fix crash with valid adapter but no device X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This crash was introduced by 8bbf2dacbfb4ead1535dea411035994f507f517d, which could incorrectly overwrite the failure result from creating the device. Fixes ticket #7108. --- libavutil/hwcontext_d3d11va.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c index 960883c9d8..d39fdd3fc8 100644 --- a/libavutil/hwcontext_d3d11va.c +++ b/libavutil/hwcontext_d3d11va.c @@ -556,8 +556,6 @@ static int d3d11va_device_create(AVHWDeviceContext *ctx, const char *device, } } - hr = mD3D11CreateDevice(pAdapter, pAdapter ? D3D_DRIVER_TYPE_UNKNOWN : D3D_DRIVER_TYPE_HARDWARE, NULL, creationFlags, NULL, 0, - D3D11_SDK_VERSION, &device_hwctx->device, NULL, NULL); if (pAdapter) { DXGI_ADAPTER_DESC2 desc; hr = IDXGIAdapter2_GetDesc(pAdapter, &desc); @@ -565,8 +563,12 @@ static int d3d11va_device_create(AVHWDeviceContext *ctx, const char *device, av_log(ctx, AV_LOG_INFO, "Using device %04x:%04x (%ls).\n", desc.VendorId, desc.DeviceId, desc.Description); } - IDXGIAdapter_Release(pAdapter); } + + hr = mD3D11CreateDevice(pAdapter, pAdapter ? D3D_DRIVER_TYPE_UNKNOWN : D3D_DRIVER_TYPE_HARDWARE, NULL, creationFlags, NULL, 0, + D3D11_SDK_VERSION, &device_hwctx->device, NULL, NULL); + if (pAdapter) + IDXGIAdapter_Release(pAdapter); if (FAILED(hr)) { av_log(ctx, AV_LOG_ERROR, "Failed to create Direct3D device (%lx)\n", (long)hr); return AVERROR_UNKNOWN;