From patchwork Tue May 16 11:09:28 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aaron Levinson X-Patchwork-Id: 3676 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.10.2 with SMTP id 2csp404632vsk; Tue, 16 May 2017 04:09:42 -0700 (PDT) X-Received: by 10.28.26.82 with SMTP id a79mr7510845wma.119.1494932982095; Tue, 16 May 2017 04:09:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1494932982; cv=none; d=google.com; s=arc-20160816; b=NTwhkwNFKaR7+F1bwjrBFDYk8fVoM/2p5uNPpR1SN+zrzrf7PFBpCAElp41yRl6Nff f7fyg1almkVaNtcOhl91iOfEfmVbjGzDsjc1BMLZwg40K8+GqqC3k8Eqpc4ozweineDf 7ZSVBFrclJwWYxbuWLSzu2h1QrU4f4tjhviHWnqX86ju1lIX4MplFs8vRwkn7GEif8Xe obBv3UofSoimacJJOEDp/c1CDU6EDI/RyIDuKiV5J6UvYjyBdjJ9pFvDSg6GNMQHQ+td ozC3oREZLzvNVwhxoG9DxXulSrA9IXqPvbr/Z63zMAvFHLHXrfxgR96qK43/hKtqVRCC cGnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:user-agent:date:message-id:from:to :delivered-to:arc-authentication-results; bh=DlzH0Nlq8EcDYRzAxK4KymEwswLR8D5zx4VB7zYLnH8=; b=Q2TPvLLBZWc2L3FTtgCnKYvGmHiRqpx1j0ZpuD2D5r1FDU9rGe0M9ceX1V67FTctaz P9mAv18qe/hswIlzwRa7nWumPGfba0jjnju0skgod9ZZRwSJ1kK5JnpSSQCBok3G3BWU 6APySdFQzuFKZcbD53m00cH/pDRYPsNtyJcS1y0wMG+uDyj2MvrilQxU0ZblgrH4Vwal 8GaaZPHniZOwrgqs/8+mNaVLxCBmdyH+BeEAN4WXWSMaUoCV4XVek1tE01CgQ+wTyl17 aDILGn6EzqTTShOJ0i3fcePrq7SHIBizwLmeuGa3K0MhkBQ8iZihfoAifINE4YkyecwX sfXA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r5si1972556wmf.103.2017.05.16.04.09.41; Tue, 16 May 2017 04:09:42 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 89EB06898CD; Tue, 16 May 2017 14:09:38 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from white.spiritone.com (white.spiritone.com [216.99.193.38]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5B187680949 for ; Tue, 16 May 2017 14:09:32 +0300 (EEST) Received: from [192.168.3.101] (184-100-204-251.ptld.qwest.net [184.100.204.251]) by white.spiritone.com (Postfix) with ESMTPSA id 1FF1C7340756 for ; Tue, 16 May 2017 04:09:30 -0700 (PDT) To: FFmpeg development discussions and patches From: Aaron Levinson Message-ID: Date: Tue, 16 May 2017 04:09:28 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avutil/hwcontext_dxva2: No longer improperly freeing IDirect3DSurface9 objects X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Purpose: No longer improperly freeing IDirect3DSurface9 objects in hwcontext_dxva2.c. Added dxva2_pool_release_dummy() and using it in call to av_buffer_create() in dxva2_pool_alloc(). Prior to this change, av_buffer_create() was called with NULL for the third argument, which indicates that av_buffer_default_free() should be used to free the buffer's data. Eventually, it gets to buffer_pool_free() and calls buf->free() on a surface object (which is av_buffer_default_free()). This can result in a crash when the debug version of the C-runtime is used on Windows. While it doesn't appear to result in a crash when the release version of the C-runtime is used on Windows, it likely results in memory corruption, since av_free() is being called on memory that was allocated using IDirectXVideoAccelerationService::CreateSurface(). Signed-off-by: Aaron Levinson --- libavutil/hwcontext_dxva2.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/libavutil/hwcontext_dxva2.c b/libavutil/hwcontext_dxva2.c index 4ed0d56..6c41788 100644 --- a/libavutil/hwcontext_dxva2.c +++ b/libavutil/hwcontext_dxva2.c @@ -121,6 +121,13 @@ static void dxva2_frames_uninit(AVHWFramesContext *ctx) } } +static void dxva2_pool_release_dummy(void *opaque, uint8_t *data) +{ + // important not to free anything here--data is a surface object + // associated with the call to CreateSurface(), and these surfaces are + // released in dxva2_frames_uninit() +} + static AVBufferRef *dxva2_pool_alloc(void *opaque, int size) { AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; @@ -130,7 +137,7 @@ static AVBufferRef *dxva2_pool_alloc(void *opaque, int size) if (s->nb_surfaces_used < hwctx->nb_surfaces) { s->nb_surfaces_used++; return av_buffer_create((uint8_t*)s->surfaces_internal[s->nb_surfaces_used - 1], - sizeof(*hwctx->surfaces), NULL, 0, 0); + sizeof(*hwctx->surfaces), dxva2_pool_release_dummy, 0, 0); } return NULL;