From patchwork Mon Oct 9 00:04:53 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ivan Kalvachev X-Patchwork-Id: 5487 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.90 with SMTP id m26csp2069649jah; Sun, 8 Oct 2017 17:05:05 -0700 (PDT) X-Received: by 10.223.176.156 with SMTP id i28mr7371689wra.45.1507507505878; Sun, 08 Oct 2017 17:05:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1507507505; cv=none; d=google.com; s=arc-20160816; b=LlLhjfSBMSsKDrqBdeC7xqbPev4Q46tdDkc0qqqtvrHxTYNHyQnEUZontTxLXKMlDH tsI8ZcYa1cjHb4jj8oFvjWwtH4s0dThRem7c17FzGVKbHUQY/80dAuhM/2T6JQhfc0Ns ee1WhYhaoJRpkltikPyepdoDkcE8neyKgklYQobq2v7RFbWGxa5Q/QxYbCxgiSdah9TB Gvcb6y8FxSlc/ZTemIszouhb/S+bmRLapuiIUKB5SfcOqEzzD53oPTGRGnfKn4j3xneD JT7lV21LNqpTVNnvoW4ixinPFc6bqHPXYCEHMBRu9iLe/+J96aZY7URQL3rDB7A63jBT AqIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject:to :message-id:date:from:mime-version:dkim-signature:delivered-to :arc-authentication-results; bh=A1+35aChLuyS/azjlqLB8I0eS9S7j23+VU8YZeNrj4g=; b=0KkAtj33TuDXNsOm0FOp4fTtoZsy1NiIAunI+lG35a71gTFKd2fGlWXFZwFKLhDbH9 IXb/S8SSSk93vzyoCd/gbsv1ds8WCYsYO3Bu7lulRacd5T/wTR4F98AomW0ja3BNUx7V o2rYFkTmgJcLhDXIDLIYpLXKAvtK3d9v0sB4/6PCJvrKqaf7tS0M2VAH4Dh5fY835sEO YD9WsQEakQz0vsn7b/QJ4LZ6HqIuSJNmQW7QnkNlVn96D7INB7pyIoSnoB0MY0lPShvH QUX6maxGyx/nYeE84+cCkTszc36ds9U1kDLwu7/CsF2oQLBgWqWJEhoft4+DskHrtN8n /KNg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=DoZ69N6u; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 60si3561954wrn.67.2017.10.08.17.05.05; Sun, 08 Oct 2017 17:05:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=DoZ69N6u; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6BB84680A69; Mon, 9 Oct 2017 03:05:01 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf0-f175.google.com (mail-pf0-f175.google.com [209.85.192.175]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9A7C5680385 for ; Mon, 9 Oct 2017 03:04:54 +0300 (EEST) Received: by mail-pf0-f175.google.com with SMTP id p87so3057055pfj.3 for ; Sun, 08 Oct 2017 17:04:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=bfeGKt9tLwzk8H3SsEU5VA0U9OKcTWg38dV2APf4WCQ=; b=DoZ69N6u4X4jSh6Ckv9OjDZrxhPSGljguvASJvhUS2aafThjdK/DaxctKRU61n+4es hn4JtSBMXgQ8h77uBJ6XqzOJ848ss+KK0NkGxnidgE/RTTS+scpLMGP5gncgHW+sFs86 j+tcO4L5zPeirGbFMpBZxtejjj0lvMsarI2/XuYsDdaFVbJC7qrDRHi/Ech8sPJPGPnX tr8vkdPUSJYRuyeP2DF0EeF+2iVDkInLmZq6cmw2A+kMJLJdvKBzRWM9zRfIRjNG3zHW Tpv2rCQL/WFRXcNUr/5Dtqbbg016XiSHZjKex39WY+6akMKViFU0H5bCFfb4xl0RBuER nTGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=bfeGKt9tLwzk8H3SsEU5VA0U9OKcTWg38dV2APf4WCQ=; b=HE3XuQRigJR0Qx/KG+3YtSlYfWk/5wYUSmoenO1+EICHgkllXIGi93np4XPiJs7Oq8 35ZkvPBeNlktkEbiFq5B72Zc+u+p1WiUwsPD6SU/8lnE6hNw/K6QRk3SFrFniANtADoK 075Wfpge6LOrq/KbF2OLHwUcdcW9EqjahxTPwHS/hly6jQbv0e5NQQWerkxzJNhavJf9 o7kcyYtqZPYLv2RMKwpvp6pcEVos30eiWb1BTz3YNApndcpKbxei9pzTUe7+af5slea/ mxfI4JzYp2Ay0e3TYXMvpGUXLruNWFKGEttJZjtMYmjOv+gfjsDF1KWRph7fvhGb+1Mc 1J/g== X-Gm-Message-State: AMCzsaU0M1qlorrsA8zAE/vwp5jAqtNJwol4cyHdO+kyr/euEXsqVy1h 4GtPuCOnpTgbVtOn0HdF6TGKUBMeOD579SmNIYU= X-Google-Smtp-Source: AOwi7QCIat3y9k5CVAY/c7JdGbFQRzoN3JuOKMyMJY5uuKaxLYFyOFYyT/3F54wIG4tFU8btw2JlLlDGstoJo9oyc24= X-Received: by 10.98.245.66 with SMTP id n63mr5133810pfh.102.1507507494625; Sun, 08 Oct 2017 17:04:54 -0700 (PDT) MIME-Version: 1.0 Received: by 10.100.151.236 with HTTP; Sun, 8 Oct 2017 17:04:53 -0700 (PDT) From: Ivan Kalvachev Date: Mon, 9 Oct 2017 03:04:53 +0300 Message-ID: To: FFmpeg development discussions and patches Subject: [FFmpeg-devel] [PATCH] Fix crash if av_vdpau_bind_context() is not used. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The public functions av_alloc_vdpaucontext() and av_vdpau_alloc_context() are allocating AVVDPAUContext structure that is supposed to be placed in avctx->hwaccel_context. However the rest of libavcodec/vdpau.c uses avctx->hwaccel_context as struct VDPAUHWContext, that is bigger and does contain AVVDPAUContext as first member. The usage includes write to the new variables in the bigger stuct, without checking for block size. Fix by always allocating the bigger structure. BTW, I have no idea why the new fields haven't simply been added to the existing struct... It seems that the programmer who wrote this has been aware of the problem, because av_vdpau_bind_context reallocates the structure. It might be good idea to check the other usages of this reallocation function. Best Regards Ivan Kalvachev From c9dafbf5402ebf8c68bf8648ecea7a74282113a8 Mon Sep 17 00:00:00 2001 From: Ivan Kalvachev Date: Mon, 9 Oct 2017 02:40:26 +0300 Subject: [PATCH] Fix crash if av_vdpau_bind_context() is not used. The public functions av_alloc_vdpaucontext() and av_vdpau_alloc_context() are allocating AVVDPAUContext structure that is supposed to be placed in avctx->hwaccel_context. However the rest of libavcodec/vdpau.c uses avctx->hwaccel_context as struct VDPAUHWContext, that is bigger and does contain AVVDPAUContext as first member. The usage includes write to the new variables in the bigger stuct, without checking for block size. Fix by always allocating the bigger structure. Signed-off-by: Ivan Kalvachev --- libavcodec/vdpau.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/vdpau.c b/libavcodec/vdpau.c index 42ebddbee..4cc51cb79 100644 --- a/libavcodec/vdpau.c +++ b/libavcodec/vdpau.c @@ -816,7 +816,7 @@ do { \ AVVDPAUContext *av_vdpau_alloc_context(void) { - return av_mallocz(sizeof(AVVDPAUContext)); + return av_mallocz(sizeof(VDPAUHWContext)); } int av_vdpau_bind_context(AVCodecContext *avctx, VdpDevice device, -- 2.14.1