From patchwork Wed Jul 27 15:21:41 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yuli Khodorkovskiy X-Patchwork-Id: 13 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.140.67 with SMTP id o64csp311583vsd; Wed, 27 Jul 2016 08:39:18 -0700 (PDT) X-Received: by 10.28.60.136 with SMTP id j130mr34892417wma.93.1469633957962; Wed, 27 Jul 2016 08:39:17 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id fs16si7687354wjc.230.2016.07.27.08.39.17; Wed, 27 Jul 2016 08:39:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7DE8468A5C1; Wed, 27 Jul 2016 18:39:01 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt0-f194.google.com (mail-qt0-f194.google.com [209.85.216.194]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5402F68A5BB for ; Wed, 27 Jul 2016 18:38:47 +0300 (EEST) Received: by mail-qt0-f194.google.com with SMTP id q11so1913129qtb.2 for ; Wed, 27 Jul 2016 08:38:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:message-id; bh=CFkZl5S5DppQkLvAys1Us7po8zKxqOH+cl76JY5tUN4=; b=ddHoa2jwV8b13Y8lEf9cJrpf0vRkF95EdmI7ImplKH3OOeUvr3c8Vi5aSqw2iZqLTs KENgvihYLan5Lv2Xevcau34vaVvO+Bq5I3MtJ+1YbKOozO6hBeNUEr1hFZw6uS+R9J0Q oKgm4q1kDwPzrDSulKAxwRN7tTt3wvZBIGuC+gSIQ49fMHt5GEzJyL43E4qKUMbnzSTo M228eYC/13q4OmDrPHPsSxOPy1KbZRO/J/VYpgj8t1B2c9jVUvLQuOrIwbkgF/UxseY7 pYw+9FOIsuXhwTCvbL77cZTYjd78Fm+AUyW04B64nTt1pJXUI/GPnsGaXHCplsUFA6ft mi5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id; bh=CFkZl5S5DppQkLvAys1Us7po8zKxqOH+cl76JY5tUN4=; b=bvJmXYDQZduybupT093LLojFynELckzCiYMDzRSSbgPKta97r7ogBCQ6A3x56wcd4A FuSjQKmAefVabDjKe8jkQiAwUT+c71YpOQHSfjly09WxgXJXgceXgl6uYWDxm1rhfDwJ u9gwdI/MWdDma9WRuld+ZaEP3xl6vshyVOnkmlC2+UJ71hoEiXuyADadqsU/Y1QfAx5H WXuTKIH/otC8NLz8d9XhJLoXG+mFsbekGWEjURrIUlTTManypnItsRuGINrSVxrm5oAT lpyTicqh2XwhGzIk5rH5z17OUsPbMfDSbTAZRzarBc2SpVnUL6R2iISsCXM26SLZcOjm 1bUA== X-Gm-Message-State: AEkoouuROWHX3RUV3p1ucBthxskbtewAJMQNxaEDIqg6HDSB31nI2g6ozaqEmAahCQitNw== X-Received: by 10.200.55.137 with SMTP id d9mr48810945qtc.46.1469632913764; Wed, 27 Jul 2016 08:21:53 -0700 (PDT) Received: from localhost.localdomain.com (static-96-244-17-66.bltmmd.fios.verizon.net. [96.244.17.66]) by smtp.gmail.com with ESMTPSA id y8sm4441214qkb.21.2016.07.27.08.21.53 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jul 2016 08:21:53 -0700 (PDT) From: Yuli Khodorkovskiy To: ffmpeg-devel@ffmpeg.org Date: Wed, 27 Jul 2016 11:21:41 -0400 Message-Id: <1469632901-16971-1-git-send-email-ykhodo@gmail.com> X-Mailer: git-send-email 1.8.3.1 Subject: [FFmpeg-devel] [PATCH] Fix double free and null dereferences in the qsv decoder X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This patch fixes the h264_qsv decoder issues mentioned in https://ffmpeg.zeranoe.com/forum/viewtopic.php?t=2962. The patch may be tested by specifying h264_qsv as the decoder to ffplay for an h264 encoded file. ffplay -vcodec h264_qsv foo.mts Signed-off-by: Yuli Khodorkovskiy --- libavcodec/qsvdec.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 9125700..b462887 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -408,7 +408,7 @@ static int do_qsv_decode(AVCodecContext *avctx, QSVContext *q, return ff_qsv_error(ret); } n_out_frames = av_fifo_size(q->async_fifo) / (sizeof(out_frame)+sizeof(sync)); - + av_freep(&sync); if (n_out_frames > q->async_depth || (flush && n_out_frames) ) { AVFrame *src_frame; @@ -555,16 +555,18 @@ void ff_qsv_decode_reset(AVCodecContext *avctx, QSVContext *q) } /* Reset output surfaces */ - av_fifo_reset(q->async_fifo); + if (q->async_fifo) + av_fifo_reset(q->async_fifo); /* Reset input packets fifo */ - while (av_fifo_size(q->pkt_fifo)) { + while (q->pkt_fifo && av_fifo_size(q->pkt_fifo)) { av_fifo_generic_read(q->pkt_fifo, &pkt, sizeof(pkt), NULL); av_packet_unref(&pkt); } /* Reset input bitstream fifo */ - av_fifo_reset(q->input_fifo); + if (q->input_fifo) + av_fifo_reset(q->input_fifo); } int ff_qsv_decode_close(QSVContext *q)