From patchwork Thu Mar 18 20:49:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 26456 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id AF22044A4BD for ; Thu, 18 Mar 2021 22:51:09 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 527BB68A6D6; Thu, 18 Mar 2021 22:51:09 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vie01a-dmta-pe03-2.mx.upcmail.net (vie01a-dmta-pe03-2.mx.upcmail.net [62.179.121.161]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4B60D68816E for ; Thu, 18 Mar 2021 22:51:00 +0200 (EET) Received: from [172.31.216.235] (helo=vie01a-pemc-psmtp-pe12.mail.upcmail.net) by vie01a-dmta-pe03.mx.upcmail.net with esmtp (Exim 4.92) (envelope-from ) id 1lMzbg-000A1H-0D for ffmpeg-devel@ffmpeg.org; Thu, 18 Mar 2021 21:51:00 +0100 Received: from localhost ([213.47.68.29]) by vie01a-pemc-psmtp-pe12.mail.upcmail.net with ESMTP id Mzahl8HtXljeHMzailwE9M; Thu, 18 Mar 2021 21:50:00 +0100 X-Env-Mailfrom: michael@niedermayer.cc X-Env-Rcptto: ffmpeg-devel@ffmpeg.org X-SourceIP: 213.47.68.29 X-CNFS-Analysis: v=2.3 cv=BoHjPrf5 c=1 sm=1 tr=0 a=2hcxjKEKjp0CzLx6oWAm4g==:117 a=2hcxjKEKjp0CzLx6oWAm4g==:17 a=MKtGQD3n3ToA:10 a=1oJP67jkp3AA:10 a=GEAsPZ9sns4A:10 a=ZZnuYtJkoWoA:10 a=DG8H0SE9ueWXgl-XrhUA:9 a=pHzHmUro8NiASowvMSCR:22 a=xoEH_sTeL_Rfw54TyV31:22 From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Thu, 18 Mar 2021 21:49:58 +0100 Message-Id: <20210318204958.21176-4-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210318204958.21176-1-michael@niedermayer.cc> References: <20210318204958.21176-1-michael@niedermayer.cc> X-CMAE-Envelope: MS4wfK/mMMFYDJiZjdxyTtby+b5pz2eaJwp4LA689Ay4cdxXYu4bAPAQiZumJPWQMzJV1yNpCFTvUH/xvbJjuYADC9Zd2vt/x7E/53CaSlp1pcVCo6A1ZjkY o2PQ+0J87nSBYvNziWpoLREznCiuqp1uPp0SlKoZ5GtMgbWIOhQ+1op+ Subject: [FFmpeg-devel] [PATCH 4/4] avcodec/mpegpicture: Keep ff_mpeg_framesize_alloc() failure state consistent X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes: null pointer dereference Fixes: ff_put_pixels16_sse2.mp4 Found-by: Rafael Dutra Signed-off-by: Michael Niedermayer --- libavcodec/mpegpicture.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/libavcodec/mpegpicture.c b/libavcodec/mpegpicture.c index e3f648895d..0652b7c879 100644 --- a/libavcodec/mpegpicture.c +++ b/libavcodec/mpegpicture.c @@ -79,8 +79,11 @@ int ff_mpeg_framesize_alloc(AVCodecContext *avctx, MotionEstContext *me, // linesize * interlaced * MBsize // we also use this buffer for encoding in encode_mb_internal() needig an additional 32 lines if (!FF_ALLOCZ_TYPED_ARRAY(sc->edge_emu_buffer, alloc_size * EMU_EDGE_HEIGHT) || - !FF_ALLOCZ_TYPED_ARRAY(me->scratchpad, alloc_size * 4 * 16 * 2)) + !FF_ALLOCZ_TYPED_ARRAY(me->scratchpad, alloc_size * 4 * 16 * 2)) { + av_freep(&sc->edge_emu_buffer); return AVERROR(ENOMEM); + } + me->temp = me->scratchpad; sc->rd_scratchpad = me->scratchpad; sc->b_scratchpad = me->scratchpad;