From patchwork Tue Sep 15 07:39:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 22398 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id E744544B636 for ; Tue, 15 Sep 2020 10:40:52 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CDAD868BC6E; Tue, 15 Sep 2020 10:40:52 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f66.google.com (mail-wm1-f66.google.com [209.85.128.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D988768BC24 for ; Tue, 15 Sep 2020 10:40:43 +0300 (EEST) Received: by mail-wm1-f66.google.com with SMTP id e17so2326593wme.0 for ; Tue, 15 Sep 2020 00:40:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=v5EDvB0XDKC+VgRsErdOTl+I6+wSfy1gaabsJ8aGTyo=; b=rEKjSeERlNSmEctWdUjp50IJ5qfHAcT4QjalSfyIFSQoevwNs0E3na4u9ORyTWc/K9 eHy0hxeiUKCAI9F23ltSh4bcBwA74E0sAKcxzCvuZGTkdsqNxZV40eysJucBMLzfIt9Z t/BY9+CohjscGp0MkJ7FcIXGCB5l88MoW6avxZ/Xkdo/pyEiJP4d/ZGj2oNdnsZknfb+ aikqyiuaJiogg3K8yFcyTH6/I8B07xwXmSJ22sstpxBOSlIEld/kdU95V7eZwPAaN5yl bE/EduX5i1Of3mkhaVFoh7cX4qCWGi4oxOvYXLc7COI69Sbc7KWodQnoSbDPavNBQn7J EIew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=v5EDvB0XDKC+VgRsErdOTl+I6+wSfy1gaabsJ8aGTyo=; b=qaCgCzGKnEJQ2cmUzMTkRAvaMRi8/LjqWV0mn+tf/u3x2MjAC5NPVsBphwPdXzEo1r QKrp8i/aVdR+gkNsTfuA/k3YBgyGyZCKnHI+jSdLhV+4bi4JNnqY4VJxrrz55NHzODBy duCGYcb7wm81Jrng0WoxJZyVi+qim1ZQNAwcFOGTXBztT8iy6Z/PuEKA1eXa8PsSHCbH SvF/5ITsO4h0df3IDi3IwQBiohcRJO2FtvXqRo2dh2/xM55y5ZVOAYd8mmLwOh2iSXi5 VidTFdbO1WrZdf3gOBi4BA3dfWDZbbdMk5QHU8Vr/XZzvrltEjqim9aDxJ+KWUoC6Il8 psYw== X-Gm-Message-State: AOAM533qjxs8aBAz2JiCN0hSgn8tlUfrNqxepSat8yLA2GVZBUChz8fi yDF8M8zLnLIDPWhRifafGIBuRsbkhv8= X-Google-Smtp-Source: ABdhPJwJXYjLk9EuYld5HSTulYn5/MWlc4ElAtuVFp3IbOKMnBuk9iCm0yukJu1EY0aqQI2aVHLj+Q== X-Received: by 2002:a1c:6484:: with SMTP id y126mr3190063wmb.177.1600155643138; Tue, 15 Sep 2020 00:40:43 -0700 (PDT) Received: from sblaptop.fritz.box (ipbcc1fb0f.dynamic.kabel-deutschland.de. [188.193.251.15]) by smtp.gmail.com with ESMTPSA id d23sm6112125wmb.6.2020.09.15.00.40.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Sep 2020 00:40:42 -0700 (PDT) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Tue, 15 Sep 2020 09:39:47 +0200 Message-Id: <20200915074000.102622-17-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200915074000.102622-1-andreas.rheinhardt@gmail.com> References: <20200915074000.102622-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 17/30] avcodec/j2kenc: Fix leaks on init failure X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The JPEG2000 encoder did not clean up after itself on error. This commit fixes this by modifying the cleanup function to be able to handle only partially allocated structures and by setting the FF_CODEC_CAP_INIT_CLEANUP flag. Signed-off-by: Andreas Rheinhardt --- libavcodec/j2kenc.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/libavcodec/j2kenc.c b/libavcodec/j2kenc.c index 4cefe6d7fb..e3c5a32188 100644 --- a/libavcodec/j2kenc.c +++ b/libavcodec/j2kenc.c @@ -459,7 +459,7 @@ static int init_tiles(Jpeg2000EncoderContext *s) s->numXtiles = ff_jpeg2000_ceildiv(s->width, s->tile_width); s->numYtiles = ff_jpeg2000_ceildiv(s->height, s->tile_height); - s->tile = av_malloc_array(s->numXtiles, s->numYtiles * sizeof(Jpeg2000Tile)); + s->tile = av_calloc(s->numXtiles, s->numYtiles * sizeof(Jpeg2000Tile)); if (!s->tile) return AVERROR(ENOMEM); for (tileno = 0, tiley = 0; tiley < s->numYtiles; tiley++) @@ -1495,12 +1495,16 @@ static void cleanup(Jpeg2000EncoderContext *s) int tileno, compno; Jpeg2000CodingStyle *codsty = &s->codsty; + if (!s->tile) + return; for (tileno = 0; tileno < s->numXtiles * s->numYtiles; tileno++){ - for (compno = 0; compno < s->ncomponents; compno++){ - Jpeg2000Component *comp = s->tile[tileno].comp + compno; - ff_jpeg2000_cleanup(comp, codsty); + if (s->tile[tileno].comp) { + for (compno = 0; compno < s->ncomponents; compno++){ + Jpeg2000Component *comp = s->tile[tileno].comp + compno; + ff_jpeg2000_cleanup(comp, codsty); + } + av_freep(&s->tile[tileno].comp); } - av_freep(&s->tile[tileno].comp); av_freep(&s->tile[tileno].layer_rates); } av_freep(&s->tile); @@ -1853,4 +1857,5 @@ AVCodec ff_jpeg2000_encoder = { AV_PIX_FMT_NONE }, .priv_class = &j2k_class, + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, };