From patchwork Wed Oct 16 01:57:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 15781 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 18A09449D91 for ; Wed, 16 Oct 2019 04:57:32 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EC4D668A403; Wed, 16 Oct 2019 04:57:31 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7795668A3AF for ; Wed, 16 Oct 2019 04:57:25 +0300 (EEST) Received: by mail-wm1-f67.google.com with SMTP id f22so938367wmc.2 for ; Tue, 15 Oct 2019 18:57:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gQCFGAxRxa6zdT18Je8C6WCfszsku4Yc131HfXVhQNk=; b=AOyvrZJQcT/+9tAZAY3zCYTJpZDgFGzq4vBbmCYaSQXxit+EeZmPIxDBjjRZn8xDgy gcjOx3Y8gkhenMnrIokRQtm0+WtcOcFBYBpj9iWxAtc3myfLxWPG0ODGBWIGNEyY4KRN 8bro7gr9pCQL6uEoQ08yWsXFybPJxG72kYtudhu8FSebROJzj8Dpx9G9bOxm9q8heTfk Rgqz/wh+DDwlo2LHpmja+d0ooobq8mEDV1p5LivvqRzfBPnqrEJMkFf1U73k1m/GWQDu pBNoBQQzBu3h9OmJtD+50OgTPNh5T4JqVq6TBbaB9LZfMuzvQ9Zb0W7MFu0WaAOlR/7G 53Sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gQCFGAxRxa6zdT18Je8C6WCfszsku4Yc131HfXVhQNk=; b=KfieA3uccPGcMSjNHKAwsleMWadhinzi1Nq4PMxIbkpoOZJ8I849TKDkay+0bLk/7z XWDUnEYdOUvQc2CfGU4GE3OLypz6byN74YRLTaxdnru4ob9K1uRctf+2hJY5A8J53FOs ArbrXoSUZT/QJ1GMgEba95x7jju3kwg26PhuR7Ja6F7X3i8cL9F5BUXpbcWtAr3O14yQ 2vatHHvyXljRf/N19WRUr4QHs0RkNh1+wmR75RnVLsFLI+WmMscQc4QPNVaXM7FpKU6w h/bah9DYk40uJ3O/XV8oPOODz6D3pWZ+pdrQ2Dr+fGA864YLcdU7ZYbq/zAT0BBOJrlw K32A== X-Gm-Message-State: APjAAAXEsPafJEMjgX6/PozBbBrKk9x1RDM+QBx6VpFpKbrVRhEaVJbH yJhfuIh6jWEu55lp4P4LhCT7Gztq X-Google-Smtp-Source: APXvYqwR/uLNa92PFInY0L53N7B5uv0se82Vw3oCgcOQtm316hpl8wY22k6VCnfRJS+mjoTCShB2yA== X-Received: by 2002:a1c:4e09:: with SMTP id g9mr1184975wmh.96.1571191044752; Tue, 15 Oct 2019 18:57:24 -0700 (PDT) Received: from sblaptop.fritz.box (ipbcc08937.dynamic.kabel-deutschland.de. [188.192.137.55]) by smtp.gmail.com with ESMTPSA id r2sm36567133wrm.3.2019.10.15.18.57.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Oct 2019 18:57:24 -0700 (PDT) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Wed, 16 Oct 2019 03:57:08 +0200 Message-Id: <20191016015709.19220-1-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191015111737.14246-1-andreas.rheinhardt@gmail.com> References: <20191015111737.14246-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/2] avformat/matroskaenc: Fix memleak upon failure X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The Matroska muxer up until now leaked memory in two scenarios: 1. If an error happened during writing the trailer, as mkv_write_trailer() returned early without cleaning up. 2. If mkv_write_header() indicated success despite an error in the underlying AVIOContext. In this case avformat_write_header() returned the IO error and according to the API the caller is not allowed to call av_write_trailer(), so that no cleanup happened for the allocations made in mkv_write_header(). This has been fixed by using a dedicated deinit function. Signed-off-by: Andreas Rheinhardt --- I wouldn't be surprised if other muxers also leaked memory in the second scenario. libavformat/matroskaenc.c | 34 ++++++++++++++++------------------ 1 file changed, 16 insertions(+), 18 deletions(-) diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index 37706e56c7..3f9bd3445d 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -388,7 +388,9 @@ static void put_xiph_size(AVIOContext *pb, int size) /** * Free the members allocated in the mux context. */ -static void mkv_free(MatroskaMuxContext *mkv) { +static void mkv_deinit(AVFormatContext *s) +{ + MatroskaMuxContext *mkv = s->priv_data; uint8_t* buf; if (mkv->cluster_bc) { avio_close_dyn_buf(mkv->cluster_bc, &buf); @@ -1875,8 +1877,7 @@ static int mkv_write_header(AVFormatContext *s) mkv->tracks = av_mallocz_array(s->nb_streams, sizeof(*mkv->tracks)); if (!mkv->tracks) { - ret = AVERROR(ENOMEM); - goto fail; + return AVERROR(ENOMEM); } ebml_header = start_ebml_master(pb, EBML_ID_HEADER, MAX_EBML_HEADER_SIZE); put_ebml_uint (pb, EBML_ID_EBMLVERSION , 1); @@ -1896,12 +1897,12 @@ static int mkv_write_header(AVFormatContext *s) // of every other currently defined level 1 element mkv->seekhead = mkv_start_seekhead(pb, mkv->segment_offset, 10); if (!mkv->seekhead) { - ret = AVERROR(ENOMEM); - goto fail; + return AVERROR(ENOMEM); } ret = mkv_add_seekhead_entry(mkv->seekhead, MATROSKA_ID_INFO, avio_tell(pb)); - if (ret < 0) goto fail; + if (ret < 0) + return ret; ret = start_ebml_master_crc32(pb, &mkv->info_bc, mkv, MATROSKA_ID_INFO); if (ret < 0) @@ -1971,38 +1972,36 @@ static int mkv_write_header(AVFormatContext *s) mkv->stream_durations = av_mallocz(s->nb_streams * sizeof(int64_t)); mkv->stream_duration_offsets = av_mallocz(s->nb_streams * sizeof(int64_t)); if (!mkv->stream_durations || !mkv->stream_duration_offsets) { - ret = AVERROR(ENOMEM); - goto fail; + return AVERROR(ENOMEM); } ret = mkv_write_tracks(s); if (ret < 0) - goto fail; + return ret; for (i = 0; i < s->nb_chapters; i++) mkv->chapter_id_offset = FFMAX(mkv->chapter_id_offset, 1LL - s->chapters[i]->id); ret = mkv_write_chapters(s); if (ret < 0) - goto fail; + return ret; if (mkv->mode != MODE_WEBM) { ret = mkv_write_attachments(s); if (ret < 0) - goto fail; + return ret; } ret = mkv_write_tags(s); if (ret < 0) - goto fail; + return ret; if (!(s->pb->seekable & AVIO_SEEKABLE_NORMAL) && !mkv->is_live) mkv_write_seekhead(pb, mkv); mkv->cues = mkv_start_cues(mkv->segment_offset); if (!mkv->cues) { - ret = AVERROR(ENOMEM); - goto fail; + return AVERROR(ENOMEM); } if (s->metadata_header_padding > 0) { @@ -2039,9 +2038,6 @@ static int mkv_write_header(AVFormatContext *s) } return 0; -fail: - mkv_free(mkv); - return ret; } static int mkv_blockgroup_size(int pkt_size) @@ -2664,7 +2660,6 @@ static int mkv_write_trailer(AVFormatContext *s) end_ebml_master(pb, mkv->segment); } - mkv_free(mkv); return 0; } @@ -2811,6 +2806,7 @@ AVOutputFormat ff_matroska_muxer = { .video_codec = CONFIG_LIBX264_ENCODER ? AV_CODEC_ID_H264 : AV_CODEC_ID_MPEG4, .init = mkv_init, + .deinit = mkv_deinit, .write_header = mkv_write_header, .write_packet = mkv_write_flush_packet, .write_trailer = mkv_write_trailer, @@ -2845,6 +2841,7 @@ AVOutputFormat ff_webm_muxer = { .video_codec = CONFIG_LIBVPX_VP9_ENCODER? AV_CODEC_ID_VP9 : AV_CODEC_ID_VP8, .subtitle_codec = AV_CODEC_ID_WEBVTT, .init = mkv_init, + .deinit = mkv_deinit, .write_header = mkv_write_header, .write_packet = mkv_write_flush_packet, .write_trailer = mkv_write_trailer, @@ -2873,6 +2870,7 @@ AVOutputFormat ff_matroska_audio_muxer = { AV_CODEC_ID_VORBIS : AV_CODEC_ID_AC3, .video_codec = AV_CODEC_ID_NONE, .init = mkv_init, + .deinit = mkv_deinit, .write_header = mkv_write_header, .write_packet = mkv_write_flush_packet, .write_trailer = mkv_write_trailer,