From patchwork Tue Jun 6 14:06:38 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Jones X-Patchwork-Id: 3858 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.10.2 with SMTP id 2csp1872637vsk; Tue, 6 Jun 2017 07:15:15 -0700 (PDT) X-Received: by 10.223.167.15 with SMTP id c15mr5602316wrd.79.1496758515909; Tue, 06 Jun 2017 07:15:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1496758515; cv=none; d=google.com; s=arc-20160816; b=Sbisvc2+H6+x7lu8GOpxD+Ogp9y+1Ahu3m3haq/v8t1jtXa5MwZsMM4xUTazSIUwcZ DxUcYTOwgJERT9BX83yqNlsuqrCwev+uOw85XmVC4+OsoojKrTzVsbLaV3eIKI9DWpdp pRtUkywFxbL2s0APcO7gd+k5ECV7miL5JHTfgjl2zQi6QJLgitrsQP8Mwoqtm7jPhFfA ods7OeGa8IP26m9jtjL/ucxfneIq/r9RefEfjbgupQO8B0la/vuC19HEjjY+cuckA29N SybokJeMZMYjmd9xUPjRIuOlnhqCnQePs1kqS3kI/X8mlHx/N7e5axRxd0c+wRY/stWh oYjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject:user-agent :mime-version:message-id:to:from:date:dkim-signature:delivered-to :arc-authentication-results; bh=H0SDBBh9LVcu98DMknoX+YkEbLmclaRHjQRuG1ukeAc=; b=O6AFJknXGDmK4tb8ua/JfSX7MckU1bpK2ZjVtPVm4Oe8zMvMN6C8Ko6KMUQ+aZf9vB FJ4FjvJMviE8MioqW9r12PkBad/INcq4HUwYEoaq6E236HKp74PHdbppCaUzjwxPrvNk u629SALC67ts31tuvmOhieooN3j4v+x4Ns6qiqBwKS5hrSOoPXm49XCmB/MqSr1LsRU0 HrZm7tAkaDOOwhMycz/j3UBOF+m9wrjBh0DSz0oJ3zeXOq8+rjO3EcfSks/Jk6wPU6cE s2UpYENNOzmlnx0941wX+E4ftB0lQNTcW33xhJuQp8y9IxN0YeQw3UF9+M0PDVr1gVs0 BCWQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c74si17290061wmc.161.2017.06.06.07.15.15; Tue, 06 Jun 2017 07:15:15 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0DE98689E4B; Tue, 6 Jun 2017 17:15:06 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-it0-f65.google.com (mail-it0-f65.google.com [209.85.214.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 25BCC689E06 for ; Tue, 6 Jun 2017 17:14:59 +0300 (EEST) Received: by mail-it0-f65.google.com with SMTP id i206so27542993ita.3 for ; Tue, 06 Jun 2017 07:15:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:subject:message-id:mime-version:content-disposition :user-agent; bh=4TRtuU8EGNrhgOKSRgtrYBut0naFNwpM+ohn4owEWH4=; b=R0LJ6rrn2cFrvGBwT91AW2F1vmJWw6HTzywqx6v7k1WLUd4gN3SX/ggZBtFYHicgz/ /jZKTzFMBJYMTcmYfxvStjVKDIsxTvyohTffqDrLw8m/RhkTGIarVpnAigJlW6DQnjU5 Fwe1UTD25tFyMblobUPHgDTnEp5Wd0c+fBIHfesuQcmN1w3oLyA80J7biSPQZ8Hwu8XK Ded7eZhhalC23pHnPC5KJN63poY5zD39LFRUvAe2YII4CQ2oZiKWk6KhnMAfEQu5tKKZ WQuqpBT1Ty8xDbcf2hIYOC78W7VZ5JRxnu1oW/CW3s0IwN7mRKv+R+U2pAZJ6N9Vs4EA cIBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:mime-version :content-disposition:user-agent; bh=4TRtuU8EGNrhgOKSRgtrYBut0naFNwpM+ohn4owEWH4=; b=j3PzhgE0u28Bzsf7Ol+BHbs2W9oE5UZ+OFFAj6ekE4YDK2fNdZJzGgJjnvfHEBZp1e 3M9xH6vFy78ujV5PIVAfzc2Mx4ViOIGAWZe9vg2IBS89FLOcql4J1WUOR9uky9bjD5tn fQ+rEQ3ffPmxOEP8IrS1iEBY4jaUbrqyajke8RFImZ2NdnDit5wj+gZJA5ttHI4sz1ED ilSEpbhs40azt7BbnFlVqqdddO3ZhDGDdpVBhY41wolnqqMM5Wu5trCOWA3/JSyfPPHP jzLhvLMxi3twBiFi2oP2WQ3RVBDnwB1t8IPJh5tmOlK/5rdppL4bk3SJ24Vha5Qk9wrq BF2Q== X-Gm-Message-State: AODbwcDbf+Ma01f12fe87QUHqKSYa/4ifrnLU4sKW2QOKgaBiO1kfTVJ /s5L8aIbmEWn6JNB X-Received: by 10.107.62.139 with SMTP id l133mr26294039ioa.213.1496758002318; Tue, 06 Jun 2017 07:06:42 -0700 (PDT) Received: from tdjones879 (host-184-167-177-46.csp-wy.client.bresnan.net. [184.167.177.46]) by smtp.gmail.com with ESMTPSA id h76sm6408939ith.24.2017.06.06.07.06.40 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Jun 2017 07:06:41 -0700 (PDT) Date: Tue, 6 Jun 2017 08:06:38 -0600 From: Tyler Jones To: ffmpeg-devel@ffmpeg.org Message-ID: <20170606140638.GA3488@tdjones879> MIME-Version: 1.0 User-Agent: Mutt/1.5.24 (2015-08-30) Subject: [FFmpeg-devel] [PATCH] avcodec/vorbisenc: Fix memory leak on errors X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Switches temporary samples for processing to be stored in the encoder's context, avoids memory leaks if any errors occur while encoding a frame. Fixes CID1412026 Signed-off-by: Tyler Jones --- libavcodec/vorbisenc.c | 49 ++++++++++++------------------------------------- 1 file changed, 12 insertions(+), 37 deletions(-) diff --git a/libavcodec/vorbisenc.c b/libavcodec/vorbisenc.c index 856f590..afded40 100644 --- a/libavcodec/vorbisenc.c +++ b/libavcodec/vorbisenc.c @@ -112,6 +112,7 @@ typedef struct vorbis_enc_context { float *samples; float *floor; // also used for tmp values for mdct float *coeffs; // also used for residue after floor + float *scratch; // used for tmp values for psy model float quality; AudioFrameQueue afq; @@ -452,7 +453,9 @@ static int create_vorbis_context(vorbis_enc_context *venc, venc->samples = av_malloc_array(sizeof(float) * venc->channels, (1 << venc->log2_blocksize[1])); venc->floor = av_malloc_array(sizeof(float) * venc->channels, (1 << venc->log2_blocksize[1]) / 2); venc->coeffs = av_malloc_array(sizeof(float) * venc->channels, (1 << venc->log2_blocksize[1]) / 2); - if (!venc->saved || !venc->samples || !venc->floor || !venc->coeffs) + venc->scratch = av_malloc_array(sizeof(float) * venc->channels, (1 << venc->log2_blocksize[1]) / 2); + + if (!venc->saved || !venc->samples || !venc->floor || !venc->coeffs || !venc->scratch) return AVERROR(ENOMEM); if ((ret = dsp_init(avctx, venc)) < 0) @@ -992,7 +995,7 @@ static int residue_encode(vorbis_enc_context *venc, vorbis_enc_residue *rc, } static int apply_window_and_mdct(vorbis_enc_context *venc, - float **audio, int samples) + float *audio, int samples) { int channel; const float * win = venc->win[0]; @@ -1017,7 +1020,7 @@ static int apply_window_and_mdct(vorbis_enc_context *venc, for (channel = 0; channel < venc->channels; channel++) { float *offset = venc->samples + channel * window_len * 2 + window_len; - fdsp->vector_fmul_reverse(offset, audio[channel], win, samples); + fdsp->vector_fmul_reverse(offset, audio + channel * window_len, win, samples); fdsp->vector_fmul_scalar(offset, offset, 1/n, samples); } } else { @@ -1034,7 +1037,7 @@ static int apply_window_and_mdct(vorbis_enc_context *venc, for (channel = 0; channel < venc->channels; channel++) { float *offset = venc->saved + channel * window_len; - fdsp->vector_fmul(offset, audio[channel], win, samples); + fdsp->vector_fmul(offset, audio + channel * window_len, win, samples); fdsp->vector_fmul_scalar(offset, offset, 1/n, samples); } venc->have_saved = 1; @@ -1068,28 +1071,8 @@ static AVFrame *spawn_empty_frame(AVCodecContext *avctx, int channels) return f; } -static float **alloc_audio_arrays(int channels, int frame_size) -{ - float **audio = av_mallocz_array(channels, sizeof(float *)); - if (!audio) - return NULL; - - for (int ch = 0; ch < channels; ch++) { - audio[ch] = av_mallocz_array(frame_size, sizeof(float)); - if (!audio[ch]) { - // alloc has failed, free everything allocated thus far - for (ch--; ch >= 0; ch--) - av_free(audio[ch]); - av_free(audio); - return NULL; - } - } - - return audio; -} - /* Concatenate audio frames into an appropriately sized array of samples */ -static void move_audio(vorbis_enc_context *venc, float **audio, int *samples, int sf_size) +static void move_audio(vorbis_enc_context *venc, float *audio, int *samples, int sf_size) { AVFrame *cur = NULL; int frame_size = 1 << (venc->log2_blocksize[1] - 1); @@ -1102,7 +1085,7 @@ static void move_audio(vorbis_enc_context *venc, float **audio, int *samples, in for (int ch = 0; ch < venc->channels; ch++) { const float *input = (float *) cur->extended_data[ch]; const size_t len = cur->nb_samples * sizeof(float); - memcpy(&audio[ch][sf*sf_size], input, len); + memcpy(audio + ch*frame_size + sf*sf_size, input, len); } av_frame_free(&cur); } @@ -1112,7 +1095,6 @@ static int vorbis_encode_frame(AVCodecContext *avctx, AVPacket *avpkt, const AVFrame *frame, int *got_packet_ptr) { vorbis_enc_context *venc = avctx->priv_data; - float **audio = NULL; int i, ret, need_more; int samples = 0, frame_size = 1 << (venc->log2_blocksize[1] - 1); vorbis_enc_mode *mode; @@ -1132,10 +1114,6 @@ static int vorbis_encode_frame(AVCodecContext *avctx, AVPacket *avpkt, if (need_more) return 0; - audio = alloc_audio_arrays(venc->channels, frame_size); - if (!audio) - return AVERROR(ENOMEM); - /* Pad the bufqueue with empty frames for encoding the last packet. */ if (!frame) { if (venc->bufqueue.available * avctx->frame_size < frame_size) { @@ -1151,9 +1129,9 @@ static int vorbis_encode_frame(AVCodecContext *avctx, AVPacket *avpkt, } } - move_audio(venc, audio, &samples, avctx->frame_size); + move_audio(venc, venc->scratch, &samples, avctx->frame_size); - if (!apply_window_and_mdct(venc, audio, samples)) + if (!apply_window_and_mdct(venc, venc->scratch, samples)) return 0; if ((ret = ff_alloc_packet2(avctx, avpkt, 8192, 0)) < 0) @@ -1213,10 +1191,6 @@ static int vorbis_encode_frame(AVCodecContext *avctx, AVPacket *avpkt, flush_put_bits(&pb); avpkt->size = put_bits_count(&pb) >> 3; - for (int ch = 0; ch < venc->channels; ch++) - av_free(audio[ch]); - av_free(audio); - ff_af_queue_remove(&venc->afq, frame_size, &avpkt->pts, &avpkt->duration); if (frame_size > avpkt->duration) { @@ -1281,6 +1255,7 @@ static av_cold int vorbis_encode_close(AVCodecContext *avctx) av_freep(&venc->samples); av_freep(&venc->floor); av_freep(&venc->coeffs); + av_freep(&venc->scratch); av_freep(&venc->fdsp); ff_mdct_end(&venc->mdct[0]);