From patchwork Thu Aug 30 09:37:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 10178 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:12c4:0:0:0:0:0 with SMTP id 65-v6csp641632jap; Thu, 30 Aug 2018 02:38:03 -0700 (PDT) X-Google-Smtp-Source: ANB0Vda8VqklFSyOqO6NRK4jzTl/f4gt4hp8WbcGb12r5js7xTm60gs8h5G+GFdWL+z5msJBn0/k X-Received: by 2002:a1c:7f93:: with SMTP id a141-v6mr1244049wmd.45.1535621882918; Thu, 30 Aug 2018 02:38:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535621882; cv=none; d=google.com; s=arc-20160816; b=TfvSYJRNdPGEcuZOg8l6fE2AFOPKtpcumSGT8TSAbDi3w1LzByf0BAu09AJvEOPI7g XbgwkoUqHeKatEqOZ6El5pk9OWquraxoKZHu0J3pNUEM40m8hjIt1alQb68PVNLTGS39 7no+KRWR7nuUDm3fIqhxB58S+P13adf5MSZKwDYhrReJvZRcKslQgF4mBiKbILMc/hl2 +Mlnt8mkGcX/qPTY74GY9QoRIE+GqohcpAM6kTAstzaK1ZwCRxN82azDkA4CEM/wTF9P V27dxMvQ2K5hNWMCoC8L4fy1ePQrsc3HeqlGzWe6QtJf5n0x53iw+BpnM/qqvom8LOp+ nMuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=P/inNovankt3C+xxYeB3SK/gDTbXtnkUQr/BvaCpxkE=; b=HqiyzZy+8oxoPwJtpfY9sJ0g6wyiTzFk50hwJKqiXOzBftzFQc8XK1BWTUMIO8I/aE ieTsTzlLMpxPIeDwb0eieVRNZpTrPQENQN/+JnUixOLifHaKcnCDrWhZboyxthRsahRB sJVh1NPk6K6I/9SuZPW8Ayui0DbWi4lKulV82yo+4O0vGeySaHxp3WVAh1cE77OQGYWQ AwtWBVJh39XFHDigyOQ7xlDRKzoEduK5hyByQjFlaKl3iyf+c1f81QGawxUrZHLYEXN7 FcDezeq+d4zx1Nku2GqgVEsgvdqRsSbwNUjz6EnC9x7E5VjZrSSq9m5tsZpLEMmDTdzs jfqg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=Hu4vIeRB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f17-v6si5547379wrj.147.2018.08.30.02.38.01; Thu, 30 Aug 2018 02:38:02 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=Hu4vIeRB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B41DF689EE8; Thu, 30 Aug 2018 12:37:55 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 78E95689C95 for ; Thu, 30 Aug 2018 12:37:49 +0300 (EEST) Received: by mail-wr1-f54.google.com with SMTP id n2-v6so7412032wrw.7 for ; Thu, 30 Aug 2018 02:37:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=qwoUmFiMLu/B/UdQyGvhUNeQIEnuDmgT6MkwMoYXSNg=; b=Hu4vIeRB+yUe8cMGGxoO5B4RvcS6cHuq9it7wXDRbZZ55MYUFQReWoDRwlwzp/Nm7i MzPDqYf2s0IwBcBz+qX/oxmuGm9gdWQK4jnsAyln1WvtafgAxWHkw18BTwl7eC5/X5HL DQY7PVjBlx3gy0NcFbHUwjt+I90ESl3wiLcPxiLTdyJFIOe/0GrbRGkSBmMRmjBI7xVj 4Xg51vUH3irnbXnIqFQvcTXOTWTrbVYE4lZy0p4J0NfCtLzpIgPvR+5F4Jm0Yi45doMx FaDME7VtvByPqbYIJyABFEyp7XJBR2pTw0/x7dM9BDtZ7O76Buuz2X1h93Mgw7H6u0+u OthA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=qwoUmFiMLu/B/UdQyGvhUNeQIEnuDmgT6MkwMoYXSNg=; b=FbefoOHybPAXhicPxULiWwKg5k4BCHtTI/PjsobnRgH/F0Y7VUcNCY+Yj0MDPmSe8c Y6ZUHzmP1ROBa2jhOTXkU1tpeH0IfqA2b/EDIpv2T3VOkpL2J4nIKIkTSr9fo4lHzS6Y HSdlX6stL5mOdImArh9jBIpdAuZweSbmN0bHep033TZfL6cJZkV/Jzo3GTgeJkJ7CBg6 K1PPNc8l6NwpjMW+d7Qj65fUYNaJyJllHOq4aCvj/QndyM/YY1DzhYWwGPm2SaVC/REI AGBGjP/Yw6XtXie88ClGwTbz5DdZZFztgjz//zmYQr0ct3+tSCx+ANPVBv38TiApnQzT byTw== X-Gm-Message-State: APzg51APreSPHLJfeZacUWOhVtFSM0poMSKRZMBfG4biAg+ab8X5nKiz bRLxFV6EVqF5hldLZD1Rf/hTK+jY X-Received: by 2002:adf:80ea:: with SMTP id 97-v6mr6794518wrl.57.1535621873332; Thu, 30 Aug 2018 02:37:53 -0700 (PDT) Received: from localhost.localdomain ([94.250.174.60]) by smtp.gmail.com with ESMTPSA id n11-v6sm5748479wra.26.2018.08.30.02.37.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 30 Aug 2018 02:37:52 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Thu, 30 Aug 2018 11:37:45 +0200 Message-Id: <20180830093745.18098-1-onemda@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] avcodec/dnxhdenc: stop leaking memory when initalization fails X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes #6593. Signed-off-by: Paul B Mahol --- libavcodec/dnxhdenc.c | 18 +++++++++--------- libavcodec/dnxhdenc.h | 2 ++ 2 files changed, 11 insertions(+), 9 deletions(-) diff --git a/libavcodec/dnxhdenc.c b/libavcodec/dnxhdenc.c index 01e22e7764..5135eedfd6 100644 --- a/libavcodec/dnxhdenc.c +++ b/libavcodec/dnxhdenc.c @@ -207,17 +207,17 @@ static av_cold int dnxhd_init_vlc(DNXHDEncContext *ctx) int i, j, level, run; int max_level = 1 << (ctx->bit_depth + 2); - FF_ALLOCZ_ARRAY_OR_GOTO(ctx->m.avctx, ctx->vlc_codes, - max_level, 4 * sizeof(*ctx->vlc_codes), fail); - FF_ALLOCZ_ARRAY_OR_GOTO(ctx->m.avctx, ctx->vlc_bits, - max_level, 4 * sizeof(*ctx->vlc_bits), fail); + FF_ALLOCZ_ARRAY_OR_GOTO(ctx->m.avctx, ctx->orig_vlc_codes, + max_level, 4 * sizeof(*ctx->orig_vlc_codes), fail); + FF_ALLOCZ_ARRAY_OR_GOTO(ctx->m.avctx, ctx->orig_vlc_bits, + max_level, 4 * sizeof(*ctx->orig_vlc_bits), fail); FF_ALLOCZ_OR_GOTO(ctx->m.avctx, ctx->run_codes, 63 * 2, fail); FF_ALLOCZ_OR_GOTO(ctx->m.avctx, ctx->run_bits, 63, fail); - ctx->vlc_codes += max_level * 2; - ctx->vlc_bits += max_level * 2; + ctx->vlc_codes = ctx->orig_vlc_codes + max_level * 2; + ctx->vlc_bits = ctx->orig_vlc_bits + max_level * 2; for (level = -max_level; level < max_level; level++) { for (run = 0; run < 2; run++) { int index = (level << 1) | run; @@ -1348,11 +1348,10 @@ FF_ENABLE_DEPRECATION_WARNINGS static av_cold int dnxhd_encode_end(AVCodecContext *avctx) { DNXHDEncContext *ctx = avctx->priv_data; - int max_level = 1 << (ctx->bit_depth + 2); int i; - av_free(ctx->vlc_codes - max_level * 2); - av_free(ctx->vlc_bits - max_level * 2); + av_freep(&ctx->orig_vlc_codes); + av_freep(&ctx->orig_vlc_bits); av_freep(&ctx->run_codes); av_freep(&ctx->run_bits); @@ -1392,6 +1391,7 @@ AVCodec ff_dnxhd_encoder = { .encode2 = dnxhd_encode_picture, .close = dnxhd_encode_end, .capabilities = AV_CODEC_CAP_SLICE_THREADS | AV_CODEC_CAP_FRAME_THREADS | AV_CODEC_CAP_INTRA_ONLY, + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV422P10, diff --git a/libavcodec/dnxhdenc.h b/libavcodec/dnxhdenc.h index 963821ac81..7b0d862e28 100644 --- a/libavcodec/dnxhdenc.h +++ b/libavcodec/dnxhdenc.h @@ -86,6 +86,8 @@ typedef struct DNXHDEncContext { unsigned frame_bits; uint8_t *src[3]; + uint32_t *orig_vlc_codes; + uint8_t *orig_vlc_bits; uint32_t *vlc_codes; uint8_t *vlc_bits; uint16_t *run_codes;