From patchwork Wed Jul 10 02:03:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Zern X-Patchwork-Id: 13884 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id E17D4449633 for ; Wed, 10 Jul 2019 05:11:01 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B11CC68AA25; Wed, 10 Jul 2019 05:11:01 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f201.google.com (mail-qt1-f201.google.com [209.85.160.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E1CF468A864 for ; Wed, 10 Jul 2019 05:10:54 +0300 (EEST) Received: by mail-qt1-f201.google.com with SMTP id l9so862245qtu.12 for ; Tue, 09 Jul 2019 19:10:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to; bh=bFt0vCh4EsvxFfuGGPW8jM+mCEm1FFe/fNFuHbqzmOs=; b=ZWZrKa2MnisWR0BtE5IJ/MKNC0frZ+HS5FXlak/+byhglY7JrCm1H5Yvq6BraSuFgO lm2XMja7oRg6KEy4KnakHYteXqCiiYCW9+H91RMEfZgPyF7L1q7uFrOcZBPCDy+L9RfV VP2oK78IGI68dtUayr+3EQ6ydtJuGQC1umf06S6kjYqqrfYY6YRYR9HwWrc0A0Z+Un7a qd4E465siRwprJBTo7lhuY3h9qHOhstQzDsXh+YQVNEqKNVlO0HcpKKDecTDrl3zYTcp gwyJ2pFztZduVoexIoLoBOnOenbExVZQUE8i2bR42IsW/TKalw3ZsO0SVXGCz0lfxlWs uX/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to; bh=bFt0vCh4EsvxFfuGGPW8jM+mCEm1FFe/fNFuHbqzmOs=; b=dChs12xh5zgByr5JoVx/xxr/5B57U9pVhcd0gF2AsZZVPwpSp2xToWGhpI31LdJkgy qAYDFBUZK+3hAUaAel9LHTDd1XxWIq/DjcxAtob1unH2+K1+sFypPvaoM5xgODXkQejp nt+i2f4fhzIiOuj6QRMBwiaLfiN9IddgSFNBuPZjzESqHGMAvxcjY4Tqm2GtDlp7sh7Y 8VD7rWlRMtiC/4VIRdh4atxmzow0Y9PnJcIHpHoOvBIVqUDJXEfB6vGlAUTcvRrCMN3n 7a6q+UDZuD7jEnL5QjabcK8VVkWzEUE7XbvUJstAkGxIjUWDBeZvZyFfFTRY/fndhBs9 ItHQ== X-Gm-Message-State: APjAAAWgD+SXHFmfh92Bbibon9+bl2n9LatfgdGHXxDIfvYlnYp+Rg/C ff8UzzJYqFuNj5wOOhpi4AZOsavUBBFkKIw+5YmOpi1nJzpp/f+GCpZurjcyDyc+ABcGG2YNMJ/ jPOy+78PSH+DB5iRaajC3hK9HVQ+d6Qj6VZPGolanQGUbBUHAzLtCWcNmHKgJ X-Google-Smtp-Source: APXvYqxDZoSyF+lTicaHngRMWpp0rCL3Ltsddbn1UBcJWebjkgVEPIDt2Cw53oiqbtznjqEpMeTYMIq3/w== X-Received: by 2002:aed:2fe1:: with SMTP id m88mr21104217qtd.77.1562724240785; Tue, 09 Jul 2019 19:04:00 -0700 (PDT) Date: Tue, 9 Jul 2019 19:03:58 -0700 Message-Id: <20190710020358.34517-1-jzern@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog From: James Zern To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH] avcodec/utils, avcodec_open2: close codec on failure X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" after a successful init if the function fails for another reason close the codec without requiring FF_CODEC_CAP_INIT_CLEANUP which is meant to cover init failures themselves. fixes a memory leak in those cases. BUG=oss-fuzz:15529 Signed-off-by: James Zern --- libavcodec/utils.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/libavcodec/utils.c b/libavcodec/utils.c index 1b1feca38c..9ff9628f7f 100644 --- a/libavcodec/utils.c +++ b/libavcodec/utils.c @@ -542,6 +542,7 @@ int attribute_align_arg ff_codec_open2_recursive(AVCodecContext *avctx, const AV int attribute_align_arg avcodec_open2(AVCodecContext *avctx, const AVCodec *codec, AVDictionary **options) { int ret = 0; + int codec_init_ok = 0; AVDictionary *tmp = NULL; const AVPixFmtDescriptor *pixdesc; @@ -935,6 +936,7 @@ FF_ENABLE_DEPRECATION_WARNINGS if (ret < 0) { goto free_and_end; } + codec_init_ok = 1; } ret=0; @@ -1023,7 +1025,8 @@ end: return ret; free_and_end: if (avctx->codec && - (avctx->codec->caps_internal & FF_CODEC_CAP_INIT_CLEANUP)) + (codec_init_ok || + (avctx->codec->caps_internal & FF_CODEC_CAP_INIT_CLEANUP))) avctx->codec->close(avctx); if (codec->priv_class && codec->priv_data_size)