From patchwork Tue Mar 10 04:55:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andriy Gelman X-Patchwork-Id: 18102 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 86E5644A89A for ; Tue, 10 Mar 2020 07:20:05 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 619D768A127; Tue, 10 Mar 2020 07:20:05 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f196.google.com (mail-qk1-f196.google.com [209.85.222.196]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D192E68A0F1 for ; Tue, 10 Mar 2020 07:19:58 +0200 (EET) Received: by mail-qk1-f196.google.com with SMTP id f198so11577956qke.11 for ; Mon, 09 Mar 2020 22:19:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lYXOTBhRz5fZPoYBFGof30wbLQyjHrRwt549NO4XdKo=; b=fWwPs3QgLvzVFRuCwUcnr1gXvoN2acomXAXogecHTfdUO7erCl9hP5iRZMghxzylqi j9gMc/bzenAwYImCjnimKFhoPmn8TteUeal2aox91Ik15WPSQioCIOIX9nA2BydlxhQ3 /XbWsl0QQCLyTy4aoDSaS5pE/1lghHQutKArgdW3rYwDm94v0OtcltmccPvRxCWJm5si i12X3cfmbeYmZBdjdRnYqZIC+hwjxHC9m2MJhpKSVkW7BlFMS1Of/o5UUX9zPZk7QaTP PVEaGbSnBZmF8EPd7suNZjujtXqdm016QLRm6BzpTi45QITyV9UTwAKXkxfUWIhYb4L3 KcdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lYXOTBhRz5fZPoYBFGof30wbLQyjHrRwt549NO4XdKo=; b=FXvdgckakf4qYKREAonLenwlESRSLulXoTJSX3SBp4hgOwwX6xhIutqnOrFIQaqGvb uthS+ayUbTNj3KI/NzJfAE0thh78xB4Ab84m1l2o3RCb/EbMCLYwFHSYUbaMGZh+zFpe F55Q0pE76BJohzCh51TaUPoCDqqj9R++BHTZ4zvM24lKwqfVw2LAiQYC0IOilScNJw9L v0UE7DYlydMHaFiNk5iDHexuTyMvnlGnpxMCpcShUBap1Ue+4bVfJHMf0Qg/OUeDeauq qPfnkhOdbbaVg6P13Oj2SKm9HEdpCm5T5JB5BxP83e7iKYTWSbru7AVbgizhDVQYTX7S UDoA== X-Gm-Message-State: ANhLgQ2IpQxpMOZxXcRJuezGNB99xVRRNhzvNB0S3CRb+G/PoS+mQaoT omFPUETpUlowRovIcVffsuNj6REF X-Google-Smtp-Source: ADFU+vvlJ0e20FpZL70uEUgeiMR0HAQXnoWYqsaekm/R1lTf6i71oqrLWrtj9ck0Wh198A26J46W6Q== X-Received: by 2002:ac8:4d0f:: with SMTP id w15mr3736611qtv.248.1583816170676; Mon, 09 Mar 2020 21:56:10 -0700 (PDT) Received: from localhost.localdomain (c-71-232-27-28.hsd1.ma.comcast.net. [71.232.27.28]) by smtp.gmail.com with ESMTPSA id j1sm20500752qtd.66.2020.03.09.21.56.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Mar 2020 21:56:10 -0700 (PDT) From: Andriy Gelman X-Google-Original-From: Andriy Gelman To: ffmpeg-devel@ffmpeg.org Date: Tue, 10 Mar 2020 00:55:41 -0400 Message-Id: <20200310045541.13048-2-andriy.gelman@gmail.com> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200310045541.13048-1-andriy.gelman@gmail.com> References: <20200310045541.13048-1-andriy.gelman@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] avcodec/v4l2_m2m{enc, dec}: Fix leaks when init fails X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andriy Gelman Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Andriy Gelman v4l2_m2m_{enc,dec} doesn't call its close function when initialization fails because FF_CODEC_CAP_INIT_CLEANUP is not set. This causes a couple possible leaks, which are fixed in the commit. Signed-off-by: Andriy Gelman --- libavcodec/v4l2_m2m_dec.c | 9 ++++++--- libavcodec/v4l2_m2m_enc.c | 7 ++++++- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/libavcodec/v4l2_m2m_dec.c b/libavcodec/v4l2_m2m_dec.c index d666edffe46..a515f3ca720 100644 --- a/libavcodec/v4l2_m2m_dec.c +++ b/libavcodec/v4l2_m2m_dec.c @@ -205,13 +205,16 @@ static av_cold int v4l2_decode_init(AVCodecContext *avctx) ret = ff_v4l2_m2m_codec_init(priv); if (ret) { av_log(avctx, AV_LOG_ERROR, "can't configure decoder\n"); - s->self_ref = NULL; - av_buffer_unref(&priv->context_ref); + ff_v4l2_m2m_codec_end(priv); return ret; } - return v4l2_prepare_decoder(s); + ret = v4l2_prepare_decoder(s); + if (ret < 0) + ff_v4l2_m2m_codec_end(priv); + + return ret; } static av_cold int v4l2_decode_close(AVCodecContext *avctx) diff --git a/libavcodec/v4l2_m2m_enc.c b/libavcodec/v4l2_m2m_enc.c index ff9ff267ea2..ca58669a976 100644 --- a/libavcodec/v4l2_m2m_enc.c +++ b/libavcodec/v4l2_m2m_enc.c @@ -329,10 +329,15 @@ static av_cold int v4l2_encode_init(AVCodecContext *avctx) if (pix_fmt_output != avctx->pix_fmt) { const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(pix_fmt_output); av_log(avctx, AV_LOG_ERROR, "Encoder requires %s pixel format.\n", desc->name); + ff_v4l2_m2m_codec_end(priv); return AVERROR(EINVAL); } - return v4l2_prepare_encoder(s); + ret = v4l2_prepare_encoder(s); + if (ret < 0) + ff_v4l2_m2m_codec_end(priv); + + return ret; } static av_cold int v4l2_encode_close(AVCodecContext *avctx)