From patchwork Tue Mar 31 15:34:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fu, Linjie" X-Patchwork-Id: 18554 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 5CBBF44BDF2 for ; Tue, 31 Mar 2020 18:40:20 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 439CA68B042; Tue, 31 Mar 2020 18:40:20 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B14D5689ECB for ; Tue, 31 Mar 2020 18:40:13 +0300 (EEST) IronPort-SDR: /gi0nQHOnheLrgyjBY7NZHyc5i1usIrRr5Q67WTm1p6fYHpzjoGM0SV9EyHM9KGYimHP40jIwM lnttgSMyXU/A== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Mar 2020 08:40:12 -0700 IronPort-SDR: lP7kllPw6th82dBRYGIlu4XC10jpb2c52EPLkefAZ+RkSJR9jX2HWDZCFroZRI6/nkLZZugcLu Lz0fE3PoHmLA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,328,1580803200"; d="scan'208";a="241946689" Received: from icl-dev.sh.intel.com ([10.239.158.73]) by fmsmga008.fm.intel.com with ESMTP; 31 Mar 2020 08:40:11 -0700 From: Linjie Fu To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Mar 2020 23:34:00 +0800 Message-Id: <1585668840-30347-1-git-send-email-linjie.fu@intel.com> X-Mailer: git-send-email 2.7.4 Subject: [FFmpeg-devel] [PATCH] lavc/vaapi_encode: add FF_CODEC_CAP_INIT_CLEANUP caps for encoders X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Linjie Fu MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" ff_vaapi_encode_close() is not enough to free the resources like cbs if initialization failure happens after codec->configure (except for vp8/vp9). We need to call avctx->codec->close() to deallocate, otherwise memory leak happens. Add FF_CODEC_CAP_INIT_CLEANUP for vaapi encoders and deallocate the resources at free_and_end inside avcodec_open2(). Signed-off-by: Linjie Fu --- libavcodec/vaapi_encode.c | 1 - libavcodec/vaapi_encode_h264.c | 1 + libavcodec/vaapi_encode_h265.c | 1 + libavcodec/vaapi_encode_mjpeg.c | 1 + libavcodec/vaapi_encode_mpeg2.c | 1 + libavcodec/vaapi_encode_vp8.c | 1 + libavcodec/vaapi_encode_vp9.c | 1 + 7 files changed, 6 insertions(+), 1 deletion(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 8ff720e..d371033 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -2361,7 +2361,6 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) return 0; fail: - ff_vaapi_encode_close(avctx); return err; } diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index f4965d8..6a86905 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -1356,6 +1356,7 @@ AVCodec ff_h264_vaapi_encoder = { .close = &vaapi_encode_h264_close, .priv_class = &vaapi_encode_h264_class, .capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE, + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, .defaults = vaapi_encode_h264_defaults, .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_VAAPI, diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 97dc5a7..4c24f7c 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -1292,6 +1292,7 @@ AVCodec ff_hevc_vaapi_encoder = { .close = &vaapi_encode_h265_close, .priv_class = &vaapi_encode_h265_class, .capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE, + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, .defaults = vaapi_encode_h265_defaults, .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_VAAPI, diff --git a/libavcodec/vaapi_encode_mjpeg.c b/libavcodec/vaapi_encode_mjpeg.c index bd029cc..c469e70 100644 --- a/libavcodec/vaapi_encode_mjpeg.c +++ b/libavcodec/vaapi_encode_mjpeg.c @@ -565,6 +565,7 @@ AVCodec ff_mjpeg_vaapi_encoder = { .priv_class = &vaapi_encode_mjpeg_class, .capabilities = AV_CODEC_CAP_HARDWARE | AV_CODEC_CAP_INTRA_ONLY, + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, .defaults = vaapi_encode_mjpeg_defaults, .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_VAAPI, diff --git a/libavcodec/vaapi_encode_mpeg2.c b/libavcodec/vaapi_encode_mpeg2.c index bac9ea1..55f9289 100644 --- a/libavcodec/vaapi_encode_mpeg2.c +++ b/libavcodec/vaapi_encode_mpeg2.c @@ -702,6 +702,7 @@ AVCodec ff_mpeg2_vaapi_encoder = { .close = &vaapi_encode_mpeg2_close, .priv_class = &vaapi_encode_mpeg2_class, .capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE, + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, .defaults = vaapi_encode_mpeg2_defaults, .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_VAAPI, diff --git a/libavcodec/vaapi_encode_vp8.c b/libavcodec/vaapi_encode_vp8.c index 6e7bf9d..d8fb031 100644 --- a/libavcodec/vaapi_encode_vp8.c +++ b/libavcodec/vaapi_encode_vp8.c @@ -257,6 +257,7 @@ AVCodec ff_vp8_vaapi_encoder = { .close = &ff_vaapi_encode_close, .priv_class = &vaapi_encode_vp8_class, .capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE, + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, .defaults = vaapi_encode_vp8_defaults, .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_VAAPI, diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index d7f415d..ea19470 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -291,6 +291,7 @@ AVCodec ff_vp9_vaapi_encoder = { .close = &ff_vaapi_encode_close, .priv_class = &vaapi_encode_vp9_class, .capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE, + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, .defaults = vaapi_encode_vp9_defaults, .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_VAAPI,