[FFmpeg-devel] avcodec/ac3enc_float: fix fdsp memory leak

Submitted by Paul B Mahol on Oct. 17, 2019, 9:20 a.m.

Details

Message ID 20191017092001.4566-1-onemda@gmail.com
State New
Headers show

Commit Message

Paul B Mahol Oct. 17, 2019, 9:20 a.m.
Fixes #8294

Signed-off-by: Paul B Mahol <onemda@gmail.com>
---
 libavcodec/ac3enc_float.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

Comments

James Almer Oct. 17, 2019, 1:27 p.m.
On 10/17/2019 6:20 AM, Paul B Mahol wrote:
> Fixes #8294
> 
> Signed-off-by: Paul B Mahol <onemda@gmail.com>
> ---
>  libavcodec/ac3enc_float.c | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/ac3enc_float.c b/libavcodec/ac3enc_float.c
> index d6e658b2b4..18df313c1a 100644
> --- a/libavcodec/ac3enc_float.c
> +++ b/libavcodec/ac3enc_float.c
> @@ -132,10 +132,16 @@ av_cold int ff_ac3_float_mdct_init(AC3EncodeContext *s)
>  av_cold int ff_ac3_float_encode_init(AVCodecContext *avctx)
>  {
>      AC3EncodeContext *s = avctx->priv_data;
> +    int ret = ff_ac3_encode_init(avctx);
> +
> +    if (ret < 0)
> +        return ret;
>      s->fdsp = avpriv_float_dsp_alloc(avctx->flags & AV_CODEC_FLAG_BITEXACT);
> -    if (!s->fdsp)
> +    if (!s->fdsp) {
> +        ff_ac3_encode_close(avctx);
>          return AVERROR(ENOMEM);
> -    return ff_ac3_encode_init(avctx);
> +    }
> +    return 0;

Nit: Do it the other way around instead. Call avpriv_float_dsp_alloc()
first, then ff_ac3_encode_init(). That way you'll only call an
av_free(s->fdsp) on failure rather than ff_ac3_encode_close().

LGTM either way.

>  }
>  
>  AVCodec ff_ac3_encoder = {
>

Patch hide | download patch | download mbox

diff --git a/libavcodec/ac3enc_float.c b/libavcodec/ac3enc_float.c
index d6e658b2b4..18df313c1a 100644
--- a/libavcodec/ac3enc_float.c
+++ b/libavcodec/ac3enc_float.c
@@ -132,10 +132,16 @@  av_cold int ff_ac3_float_mdct_init(AC3EncodeContext *s)
 av_cold int ff_ac3_float_encode_init(AVCodecContext *avctx)
 {
     AC3EncodeContext *s = avctx->priv_data;
+    int ret = ff_ac3_encode_init(avctx);
+
+    if (ret < 0)
+        return ret;
     s->fdsp = avpriv_float_dsp_alloc(avctx->flags & AV_CODEC_FLAG_BITEXACT);
-    if (!s->fdsp)
+    if (!s->fdsp) {
+        ff_ac3_encode_close(avctx);
         return AVERROR(ENOMEM);
-    return ff_ac3_encode_init(avctx);
+    }
+    return 0;
 }
 
 AVCodec ff_ac3_encoder = {