diff mbox series

[FFmpeg-devel,08/21] avfilter/vf_paletteuse: Fix leaks of AVFilterFormats on error

Message ID 20200809155748.30092-2-andreas.rheinhardt@gmail.com
State Accepted
Commit 6a65449954d466e76c1166f524d2f6cde28c3c96
Headers show
Series [FFmpeg-devel,1/6] avfilter/formats: Remove ff_make_formatu64_list() | expand

Checks

Context Check Description
andriy/default pending
andriy/make success Make finished
andriy/make_fate success Make fate finished

Commit Message

Andreas Rheinhardt Aug. 9, 2020, 3:57 p.m. UTC
The paletteuse's query_formats function allocated three AVFilterFormats
before storing them permanently. If allocating one of them failed, the
three AVFilterFormats structures would be freed with av_freep() which
does not free separately allocated subelements (namely the formats
array) which leak.

Furthermore, if storing one of the first two fails, the function simply
returns and the ones not yet stored leak.

These leaks have been fixed by only creating a new AVFilterFormats after
the last one has already been permanently stored. Furthermore, it is
enough to check whether the elements have been properly stored as
ff_formats_ref() by design returns AVERROR(ENOMEM) if it is provided a
NULL AVFilterFormats *.

Fixes Coverity issues #1270818 and #1270819.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
---
 libavfilter/vf_paletteuse.c | 18 ++++++------------
 1 file changed, 6 insertions(+), 12 deletions(-)
diff mbox series

Patch

diff --git a/libavfilter/vf_paletteuse.c b/libavfilter/vf_paletteuse.c
index b32ff817d0..80e2ba583a 100644
--- a/libavfilter/vf_paletteuse.c
+++ b/libavfilter/vf_paletteuse.c
@@ -142,18 +142,12 @@  static int query_formats(AVFilterContext *ctx)
     static const enum AVPixelFormat inpal_fmts[] = {AV_PIX_FMT_RGB32, AV_PIX_FMT_NONE};
     static const enum AVPixelFormat out_fmts[]   = {AV_PIX_FMT_PAL8,  AV_PIX_FMT_NONE};
     int ret;
-    AVFilterFormats *in    = ff_make_format_list(in_fmts);
-    AVFilterFormats *inpal = ff_make_format_list(inpal_fmts);
-    AVFilterFormats *out   = ff_make_format_list(out_fmts);
-    if (!in || !inpal || !out) {
-        av_freep(&in);
-        av_freep(&inpal);
-        av_freep(&out);
-        return AVERROR(ENOMEM);
-    }
-    if ((ret = ff_formats_ref(in   , &ctx->inputs[0]->out_formats)) < 0 ||
-        (ret = ff_formats_ref(inpal, &ctx->inputs[1]->out_formats)) < 0 ||
-        (ret = ff_formats_ref(out  , &ctx->outputs[0]->in_formats)) < 0)
+    if ((ret = ff_formats_ref(ff_make_format_list(in_fmts),
+                              &ctx->inputs[0]->out_formats)) < 0 ||
+        (ret = ff_formats_ref(ff_make_format_list(inpal_fmts),
+                              &ctx->inputs[1]->out_formats)) < 0 ||
+        (ret = ff_formats_ref(ff_make_format_list(out_fmts),
+                              &ctx->outputs[0]->in_formats)) < 0)
         return ret;
     return 0;
 }