[FFmpeg-devel,2/5] avcodec/magicyuv: Free previous VLC table

Fixes: memleak
Fixes: 26788/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MAGICYUV_fuzzer-5184116808744960

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/magicyuv.c | 1 +
 1 file changed, 1 insertion(+)

Michael Niedermayer:
> Fixes: memleak
> Fixes: 26788/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MAGICYUV_fuzzer-5184116808744960
> 
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/magicyuv.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/libavcodec/magicyuv.c b/libavcodec/magicyuv.c
> index f13351e5b5..be9d2ac561 100644
> --- a/libavcodec/magicyuv.c
> +++ b/libavcodec/magicyuv.c
> @@ -411,6 +411,7 @@ static int build_huffman(AVCodecContext *avctx, const uint8_t *table,
>  
>          if (j == max) {
>              j = 0;
> +            ff_free_vlc(&s->vlc[i]);
>              if (huff_build(he, length_count, &s->vlc[i], max)) {
>                  av_log(avctx, AV_LOG_ERROR, "Cannot build Huffman codes\n");
>                  return AVERROR_INVALIDDATA;
> 
Seems I inexplicably deleted this this in
1bf30a1beb7527887c9b466d07765d20fa3b5412; I did not notice it because
all my tests were apparently done on files with only very few frames, so
that nothing was overwritten; also all fate samples for magicyuv contain
only one frame. Good that this has been caught; but could you add the
ff_free_vlc() directly in front of ff_init_vlc_sparse() (where it used
to be)?

- Andreas

On Mon, Nov 02, 2020 at 01:41:13AM +0100, Andreas Rheinhardt wrote:
> Michael Niedermayer:
> > Fixes: memleak
> > Fixes: 26788/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MAGICYUV_fuzzer-5184116808744960
> > 
> > Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> > ---
> >  libavcodec/magicyuv.c | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/libavcodec/magicyuv.c b/libavcodec/magicyuv.c
> > index f13351e5b5..be9d2ac561 100644
> > --- a/libavcodec/magicyuv.c
> > +++ b/libavcodec/magicyuv.c
> > @@ -411,6 +411,7 @@ static int build_huffman(AVCodecContext *avctx, const uint8_t *table,
> >  
> >          if (j == max) {
> >              j = 0;
> > +            ff_free_vlc(&s->vlc[i]);
> >              if (huff_build(he, length_count, &s->vlc[i], max)) {
> >                  av_log(avctx, AV_LOG_ERROR, "Cannot build Huffman codes\n");
> >                  return AVERROR_INVALIDDATA;
> > 
> Seems I inexplicably deleted this this in
> 1bf30a1beb7527887c9b466d07765d20fa3b5412; I did not notice it because
> all my tests were apparently done on files with only very few frames, so
> that nothing was overwritten; also all fate samples for magicyuv contain
> only one frame. Good that this has been caught; but could you add the
> ff_free_vlc() directly in front of ff_init_vlc_sparse() (where it used
> to be)?

sure, will apply with that changed

thx

[...]