[FFmpeg-devel] avcodec/mpeg4videodec: Clear bits_per_raw_sample if it has originated from a previous instance

Submitted by Michael Niedermayer on June 9, 2018, 8:57 p.m.

Details

Message ID 20180609205705.2820-1-michael@niedermayer.cc
State Accepted
Commit 2fc108f60f98cd00813418a8754a46476b404a3c
Headers show

Commit Message

Michael Niedermayer June 9, 2018, 8:57 p.m.
Fixes: assertion failure
Fixes: ffmpeg_crash_5.avi

Found-by: Thuan Pham <thuanpv@comp.nus.edu.sg>, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/mpeg4videodec.c | 6 ++++++
 1 file changed, 6 insertions(+)

Comments

Michael Niedermayer June 13, 2018, 5:50 p.m.
On Sat, Jun 09, 2018 at 10:57:05PM +0200, Michael Niedermayer wrote:
> Fixes: assertion failure
> Fixes: ffmpeg_crash_5.avi
> 
> Found-by: Thuan Pham <thuanpv@comp.nus.edu.sg>, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/mpeg4videodec.c | 6 ++++++
>  1 file changed, 6 insertions(+)

will apply

[...]

Patch hide | download patch | download mbox

diff --git a/libavcodec/mpeg4videodec.c b/libavcodec/mpeg4videodec.c
index 51b58fac02..d0ebaac6e8 100644
--- a/libavcodec/mpeg4videodec.c
+++ b/libavcodec/mpeg4videodec.c
@@ -3109,6 +3109,12 @@  int ff_mpeg4_decode_picture_header(Mpeg4DecContext *ctx, GetBitContext *gb)
     /* search next start code */
     align_get_bits(gb);
 
+    // If we have not switched to studio profile than we also did not switch bps
+    // that means something else (like a previous instance) outside set bps which
+    // would be inconsistant with the currect state, thus reset it
+    if (!s->studio_profile && s->avctx->bits_per_raw_sample != 8)
+        s->avctx->bits_per_raw_sample = 0;
+
     if (s->codec_tag == AV_RL32("WV1F") && show_bits(gb, 24) == 0x575630) {
         skip_bits(gb, 24);
         if (get_bits(gb, 8) == 0xF0)