diff mbox

[FFmpeg-devel] mpegpicture: use coded_width/coded_height to allocate frame

Message ID 48db2c6a-24e9-2c7c-f924-860f9b90d947@googlemail.com
State Accepted
Headers show

Commit Message

Andreas Cadhalpun Nov. 25, 2016, 1:26 a.m. UTC
On 25.11.2016 01:38, Michael Niedermayer wrote:
> On Fri, Nov 25, 2016 at 12:03:30AM +0100, Andreas Cadhalpun wrote:
>>  mss2.c |   13 ++++++++++---
>>  1 file changed, 10 insertions(+), 3 deletions(-)
>> 884b912643244a4205bac63faedfa0c048bcc97a  0001-mss2-only-use-error-correction-for-matching-block-co.patch
>> From df9241d8b575cc0fbf570e714c586ff37a4821fd Mon Sep 17 00:00:00 2001
>> From: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
>> Date: Thu, 24 Nov 2016 23:57:46 +0100
>> Subject: [PATCH] mss2: only use error correction for matching block counts
>>
>> This fixes a heap-buffer-overflow in ff_er_frame_end when decoding mss2
>> with coded_width/coded_height larger than width/height.
>>
>> Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
>> ---
>>  libavcodec/mss2.c | 13 ++++++++++---
>>  1 file changed, 10 insertions(+), 3 deletions(-)
>>
>> diff --git a/libavcodec/mss2.c b/libavcodec/mss2.c
>> index 1e24568..62761e8 100644
>> --- a/libavcodec/mss2.c
>> +++ b/libavcodec/mss2.c
>> @@ -409,8 +409,6 @@ static int decode_wmv9(AVCodecContext *avctx, const uint8_t *buf, int buf_size,
>>          return ret;
>>      }
>>  
>> -    ff_mpeg_er_frame_start(s);
>> -
>>      v->bits = buf_size * 8;
>>  
>>      v->end_mb_x = (w + 15) >> 4;
>> @@ -420,9 +418,18 @@ static int decode_wmv9(AVCodecContext *avctx, const uint8_t *buf, int buf_size,
>>      if (v->respic & 2)
>>          s->end_mb_y = s->end_mb_y + 1 >> 1;
>>  
>> +    if (v->end_mb_x == s->mb_width && s->end_mb_y == s->mb_height) {
>> +        ff_mpeg_er_frame_start(s);
>> +    } else {
>> +        av_log(v->s.avctx, AV_LOG_WARNING,
>> +               "disabling error correction due to block count mismatch %dx%d != %dx%d\n",
>> +               v->end_mb_x, s->end_mb_y, s->mb_width, s->mb_height);
>> +    }
>> +
>>      ff_vc1_decode_blocks(v);
>>  
>> -    ff_er_frame_end(&s->er);
>> +    if (v->end_mb_x == s->mb_width && s->end_mb_y == s->mb_height)
>> +        ff_er_frame_end(&s->er);
> 
> there are still ff_er_add_slice() calls in the block decode code i think
> It seems not to matter but skiping just ff_er_frame_end() and
> not ff_mpeg_er_frame_start() feels less inconsistent

OK, update patch is attached.

Best regards,
Andreas

Comments

Michael Niedermayer Nov. 25, 2016, 1:59 a.m. UTC | #1
On Fri, Nov 25, 2016 at 02:26:24AM +0100, Andreas Cadhalpun wrote:
> On 25.11.2016 01:38, Michael Niedermayer wrote:
> > On Fri, Nov 25, 2016 at 12:03:30AM +0100, Andreas Cadhalpun wrote:
> >>  mss2.c |   13 ++++++++++---
> >>  1 file changed, 10 insertions(+), 3 deletions(-)
> >> 884b912643244a4205bac63faedfa0c048bcc97a  0001-mss2-only-use-error-correction-for-matching-block-co.patch
> >> From df9241d8b575cc0fbf570e714c586ff37a4821fd Mon Sep 17 00:00:00 2001
> >> From: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
> >> Date: Thu, 24 Nov 2016 23:57:46 +0100
> >> Subject: [PATCH] mss2: only use error correction for matching block counts
> >>
> >> This fixes a heap-buffer-overflow in ff_er_frame_end when decoding mss2
> >> with coded_width/coded_height larger than width/height.
> >>
> >> Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
> >> ---
> >>  libavcodec/mss2.c | 13 ++++++++++---
> >>  1 file changed, 10 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/libavcodec/mss2.c b/libavcodec/mss2.c
> >> index 1e24568..62761e8 100644
> >> --- a/libavcodec/mss2.c
> >> +++ b/libavcodec/mss2.c
> >> @@ -409,8 +409,6 @@ static int decode_wmv9(AVCodecContext *avctx, const uint8_t *buf, int buf_size,
> >>          return ret;
> >>      }
> >>  
> >> -    ff_mpeg_er_frame_start(s);
> >> -
> >>      v->bits = buf_size * 8;
> >>  
> >>      v->end_mb_x = (w + 15) >> 4;
> >> @@ -420,9 +418,18 @@ static int decode_wmv9(AVCodecContext *avctx, const uint8_t *buf, int buf_size,
> >>      if (v->respic & 2)
> >>          s->end_mb_y = s->end_mb_y + 1 >> 1;
> >>  
> >> +    if (v->end_mb_x == s->mb_width && s->end_mb_y == s->mb_height) {
> >> +        ff_mpeg_er_frame_start(s);
> >> +    } else {
> >> +        av_log(v->s.avctx, AV_LOG_WARNING,
> >> +               "disabling error correction due to block count mismatch %dx%d != %dx%d\n",
> >> +               v->end_mb_x, s->end_mb_y, s->mb_width, s->mb_height);
> >> +    }
> >> +
> >>      ff_vc1_decode_blocks(v);
> >>  
> >> -    ff_er_frame_end(&s->er);
> >> +    if (v->end_mb_x == s->mb_width && s->end_mb_y == s->mb_height)
> >> +        ff_er_frame_end(&s->er);
> > 
> > there are still ff_er_add_slice() calls in the block decode code i think
> > It seems not to matter but skiping just ff_er_frame_end() and
> > not ff_mpeg_er_frame_start() feels less inconsistent
> 
> OK, update patch is attached.
> 
> Best regards,
> Andreas

>  mss2.c |    8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 958ee0811485404a0662a1540fcb8b131423947b  0001-mss2-only-use-error-correction-for-matching-block-co.patch
> From 6d8b5136c67f3a8cb3f4a4c818f311d748bbab5d Mon Sep 17 00:00:00 2001
> From: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
> Date: Thu, 24 Nov 2016 23:57:46 +0100
> Subject: [PATCH] mss2: only use error correction for matching block counts
> 
> This fixes a heap-buffer-overflow in ff_er_frame_end when decoding mss2
> with coded_width/coded_height larger than width/height.
> 
> Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
> ---
>  libavcodec/mss2.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)

LGTM

thx

[...]
Andreas Cadhalpun Nov. 25, 2016, 8:06 p.m. UTC | #2
On 25.11.2016 02:59, Michael Niedermayer wrote:
> On Fri, Nov 25, 2016 at 02:26:24AM +0100, Andreas Cadhalpun wrote:
>>  mss2.c |    8 +++++++-
>>  1 file changed, 7 insertions(+), 1 deletion(-)
>> 958ee0811485404a0662a1540fcb8b131423947b  0001-mss2-only-use-error-correction-for-matching-block-co.patch
>> From 6d8b5136c67f3a8cb3f4a4c818f311d748bbab5d Mon Sep 17 00:00:00 2001
>> From: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
>> Date: Thu, 24 Nov 2016 23:57:46 +0100
>> Subject: [PATCH] mss2: only use error correction for matching block counts
>>
>> This fixes a heap-buffer-overflow in ff_er_frame_end when decoding mss2
>> with coded_width/coded_height larger than width/height.
>>
>> Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
>> ---
>>  libavcodec/mss2.c | 8 +++++++-
>>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> LGTM

Pushed.

Best regards,
Andreas
diff mbox

Patch

From 6d8b5136c67f3a8cb3f4a4c818f311d748bbab5d Mon Sep 17 00:00:00 2001
From: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Date: Thu, 24 Nov 2016 23:57:46 +0100
Subject: [PATCH] mss2: only use error correction for matching block counts

This fixes a heap-buffer-overflow in ff_er_frame_end when decoding mss2
with coded_width/coded_height larger than width/height.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
---
 libavcodec/mss2.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/libavcodec/mss2.c b/libavcodec/mss2.c
index 1e24568..581865b 100644
--- a/libavcodec/mss2.c
+++ b/libavcodec/mss2.c
@@ -422,7 +422,13 @@  static int decode_wmv9(AVCodecContext *avctx, const uint8_t *buf, int buf_size,
 
     ff_vc1_decode_blocks(v);
 
-    ff_er_frame_end(&s->er);
+    if (v->end_mb_x == s->mb_width && s->end_mb_y == s->mb_height) {
+        ff_er_frame_end(&s->er);
+    } else {
+        av_log(v->s.avctx, AV_LOG_WARNING,
+               "disabling error correction due to block count mismatch %dx%d != %dx%d\n",
+               v->end_mb_x, s->end_mb_y, s->mb_width, s->mb_height);
+    }
 
     ff_mpv_frame_end(s);
 
-- 
2.10.2