diff mbox series

[FFmpeg-devel,1/4] avcodec/mpegvideo_dec: Check for existence of planes before accesses

Message ID AS8P250MB0744CC369A3887A6A38474498FC7A@AS8P250MB0744.EURP250.PROD.OUTLOOK.COM
State Accepted
Commit 3ede6cc0f18c3da842d18153b860d911bdbfbde9
Headers show
Series [FFmpeg-devel,1/4] avcodec/mpegvideo_dec: Check for existence of planes before accesses | expand

Checks

Context Check Description
yinshiyou/make_loongarch64 success Make finished
yinshiyou/make_fate_loongarch64 success Make fate finished
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished

Commit Message

Andreas Rheinhardt Sept. 30, 2023, 6:02 p.m. UTC 
Fixes segfaults with -debug +nomc -flags +gray (presuming
a build with --enable-gray).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
---
 libavcodec/mpegvideo_dec.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

Comments

Andreas Rheinhardt Oct. 2, 2023, 11:39 p.m. UTC | #1 
Andreas Rheinhardt:
> Fixes segfaults with -debug +nomc -flags +gray (presuming
> a build with --enable-gray).
> 
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
> ---
>  libavcodec/mpegvideo_dec.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/mpegvideo_dec.c b/libavcodec/mpegvideo_dec.c
> index 7aa46a4e25..77d90104f6 100644
> --- a/libavcodec/mpegvideo_dec.c
> +++ b/libavcodec/mpegvideo_dec.c
> @@ -249,10 +249,12 @@ static void gray_frame(AVFrame *frame)
>  {
>      int h_chroma_shift, v_chroma_shift;
>  
> -    av_pix_fmt_get_chroma_sub_sample(frame->format, &h_chroma_shift, &v_chroma_shift);
> -
>      for (int i = 0; i < frame->height; i++)
>          memset(frame->data[0] + frame->linesize[0] * i, 0x80, frame->width);
> +
> +    if (!frame->data[1])
> +        return;
> +    av_pix_fmt_get_chroma_sub_sample(frame->format, &h_chroma_shift, &v_chroma_shift);
>      for (int i = 0; i < AV_CEIL_RSHIFT(frame->height, v_chroma_shift); i++) {
>          memset(frame->data[1] + frame->linesize[1] * i,
>                 0x80, AV_CEIL_RSHIFT(frame->width, h_chroma_shift));

Will apply patches one, two and four of this patchset tonight unless
there are objections.

- Andreas
diff mbox series

Patch

diff --git a/libavcodec/mpegvideo_dec.c b/libavcodec/mpegvideo_dec.c
index 7aa46a4e25..77d90104f6 100644
--- a/libavcodec/mpegvideo_dec.c
+++ b/libavcodec/mpegvideo_dec.c
@@ -249,10 +249,12 @@  static void gray_frame(AVFrame *frame)
 {
     int h_chroma_shift, v_chroma_shift;
 
-    av_pix_fmt_get_chroma_sub_sample(frame->format, &h_chroma_shift, &v_chroma_shift);
-
     for (int i = 0; i < frame->height; i++)
         memset(frame->data[0] + frame->linesize[0] * i, 0x80, frame->width);
+
+    if (!frame->data[1])
+        return;
+    av_pix_fmt_get_chroma_sub_sample(frame->format, &h_chroma_shift, &v_chroma_shift);
     for (int i = 0; i < AV_CEIL_RSHIFT(frame->height, v_chroma_shift); i++) {
         memset(frame->data[1] + frame->linesize[1] * i,
                0x80, AV_CEIL_RSHIFT(frame->width, h_chroma_shift));