[FFmpeg-devel,v3] vaapi_encode_mjpeg: fix bad component id bug

Submitted by U. Artie Eoff on June 7, 2019, 9:41 p.m.

Details

Message ID 20190607214137.20333-1-ullysses.a.eoff@intel.com
State New
Headers show

Commit Message

U. Artie Eoff June 7, 2019, 9:41 p.m.
The compound literals assigned to "components"
only exist within the scope of the if/else
block (thanks Mike Thompson for the better
explanation).

Thus, after this if/else block, "components"
ends up pointing to an arbitrary/undefined
array.  With some compilers and depending on
optimization settings, these arbitrary values
may end up being the same value (i.e. 0 with
GNU GCC 9.x).  Unfortunately, the GNU GCC
compiler, at least, never prints any warnings
about this.

This patch fixes this issue by assigning the
constant arrays to local variables at function
scope and then pointing "components" to those
as necessary.

Fixes #7915

Signed-off-by: U. Artie Eoff <ullysses.a.eoff@intel.com>
---
 libavcodec/vaapi_encode_mjpeg.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

Comments

U. Artie Eoff June 7, 2019, 9:44 p.m.
> -----Original Message-----
> From: Eoff, Ullysses A
> Sent: Friday, June 07, 2019 2:42 PM
> To: ffmpeg-devel@ffmpeg.org
> Cc: Eoff, Ullysses A <ullysses.a.eoff@intel.com>
> Subject: [PATCH v3] vaapi_encode_mjpeg: fix bad component id bug
> 
> The compound literals assigned to "components"
> only exist within the scope of the if/else
> block (thanks Mike Thompson for the better

Oh crap, I got your name wrong.  It must be a Friday!  V4 coming up!

> explanation).
> 
> Thus, after this if/else block, "components"
> ends up pointing to an arbitrary/undefined
> array.  With some compilers and depending on
> optimization settings, these arbitrary values
> may end up being the same value (i.e. 0 with
> GNU GCC 9.x).  Unfortunately, the GNU GCC
> compiler, at least, never prints any warnings
> about this.
> 
> This patch fixes this issue by assigning the
> constant arrays to local variables at function
> scope and then pointing "components" to those
> as necessary.
> 
> Fixes #7915
> 
> Signed-off-by: U. Artie Eoff <ullysses.a.eoff@intel.com>
> ---
>  libavcodec/vaapi_encode_mjpeg.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/vaapi_encode_mjpeg.c b/libavcodec/vaapi_encode_mjpeg.c
> index 4dcdc3d16bb0..bd029cc90315 100644
> --- a/libavcodec/vaapi_encode_mjpeg.c
> +++ b/libavcodec/vaapi_encode_mjpeg.c
> @@ -227,6 +227,8 @@ static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx,
>      JPEGRawScanHeader                 *sh = &priv->scan.header;
>      VAEncPictureParameterBufferJPEG *vpic = pic->codec_picture_params;
>      const AVPixFmtDescriptor *desc;
> +    const uint8_t components_rgb[3] = { 'R', 'G', 'B' };
> +    const uint8_t components_yuv[3] = {  1,   2,   3  };
>      const uint8_t *components;
>      int t, i, quant_scale, len;
> 
> @@ -235,9 +237,9 @@ static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx,
>      desc = av_pix_fmt_desc_get(priv->common.input_frames->sw_format);
>      av_assert0(desc);
>      if (desc->flags & AV_PIX_FMT_FLAG_RGB)
> -        components = (uint8_t[3]) { 'R', 'G', 'B' };
> +        components = components_rgb;
>      else
> -        components = (uint8_t[3]) {  1,   2,   3  };
> +        components = components_yuv;
> 
>      // Frame header.
> 
> --
> 2.20.1

Patch hide | download patch | download mbox

diff --git a/libavcodec/vaapi_encode_mjpeg.c b/libavcodec/vaapi_encode_mjpeg.c
index 4dcdc3d16bb0..bd029cc90315 100644
--- a/libavcodec/vaapi_encode_mjpeg.c
+++ b/libavcodec/vaapi_encode_mjpeg.c
@@ -227,6 +227,8 @@  static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx,
     JPEGRawScanHeader                 *sh = &priv->scan.header;
     VAEncPictureParameterBufferJPEG *vpic = pic->codec_picture_params;
     const AVPixFmtDescriptor *desc;
+    const uint8_t components_rgb[3] = { 'R', 'G', 'B' };
+    const uint8_t components_yuv[3] = {  1,   2,   3  };
     const uint8_t *components;
     int t, i, quant_scale, len;
 
@@ -235,9 +237,9 @@  static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx,
     desc = av_pix_fmt_desc_get(priv->common.input_frames->sw_format);
     av_assert0(desc);
     if (desc->flags & AV_PIX_FMT_FLAG_RGB)
-        components = (uint8_t[3]) { 'R', 'G', 'B' };
+        components = components_rgb;
     else
-        components = (uint8_t[3]) {  1,   2,   3  };
+        components = components_yuv;
 
     // Frame header.