| Message ID | 20190924220310.31157-10-andreas.rheinhardt@gmail.com |
|---|---|
| State | Superseded |
| Headers | show |
On Wed, Sep 25, 2019 at 12:03:04AM +0200, Andreas Rheinhardt wrote: > libavcodec/ffv1enc.c accessed an array of uint8_t [32] via array[0][j] > in order to loop over all the uint8_t in this array of arrays. Of course > this implied an out-of-bounds access for array[0] and UBSan complained > about this. So perform the access via an ordinary pointer to uint8_t. > > This affected the FATE-tests vsynth1-ffv1, vsynth1-ffv1-v3-yuv420p, > vsynth1-ffv1-v3-yuv422p10, vsynth1-ffv1-v3-yuv444p16, > vsynth1-ffv1-v3-bgr0, vsynth1-ffv1-ffv1-v3-rgb48 as well as the > corresponding vsynth2-*, vsynth3-* and the vsynth_lena-* tests. > > Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> > --- > libavcodec/ffv1enc.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/libavcodec/ffv1enc.c b/libavcodec/ffv1enc.c > index 1bf9663053..5eb439135c 100644 > --- a/libavcodec/ffv1enc.c > +++ b/libavcodec/ffv1enc.c > @@ -429,8 +429,9 @@ static int write_extradata(FFV1Context *f) > > for (i = 0; i < f->quant_table_count; i++) { > if (f->initial_states[i]) { > + uint8_t *initial_state = &f->initial_states[i][0][0]; > for (j = 0; j < f->context_count[i] * CONTEXT_SIZE; j++) > - if (f->initial_states[i][0][j] != 128) > + if (initial_state[j] != 128) I think if the code is changed, it might make sense to access the array through the correct indexes. Not so much for C compliance as for ease of understanding of whoever looks at this thx [...]
diff --git a/libavcodec/ffv1enc.c b/libavcodec/ffv1enc.c index 1bf9663053..5eb439135c 100644 --- a/libavcodec/ffv1enc.c +++ b/libavcodec/ffv1enc.c @@ -429,8 +429,9 @@ static int write_extradata(FFV1Context *f) for (i = 0; i < f->quant_table_count; i++) { if (f->initial_states[i]) { + uint8_t *initial_state = &f->initial_states[i][0][0]; for (j = 0; j < f->context_count[i] * CONTEXT_SIZE; j++) - if (f->initial_states[i][0][j] != 128) + if (initial_state[j] != 128) break; } else { j = f->context_count[i] * CONTEXT_SIZE;
libavcodec/ffv1enc.c accessed an array of uint8_t [32] via array[0][j] in order to loop over all the uint8_t in this array of arrays. Of course this implied an out-of-bounds access for array[0] and UBSan complained about this. So perform the access via an ordinary pointer to uint8_t. This affected the FATE-tests vsynth1-ffv1, vsynth1-ffv1-v3-yuv420p, vsynth1-ffv1-v3-yuv422p10, vsynth1-ffv1-v3-yuv444p16, vsynth1-ffv1-v3-bgr0, vsynth1-ffv1-ffv1-v3-rgb48 as well as the corresponding vsynth2-*, vsynth3-* and the vsynth_lena-* tests. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> --- libavcodec/ffv1enc.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)