diff mbox

[FFmpeg-devel] avcodec/mediacodec_sw_buffer: Fix segmentation fault with decoding on android oreo (corrected)

Message ID 20180719170015.23608-1-pbennett@mythtv.org
State New
Headers show

Commit Message

Peter Bennett July 19, 2018, 5 p.m. UTC 
This is a correction of the earlier submission of this patch.

avcodec_receive_frame consistently causes a seg fault when decoding 1080i mpeg2
on android version oreo. When copying the frame, the second plane in the buffer
follows on immediately after 1080 lines of the first plane, but the code assumes
it is after 1088 lines of the first plane, based on slice_height. It crashes on
copying data for the second plane when it hits the actual end of the data and
starts accessing addresses beyond that.

Instead of using slice_height here, change to use use height. slice_height is
used at other places in this module and I do not know if they also need to be
changed. I have confirmed that with this change, decoding works correctly
on android oreo as well as on the prior version, android nougat.
---
 libavcodec/mediacodec_sw_buffer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Aman Karmani July 31, 2018, 9:31 p.m. UTC | #1 
On Thu, Jul 19, 2018 at 10:08 AM Peter Bennett <pb.mythtv@gmail.com> wrote:

> This is a correction of the earlier submission of this patch.
>
> avcodec_receive_frame consistently causes a seg fault when decoding 1080i
> mpeg2
> on android version oreo. When copying the frame, the second plane in the
> buffer
> follows on immediately after 1080 lines of the first plane, but the code
> assumes
> it is after 1088 lines of the first plane, based on slice_height. It
> crashes on
> copying data for the second plane when it hits the actual end of the data
> and
> starts accessing addresses beyond that.
>
> Instead of using slice_height here, change to use use height. slice_height
> is
> used at other places in this module and I do not know if they also need to
> be
> changed. I have confirmed that with this change, decoding works correctly
> on android oreo as well as on the prior version, android nougat.
>

This issue is not specific to 1080i mpeg2, and can be reproduced with 1080p
h264 as well.

slice_height is definitely the correct variable to use here. The problem is
it is not being set correctly. I will send an updated patch.

Aman


> ---
>  libavcodec/mediacodec_sw_buffer.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/libavcodec/mediacodec_sw_buffer.c
> b/libavcodec/mediacodec_sw_buffer.c
> index 92428e85f0..30a53f05b3 100644
> --- a/libavcodec/mediacodec_sw_buffer.c
> +++ b/libavcodec/mediacodec_sw_buffer.c
> @@ -150,7 +150,7 @@ void
> ff_mediacodec_sw_buffer_copy_yuv420_semi_planar(AVCodecContext *avctx,
>          } else if (i == 1) {
>              height = avctx->height / 2;
>
> -            src += s->slice_height * s->stride;
> +            src += s->height * s->stride;
>              src += s->crop_top * s->stride;
>              src += s->crop_left;
>          }
> --
> 2.17.1
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
diff mbox

Patch

diff --git a/libavcodec/mediacodec_sw_buffer.c b/libavcodec/mediacodec_sw_buffer.c
index 92428e85f0..30a53f05b3 100644
--- a/libavcodec/mediacodec_sw_buffer.c
+++ b/libavcodec/mediacodec_sw_buffer.c
@@ -150,7 +150,7 @@  void ff_mediacodec_sw_buffer_copy_yuv420_semi_planar(AVCodecContext *avctx,
         } else if (i == 1) {
             height = avctx->height / 2;
 
-            src += s->slice_height * s->stride;
+            src += s->height * s->stride;
             src += s->crop_top * s->stride;
             src += s->crop_left;
         }