diff mbox

[FFmpeg-devel] avcodec/mediacodec_sw_buffer: Fix segmentation fault with decoding on android oreo

Message ID 20180719155725.21165-1-pbennett@mythtv.org
State New
Headers show

Commit Message

Peter Bennett July 19, 2018, 3:57 p.m. UTC 
avcodec_receive_frame consistently causes a seg fault when decoding 1080i mpeg2
on android version oreo. When copying the frame, the second plane in the buffer
follows on immediately after 1080 lines of the first plane, but the code assumes
it is after 1088 lines of the first plane, based on slice_height. It crashes on
copying data for the second plane when it hits the actual end of the data and
starts accessing addresses beyond that.

Instead of using slice_height here, change to use use height. slice_height is
used at other places in this module and I do not know if they also need to be
changed. I have confirmed that with this change, decoding works correctly
on android oreo as well as on the prior version, android nougat.
---
 libavcodec/mediacodec_sw_buffer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Peter Bennett July 19, 2018, 4:44 p.m. UTC | #1 
There is an error in this patch. I will submit a corrected version. 
Please disregard this one.

Peter


On 07/19/2018 11:57 AM, Peter Bennett wrote:
> avcodec_receive_frame consistently causes a seg fault when decoding 1080i mpeg2
> on android version oreo. When copying the frame, the second plane in the buffer
> follows on immediately after 1080 lines of the first plane, but the code assumes
> it is after 1088 lines of the first plane, based on slice_height. It crashes on
> copying data for the second plane when it hits the actual end of the data and
> starts accessing addresses beyond that.
>
> Instead of using slice_height here, change to use use height. slice_height is
> used at other places in this module and I do not know if they also need to be
> changed. I have confirmed that with this change, decoding works correctly
> on android oreo as well as on the prior version, android nougat.
> ---
>   libavcodec/mediacodec_sw_buffer.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/libavcodec/mediacodec_sw_buffer.c b/libavcodec/mediacodec_sw_buffer.c
> index 92428e85f0..3b80e1fb59 100644
> --- a/libavcodec/mediacodec_sw_buffer.c
> +++ b/libavcodec/mediacodec_sw_buffer.c
> @@ -100,7 +100,7 @@ void ff_mediacodec_sw_buffer_copy_yuv420_planar(AVCodecContext *avctx,
>               src += s->slice_height * s->stride;
>   
>               if (i == 2) {
> -                src += ((s->slice_height + 1) / 2) * stride;
> +                src += ((s->height + 1) / 2) * stride;
>               }
>   
>               src += s->crop_top * stride;
diff mbox

Patch

diff --git a/libavcodec/mediacodec_sw_buffer.c b/libavcodec/mediacodec_sw_buffer.c
index 92428e85f0..3b80e1fb59 100644
--- a/libavcodec/mediacodec_sw_buffer.c
+++ b/libavcodec/mediacodec_sw_buffer.c
@@ -100,7 +100,7 @@  void ff_mediacodec_sw_buffer_copy_yuv420_planar(AVCodecContext *avctx,
             src += s->slice_height * s->stride;
 
             if (i == 2) {
-                src += ((s->slice_height + 1) / 2) * stride;
+                src += ((s->height + 1) / 2) * stride;
             }
 
             src += s->crop_top * stride;