| Message ID | 6c55b4a0-4c84-a1b9-cdf3-ae3e7e527f68@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel] swscale/x86/yuv_2_rgb: fix access to memory past the frame data in yuv to rgb conversion | expand |
| Context | Check | Description |
|---|---|---|
| andriy/configure_x86 | warning | Failed to apply patch |
| yinshiyou/configure_loongarch64 | warning | Failed to apply patch |
| andriy/configure_armv7_RPi4 | warning | Failed to apply patch |
Quoting Vardan Margaryan (2022-05-23 18:06:16) > Y, U, V data is loaded at the end of the current iteration for the next > iteration. > It results in memory access past the frame data on the last iteration > (that data is never used after the loading). > > So load data at the start of the iteration, so that only useful data is > loaded. > > Signed-off-by: Vardan Margaryan <v.t.margaryan@gmail.com> > --- > libswscale/x86/yuv_2_rgb.asm | 5 +---- > 1 file changed, 1 insertion(+), 4 deletions(-) I'm not an expert on this code, but the patch looks good and it fixes some invalid reads for me, so will push in a few days if nobody objects. Also note that your mailer mangled the patch, so it could not be applied automatically with 'git am', please fix this in future submissions e.g. by attaching the patch or sending it with git send-email.
diff --git a/libswscale/x86/yuv_2_rgb.asm b/libswscale/x86/yuv_2_rgb.asm index 2f0b4130df..f968b3a0a2 100644 --- a/libswscale/x86/yuv_2_rgb.asm +++ b/libswscale/x86/yuv_2_rgb.asm @@ -139,10 +139,10 @@ cglobal %1_420_%2%3, GPR_num, GPR_num, reg_num, parameters VBROADCASTSD vr_coff, [pointer_c_ditherq + 4 * 8] %endif %endif +.loop0: movu m_y, [py_2indexq + 2 * indexq] movh m_u, [pu_indexq + indexq] movh m_v, [pv_indexq + indexq] -.loop0: pxor m4, m4 mova m7, m6
Y, U, V data is loaded at the end of the current iteration for the next iteration. It results in memory access past the frame data on the last iteration (that data is never used after the loading). So load data at the start of the iteration, so that only useful data is loaded. Signed-off-by: Vardan Margaryan <v.t.margaryan@gmail.com> --- libswscale/x86/yuv_2_rgb.asm | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) punpcklbw m0, m4 @@ -347,9 +347,6 @@ cglobal %1_420_%2%3, GPR_num, GPR_num, reg_num, parameters %endif ; PACK RGB15/16 %endif ; PACK RGB15/16/32 -movu m_y, [py_2indexq + 2 * indexq + 8 * time_num] -movh m_v, [pv_indexq + indexq + 4 * time_num] -movh m_u, [pu_indexq + indexq + 4 * time_num] add imageq, 8 * depth * time_num add indexq, 4 * time_num js .loop0