diff mbox

[FFmpeg-devel,v2,1/4] avcodec/tiff: correct unpacking of border YUV pixels

Message ID 5bb1e2e7-faf8-7244-b48b-9bc21842b010@gmail.com
State New
Headers show

Commit Message

Skakov Pavel Oct. 10, 2019, 12:05 a.m. UTC 
Corrected handling of border pixels when unpacking YUV. Example (bottom line): https://github.com/BitMiracle/libtiff.net/blob/master/TestCase/ycbcr-cat.tif

Comments

Michael Niedermayer Oct. 11, 2019, 7:55 p.m. UTC | #1 
On Thu, Oct 10, 2019 at 03:05:11AM +0300, Skakov Pavel wrote:
> Corrected handling of border pixels when unpacking YUV. Example (bottom line): https://github.com/BitMiracle/libtiff.net/blob/master/TestCase/ycbcr-cat.tif

>  tiff.c |    7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 3959d6f0fc427fce64684a2768d3ac154ff49993  0001-avcodec-tiff-correct-unpacking-of-border-YUV-pixels.patch
> From 09f9822d8d38f26ff386df3d01794fcafe3f3869 Mon Sep 17 00:00:00 2001
> From: Pavel Skakov <pavelsx@gmail.com>
> Date: Thu, 10 Oct 2019 02:21:06 +0300
> Subject: [PATCH 1/4] avcodec/tiff: correct unpacking of border YUV pixels
> 
> Signed-off-by: Pavel Skakov <pavelsx@gmail.com>
> ---
>  libavcodec/tiff.c | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/tiff.c b/libavcodec/tiff.c
> index 069792b4c3..70c49f7d44 100644
> --- a/libavcodec/tiff.c
> +++ b/libavcodec/tiff.c
> @@ -369,8 +369,11 @@ static void unpack_yuv(TiffContext *s, AVFrame *p,
>          for (i = 0; i < w; i++) {
>              for (j = 0; j < s->subsampling[1]; j++)
>                  for (k = 0; k < s->subsampling[0]; k++)
> -                    p->data[0][FFMIN(lnum + j, s->height-1) * p->linesize[0] +
> -                               FFMIN(i * s->subsampling[0] + k, s->width-1)] = *src++;
> +                    if (j < s->height - lnum && i * s->subsampling[0] + k < s->width) {
> +                        p->data[0][(lnum + j) * p->linesize[0] + i * s->subsampling[0] + k] = *src++;
> +                    } else {
> +                        src++;
> +                    }
>              *pu++ = *src++;
>              *pv++ = *src++;
>          }

Its probably better to ensure the buffer is large enough instead of using
special cases

thx

[...]
diff mbox

Patch

From 09f9822d8d38f26ff386df3d01794fcafe3f3869 Mon Sep 17 00:00:00 2001
From: Pavel Skakov <pavelsx@gmail.com>
Date: Thu, 10 Oct 2019 02:21:06 +0300
Subject: [PATCH 1/4] avcodec/tiff: correct unpacking of border YUV pixels

Signed-off-by: Pavel Skakov <pavelsx@gmail.com>
---
 libavcodec/tiff.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/libavcodec/tiff.c b/libavcodec/tiff.c
index 069792b4c3..70c49f7d44 100644
--- a/libavcodec/tiff.c
+++ b/libavcodec/tiff.c
@@ -369,8 +369,11 @@  static void unpack_yuv(TiffContext *s, AVFrame *p,
         for (i = 0; i < w; i++) {
             for (j = 0; j < s->subsampling[1]; j++)
                 for (k = 0; k < s->subsampling[0]; k++)
-                    p->data[0][FFMIN(lnum + j, s->height-1) * p->linesize[0] +
-                               FFMIN(i * s->subsampling[0] + k, s->width-1)] = *src++;
+                    if (j < s->height - lnum && i * s->subsampling[0] + k < s->width) {
+                        p->data[0][(lnum + j) * p->linesize[0] + i * s->subsampling[0] + k] = *src++;
+                    } else {
+                        src++;
+                    }
             *pu++ = *src++;
             *pv++ = *src++;
         }
-- 
2.13.2.windows.1