diff mbox

[FFmpeg-devel] dvbsubdec: fix division by zero in compute_default_clut

Message ID 6d200416-f02f-370b-3e15-c0bc58792bd1@googlemail.com
State Superseded
Headers show

Commit Message

Andreas Cadhalpun Nov. 8, 2016, 9:36 p.m. UTC
Since the loop goes down to i = 0 the number of iterations (variable
count) is the original i, not i - 1.
In the case of i originally being 1, this miscalculation resulted in
a division by zero.

This problem was introduced in commit
4b90dcb8493552c17a811c8b1e6538dae4061f9d.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
---
 libavcodec/dvbsubdec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Michael Niedermayer Nov. 9, 2016, 1:27 a.m. UTC | #1
On Tue, Nov 08, 2016 at 10:36:01PM +0100, Andreas Cadhalpun wrote:
> Since the loop goes down to i = 0 the number of iterations (variable
> count) is the original i, not i - 1.
> In the case of i originally being 1, this miscalculation resulted in
> a division by zero.
> 
> This problem was introduced in commit
> 4b90dcb8493552c17a811c8b1e6538dae4061f9d.
> 
> Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
> ---
>  libavcodec/dvbsubdec.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c
> index e9f4765..31898ea 100644
> --- a/libavcodec/dvbsubdec.c
> +++ b/libavcodec/dvbsubdec.c
> @@ -810,7 +810,7 @@ static void compute_default_clut(AVSubtitleRect *rect, int w, int h)
>          list_inv[     i ] = bestv;
>      }
>  
> -    count = i - 1;
> +    count = i;
>      for (i--; i>=0; i--) {
>          int v = i*255/count;
>          AV_WN32(rect->data[1] + 4*list_inv[i], RGBA(v/2,v,v/2,v));

with this a 2 color text would be 50% and 100% transparent
it should be transparent background and opaque text,
count = FFMAX(i - 1, 1);
seems better to me


[...]
diff mbox

Patch

diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c
index e9f4765..31898ea 100644
--- a/libavcodec/dvbsubdec.c
+++ b/libavcodec/dvbsubdec.c
@@ -810,7 +810,7 @@  static void compute_default_clut(AVSubtitleRect *rect, int w, int h)
         list_inv[     i ] = bestv;
     }
 
-    count = i - 1;
+    count = i;
     for (i--; i>=0; i--) {
         int v = i*255/count;
         AV_WN32(rect->data[1] + 4*list_inv[i], RGBA(v/2,v,v/2,v));