diff mbox

[FFmpeg-devel] dvbsubdec: fix division by zero in compute_default_clut

Message ID df67a789-79e7-937e-5111-96181239af5f@googlemail.com
State Accepted
Headers show

Commit Message

Andreas Cadhalpun Nov. 9, 2016, 7:52 p.m. UTC
On 09.11.2016 02:27, Michael Niedermayer wrote:
> On Tue, Nov 08, 2016 at 10:36:01PM +0100, Andreas Cadhalpun wrote:
>> Since the loop goes down to i = 0 the number of iterations (variable
>> count) is the original i, not i - 1.
>> In the case of i originally being 1, this miscalculation resulted in
>> a division by zero.
>>
>> This problem was introduced in commit
>> 4b90dcb8493552c17a811c8b1e6538dae4061f9d.
>>
>> Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
>> ---
>>  libavcodec/dvbsubdec.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c
>> index e9f4765..31898ea 100644
>> --- a/libavcodec/dvbsubdec.c
>> +++ b/libavcodec/dvbsubdec.c
>> @@ -810,7 +810,7 @@ static void compute_default_clut(AVSubtitleRect *rect, int w, int h)
>>          list_inv[     i ] = bestv;
>>      }
>>  
>> -    count = i - 1;
>> +    count = i;
>>      for (i--; i>=0; i--) {
>>          int v = i*255/count;
>>          AV_WN32(rect->data[1] + 4*list_inv[i], RGBA(v/2,v,v/2,v));
> 
> with this a 2 color text would be 50% and 100% transparent
> it should be transparent background and opaque text,
> count = FFMAX(i - 1, 1);
> seems better to me

OK, patch doing that is attached.

Best regards,
Andreas

Comments

Michael Niedermayer Nov. 10, 2016, 4:42 p.m. UTC | #1
On Wed, Nov 09, 2016 at 08:52:54PM +0100, Andreas Cadhalpun wrote:
> On 09.11.2016 02:27, Michael Niedermayer wrote:
> > On Tue, Nov 08, 2016 at 10:36:01PM +0100, Andreas Cadhalpun wrote:
> >> Since the loop goes down to i = 0 the number of iterations (variable
> >> count) is the original i, not i - 1.
> >> In the case of i originally being 1, this miscalculation resulted in
> >> a division by zero.
> >>
> >> This problem was introduced in commit
> >> 4b90dcb8493552c17a811c8b1e6538dae4061f9d.
> >>
> >> Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
> >> ---
> >>  libavcodec/dvbsubdec.c | 2 +-
> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c
> >> index e9f4765..31898ea 100644
> >> --- a/libavcodec/dvbsubdec.c
> >> +++ b/libavcodec/dvbsubdec.c
> >> @@ -810,7 +810,7 @@ static void compute_default_clut(AVSubtitleRect *rect, int w, int h)
> >>          list_inv[     i ] = bestv;
> >>      }
> >>  
> >> -    count = i - 1;
> >> +    count = i;
> >>      for (i--; i>=0; i--) {
> >>          int v = i*255/count;
> >>          AV_WN32(rect->data[1] + 4*list_inv[i], RGBA(v/2,v,v/2,v));
> > 
> > with this a 2 color text would be 50% and 100% transparent
> > it should be transparent background and opaque text,
> > count = FFMAX(i - 1, 1);
> > seems better to me
> 
> OK, patch doing that is attached.
> 
> Best regards,
> Andreas
> 

>  dvbsubdec.c |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 10b117a15971daf75d61efe486b961d3cc59d480  0001-dvbsubdec-fix-division-by-zero-in-compute_default_cl.patch
> From 436318575abf482e1ea144f651c47bc2730e41a2 Mon Sep 17 00:00:00 2001
> From: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
> Date: Tue, 8 Nov 2016 22:32:42 +0100
> Subject: [PATCH] dvbsubdec: fix division by zero in compute_default_clut
> 
> This problem was introduced in commit
> 4b90dcb8493552c17a811c8b1e6538dae4061f9d.
> 
> Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
> ---
>  libavcodec/dvbsubdec.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

LGTM

thx

[...]
Andreas Cadhalpun Nov. 10, 2016, 8:04 p.m. UTC | #2
On 10.11.2016 17:42, Michael Niedermayer wrote:
> On Wed, Nov 09, 2016 at 08:52:54PM +0100, Andreas Cadhalpun wrote:
>>  dvbsubdec.c |    2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>> 10b117a15971daf75d61efe486b961d3cc59d480  0001-dvbsubdec-fix-division-by-zero-in-compute_default_cl.patch
>> From 436318575abf482e1ea144f651c47bc2730e41a2 Mon Sep 17 00:00:00 2001
>> From: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
>> Date: Tue, 8 Nov 2016 22:32:42 +0100
>> Subject: [PATCH] dvbsubdec: fix division by zero in compute_default_clut
>>
>> This problem was introduced in commit
>> 4b90dcb8493552c17a811c8b1e6538dae4061f9d.
>>
>> Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
>> ---
>>  libavcodec/dvbsubdec.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> LGTM

Pushed.

Best regards,
Andreas
diff mbox

Patch

From 436318575abf482e1ea144f651c47bc2730e41a2 Mon Sep 17 00:00:00 2001
From: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Date: Tue, 8 Nov 2016 22:32:42 +0100
Subject: [PATCH] dvbsubdec: fix division by zero in compute_default_clut

This problem was introduced in commit
4b90dcb8493552c17a811c8b1e6538dae4061f9d.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
---
 libavcodec/dvbsubdec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c
index e9f4765..bf3b1a1 100644
--- a/libavcodec/dvbsubdec.c
+++ b/libavcodec/dvbsubdec.c
@@ -810,7 +810,7 @@  static void compute_default_clut(AVSubtitleRect *rect, int w, int h)
         list_inv[     i ] = bestv;
     }
 
-    count = i - 1;
+    count = FFMAX(i - 1, 1);
     for (i--; i>=0; i--) {
         int v = i*255/count;
         AV_WN32(rect->data[1] + 4*list_inv[i], RGBA(v/2,v,v/2,v));
-- 
2.10.2