diff mbox

[FFmpeg-devel,v2,02/15] avfilter/vf_hqx: Fix undefined left shifts of negative numbers

Message ID 20190928022610.5903-2-andreas.rheinhardt@gmail.com
State Accepted
Commit fa211943265ca991548a4cc2f85a6df9cedcd092
Headers show

Commit Message

Andreas Rheinhardt Sept. 28, 2019, 2:25 a.m. UTC 
Affected every usage of this filter; in particular, it affected the
FATE-tests filter-2xbr, filter-3xbr and filter-4xbr.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
---
 libavfilter/vf_hqx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Paul B Mahol Sept. 28, 2019, 1:09 p.m. UTC | #1 
lgtm

On 9/28/19, Andreas Rheinhardt <andreas.rheinhardt@gmail.com> wrote:
> Affected every usage of this filter; in particular, it affected the
> FATE-tests filter-2xbr, filter-3xbr and filter-4xbr.
>
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
> ---
>  libavfilter/vf_hqx.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/libavfilter/vf_hqx.c b/libavfilter/vf_hqx.c
> index 16a1be7bd4..4f768c7a13 100644
> --- a/libavfilter/vf_hqx.c
> +++ b/libavfilter/vf_hqx.c
> @@ -523,7 +523,7 @@ static av_cold int init(AVFilterContext *ctx)
>              int startg = FFMAX3(-bg, -rg, 0);
>              int endg = FFMIN3(255-bg, 255-rg, 255);
>              uint32_t y = (uint32_t)(( 299*rg + 1000*startg + 114*bg)/1000);
> -            c = bg + (rg<<16) + 0x010101 * startg;
> +            c = bg + rg * (1 << 16) + 0x010101 * startg;
>              for (g = startg; g <= endg; g++) {
>                  hqx->rgbtoyuv[c] = ((y++) << 16) + (u << 8) + v;
>                  c+= 0x010101;
> --
> 2.20.1
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Michael Niedermayer Sept. 28, 2019, 3:36 p.m. UTC | #2 
On Sat, Sep 28, 2019 at 03:09:07PM +0200, Paul B Mahol wrote:
> lgtm

will apply

thx

[...]
diff mbox

Patch

diff --git a/libavfilter/vf_hqx.c b/libavfilter/vf_hqx.c
index 16a1be7bd4..4f768c7a13 100644
--- a/libavfilter/vf_hqx.c
+++ b/libavfilter/vf_hqx.c
@@ -523,7 +523,7 @@  static av_cold int init(AVFilterContext *ctx)
             int startg = FFMAX3(-bg, -rg, 0);
             int endg = FFMIN3(255-bg, 255-rg, 255);
             uint32_t y = (uint32_t)(( 299*rg + 1000*startg + 114*bg)/1000);
-            c = bg + (rg<<16) + 0x010101 * startg;
+            c = bg + rg * (1 << 16) + 0x010101 * startg;
             for (g = startg; g <= endg; g++) {
                 hqx->rgbtoyuv[c] = ((y++) << 16) + (u << 8) + v;
                 c+= 0x010101;