| Message ID | 20180312233622.13526-1-atomnuker@gmail.com |
|---|---|
| State | Accepted |
| Commit | 688060fbb7233d9212a92ce171e3b94784f95ca1 |
| Headers | show |
On 12 March 2018 at 23:36, Rostislav Pehlivanov <atomnuker@gmail.com> wrote: > On Windows machines, the UL suffix still means 32 bits. > The only parts that need 64 bits are (1ULL << (m + 32)) and > (t*qf + qf). Hence, use the proper ULL suffix for the former > and just increase the type of the qf constant for the latter. > No overflows can happen as long as these are done in 64 bits and > the quantization table doesn't change. > > Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com> > --- > libavcodec/vc2enc.c | 10 +++++----- > 1 file changed, 5 insertions(+), 5 deletions(-) > > diff --git a/libavcodec/vc2enc.c b/libavcodec/vc2enc.c > index 2e480ba8d0..d0101e01e4 100644 > --- a/libavcodec/vc2enc.c > +++ b/libavcodec/vc2enc.c > @@ -1163,14 +1163,14 @@ static av_cold int vc2_encode_init(AVCodecContext > *avctx) > goto alloc_fail; > > for (i = 0; i < 116; i++) { > - const uint32_t qf = ff_dirac_qscale_tab[i]; > - const int m = av_log2(qf); > - const uint32_t t = (1UL << (m + 32)) / qf; > - const uint32_t r = (t*qf + qf) & ((1UL << 32) - 1); > + const uint64_t qf = ff_dirac_qscale_tab[i]; > + const uint32_t m = av_log2(qf); > + const uint32_t t = (1ULL << (m + 32)) / qf; > + const uint32_t r = (t*qf + qf) & UINT32_MAX; > if (!(qf & (qf - 1))) { > s->qmagic_lut[i][0] = 0xFFFFFFFF; > s->qmagic_lut[i][1] = 0xFFFFFFFF; > - } else if (r <= 1UL << m) { > + } else if (r <= 1 << m) { > s->qmagic_lut[i][0] = t + 1; > s->qmagic_lut[i][1] = 0; > } else { > -- > 2.16.2 > > Fixes what its meant to fix, pushed
diff --git a/libavcodec/vc2enc.c b/libavcodec/vc2enc.c index 2e480ba8d0..d0101e01e4 100644 --- a/libavcodec/vc2enc.c +++ b/libavcodec/vc2enc.c @@ -1163,14 +1163,14 @@ static av_cold int vc2_encode_init(AVCodecContext *avctx) goto alloc_fail; for (i = 0; i < 116; i++) { - const uint32_t qf = ff_dirac_qscale_tab[i]; - const int m = av_log2(qf); - const uint32_t t = (1UL << (m + 32)) / qf; - const uint32_t r = (t*qf + qf) & ((1UL << 32) - 1); + const uint64_t qf = ff_dirac_qscale_tab[i]; + const uint32_t m = av_log2(qf); + const uint32_t t = (1ULL << (m + 32)) / qf; + const uint32_t r = (t*qf + qf) & UINT32_MAX; if (!(qf & (qf - 1))) { s->qmagic_lut[i][0] = 0xFFFFFFFF; s->qmagic_lut[i][1] = 0xFFFFFFFF; - } else if (r <= 1UL << m) { + } else if (r <= 1 << m) { s->qmagic_lut[i][0] = t + 1; s->qmagic_lut[i][1] = 0; } else {
On Windows machines, the UL suffix still means 32 bits. The only parts that need 64 bits are (1ULL << (m + 32)) and (t*qf + qf). Hence, use the proper ULL suffix for the former and just increase the type of the qf constant for the latter. No overflows can happen as long as these are done in 64 bits and the quantization table doesn't change. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com> --- libavcodec/vc2enc.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)