diff mbox series

[FFmpeg-devel,1/9] avcodec/vlc: merge lost 16bit end of array check

Message ID 20231022215113.3469-1-michael@niedermayer.cc
State Accepted
Commit 9b546a071764871dee20a690a31f97c8f33da769
Headers show
Series [FFmpeg-devel,1/9] avcodec/vlc: merge lost 16bit end of array check | expand

Checks

Context Check Description
yinshiyou/make_loongarch64 success Make finished
yinshiyou/make_fate_loongarch64 success Make fate finished
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished

Commit Message

Michael Niedermayer Oct. 22, 2023, 9:51 p.m. UTC 
Also cleanup related code

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/vlc.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

Comments

Paul B Mahol Oct. 22, 2023, 11:02 p.m. UTC | #1 
I have no time to review these hacks.

Make sure you do not break >8 bit support in utvideo and magicyuv decoders.
Michael Niedermayer Oct. 23, 2023, 4:05 p.m. UTC | #2 
On Mon, Oct 23, 2023 at 01:02:16AM +0200, Paul B Mahol wrote:
[...]

> Make sure you do not break >8 bit support in utvideo and magicyuv decoders.

Make sure the fate tests cover these cases

thx

[...]
Michael Niedermayer Oct. 26, 2023, 10:35 p.m. UTC | #3 
On Sun, Oct 22, 2023 at 11:51:05PM +0200, Michael Niedermayer wrote:
> Also cleanup related code
> 
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/vlc.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)

will apply patches 1 and 2

[...]
diff mbox series

Patch

diff --git a/libavcodec/vlc.c b/libavcodec/vlc.c
index e4bbf2945e1..4f62eddf0fb 100644
--- a/libavcodec/vlc.c
+++ b/libavcodec/vlc.c
@@ -358,8 +358,8 @@  static void add_level(VLC_MULTI_ELEM *table, const int nb_elems,
                       const int minlen, const int max,
                       unsigned* levelcnt, VLC_MULTI_ELEM *info)
 {
-    if (nb_elems > 256 && curlevel > 2)
-        return; // No room
+    int is16bit = nb_elems>256;
+    int max_symbols = VLC_MULTI_MAX_SYMBOLS >> is16bit;
     for (int i = num-1; i > max; i--) {
         for (int j = 0; j < 2; j++) {
             int newlimit, sym;
@@ -373,7 +373,7 @@  static void add_level(VLC_MULTI_ELEM *table, const int nb_elems,
             code = curcode + (buf[t].code >> curlen);
             newlimit = curlimit - l;
             l  += curlen;
-            if (nb_elems>256) AV_WN16(info->val+2*curlevel, sym);
+            if (is16bit) AV_WN16(info->val+2*curlevel, sym);
             else info->val[curlevel] = sym&0xFF;
 
             if (curlevel) { // let's not add single entries
@@ -386,7 +386,7 @@  static void add_level(VLC_MULTI_ELEM *table, const int nb_elems,
                 levelcnt[curlevel-1]++;
             }
 
-            if (curlevel+1 < VLC_MULTI_MAX_SYMBOLS && newlimit >= minlen) {
+            if (curlevel+1 < max_symbols && newlimit >= minlen) {
                 add_level(table, nb_elems, num, numbits, buf,
                           code, l, newlimit, curlevel+1,
                           minlen, max, levelcnt, info);