diff mbox series

[FFmpeg-devel,v2,1/9] cbs_av1: fix incorrect data type

Message ID 20210705022933.29068-1-fei.w.wang@intel.com
State Superseded
Headers show
Series [FFmpeg-devel,v2,1/9] cbs_av1: fix incorrect data type | expand

Checks

Context Check Description
andriy/x86_make success Make finished
andriy/x86_make_fate success Make fate finished
andriy/PPC64_make success Make finished
andriy/PPC64_make_fate success Make fate finished

Commit Message

Fei Wang July 5, 2021, 2:29 a.m. UTC 
Since order_hint_bits_minus_1 range is 0~7, cur_frame_hint can be
most 128. And similar return value for cbs_av1_get_relative_dist.
So if plus them and use int8_t for the result may lose its precision.

Signed-off-by: Fei Wang <fei.w.wang@intel.com>
---
v2 update:
1. use int16_t instead of int.
2. updated commit message.

 libavcodec/cbs_av1_syntax_template.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
diff mbox series

Patch

diff --git a/libavcodec/cbs_av1_syntax_template.c b/libavcodec/cbs_av1_syntax_template.c
index 6fe6e9a4f3..d98d3d42de 100644
--- a/libavcodec/cbs_av1_syntax_template.c
+++ b/libavcodec/cbs_av1_syntax_template.c
@@ -355,7 +355,7 @@  static int FUNC(set_frame_refs)(CodedBitstreamContext *ctx, RWContext *rw,
         AV1_REF_FRAME_ALTREF2, AV1_REF_FRAME_ALTREF
     };
     int8_t ref_frame_idx[AV1_REFS_PER_FRAME], used_frame[AV1_NUM_REF_FRAMES];
-    int8_t shifted_order_hints[AV1_NUM_REF_FRAMES];
+    int16_t shifted_order_hints[AV1_NUM_REF_FRAMES];
     int cur_frame_hint, latest_order_hint, earliest_order_hint, ref;
     int i, j;