| Message ID | GV1P250MB073794032AB549EE757CCB2F8F549@GV1P250MB0737.EURP250.PROD.OUTLOOK.COM |
|---|---|
| State | Accepted |
| Commit | 98aaaf08b39fe088dc308c951a82124ec1d3b10a |
| Headers | show |
| Series | [FFmpeg-devel,1/5] avcodec/rl2: Remove wrong check | expand |
| Context | Check | Description |
|---|---|---|
| andriy/make_x86 | success | Make finished |
| andriy/make_fate_x86 | fail | Make fate failed |
Andreas Rheinhardt: > This check is intended to be avoid buffer overflows, > yet there are four problems with it: > 1. It has an in-built off-by-one error: len == out_end - out > is perfectly fine and nothing to worry about. > This off-by-one error led to the pixel in the lower-right corner > not being set properly for the back frame of the sample from > the rl2 FATE-test. This pixel is copied to every frame which > is the reason for the update to the reference file of said test. > With this patch, the output of the decoder matches the output > as captured from the reference decoder* (apart from the fact > that said reference somehow lacks the top part of the frame > (copied over from the background frame)). > 2. Given that the stride of the buffer may be different > from the width of the video (despite one pixel taking one byte), > there is a second check lateron making the first check redundant > (if one returns immediately; a simple break at the second check > is not sufficient, because it only exits the inner loop). > 3. The check is based around the assumption of the stride being > positive (it has this in common with the other check which > will be fixed in a future commit). > 4. Even after fixing the off-by-one error, the check in > question is still triggered by all the non-background frames > in the FATE sample as well as by A1100100.RL2. In all these > cases, they use len == 255 and val == 128. For videos with > background frame this just means "copy from the background > frame", which would be done anyway lateron.* Yet for videos > without it copying it is necessary to avoid leaving > uninitialized parts in the video. > > *: Available in https://samples.mplayerhq.hu/game-formats/voyeur-rl2/ > **: Due to this, the code that copies the rest from the > back frame is no longer executed for any of the samples > available on the sample server. Given that these are only > the files from the demo version of this game, I don't know > whether this code is executed for any file in existence or not. > > Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com> > --- > libavcodec/rl2.c | 5 +- > tests/ref/fate/rl2 | 216 ++++++++++++++++++++++----------------------- > 2 files changed, 109 insertions(+), 112 deletions(-) > > diff --git a/libavcodec/rl2.c b/libavcodec/rl2.c > index 5dedb96266..2464ad59ac 100644 > --- a/libavcodec/rl2.c > +++ b/libavcodec/rl2.c > @@ -91,9 +91,6 @@ static void rl2_rle_decode(Rl2Context *s, const uint8_t *in, int size, > break; > } > > - if (len >= out_end - out) > - break; > - > if (s->back_frame) > val |= 0x80; > else > @@ -106,7 +103,7 @@ static void rl2_rle_decode(Rl2Context *s, const uint8_t *in, int size, > out += stride_adj; > line_end += stride; > if (len >= out_end - out) > - break; > + return; > } > } > } > diff --git a/tests/ref/fate/rl2 b/tests/ref/fate/rl2 > index 9189822503..a3c6a75d05 100644 > --- a/tests/ref/fate/rl2 > +++ b/tests/ref/fate/rl2 > @@ -3,111 +3,111 @@ > #codec_id 0: rawvideo > #dimensions 0: 320x200 > #sar 0: 0/1 > -0, 0, 0, 1, 192000, 0x7112a667 > -0, 1, 1, 1, 192000, 0x6936abf3 > -0, 2, 2, 1, 192000, 0xb1f08981 > -0, 3, 3, 1, 192000, 0x4ce7fece > -0, 4, 4, 1, 192000, 0xf04decde > -0, 5, 5, 1, 192000, 0x47fef74b > -0, 6, 6, 1, 192000, 0x99b42ac2 > -0, 7, 7, 1, 192000, 0x3c4c419d > -0, 8, 8, 1, 192000, 0x66bf5588 > -0, 9, 9, 1, 192000, 0xe1de4725 > -0, 10, 10, 1, 192000, 0x348b2af9 > -0, 11, 11, 1, 192000, 0x1ce73e83 > -0, 12, 12, 1, 192000, 0xcdaa6e02 > -0, 13, 13, 1, 192000, 0x370dc2ce > -0, 14, 14, 1, 192000, 0x1e1e40fe > -0, 15, 15, 1, 192000, 0x491470a4 > -0, 16, 16, 1, 192000, 0x88c43e9a > -0, 17, 17, 1, 192000, 0x036f3f44 > -0, 18, 18, 1, 192000, 0xc8be7e25 > -0, 19, 19, 1, 192000, 0xbedb397d > -0, 20, 20, 1, 192000, 0x97c410f4 > -0, 21, 21, 1, 192000, 0xc2c8225d > -0, 22, 22, 1, 192000, 0xe396bccb > -0, 23, 23, 1, 192000, 0x7e89c24c > -0, 24, 24, 1, 192000, 0xb044954c > -0, 25, 25, 1, 192000, 0x335890dd > -0, 26, 26, 1, 192000, 0x58a457c0 > -0, 27, 27, 1, 192000, 0xeb0f4798 > -0, 28, 28, 1, 192000, 0x0bfc39a1 > -0, 29, 29, 1, 192000, 0x06a6905a > -0, 30, 30, 1, 192000, 0x5300c99b > -0, 31, 31, 1, 192000, 0x38f3f845 > -0, 32, 32, 1, 192000, 0x6afa3543 > -0, 33, 33, 1, 192000, 0x5106a8e0 > -0, 34, 34, 1, 192000, 0xc76f0dab > -0, 35, 35, 1, 192000, 0x8efa6939 > -0, 36, 36, 1, 192000, 0x64ea23d7 > -0, 37, 37, 1, 192000, 0x421a2817 > -0, 38, 38, 1, 192000, 0xc854fa18 > -0, 39, 39, 1, 192000, 0x4a10d59c > -0, 40, 40, 1, 192000, 0x72637829 > -0, 41, 41, 1, 192000, 0xdbbe2796 > -0, 42, 42, 1, 192000, 0xff742e6b > -0, 43, 43, 1, 192000, 0xf94b9346 > -0, 44, 44, 1, 192000, 0xc90ea53c > -0, 45, 45, 1, 192000, 0x177483bb > -0, 46, 46, 1, 192000, 0x3510369c > -0, 47, 47, 1, 192000, 0x501034bf > -0, 48, 48, 1, 192000, 0x315c744b > -0, 49, 49, 1, 192000, 0xdb5048ae > -0, 50, 50, 1, 192000, 0x09a86221 > -0, 51, 51, 1, 192000, 0xb9c9568a > -0, 52, 52, 1, 192000, 0x5eee665b > -0, 53, 53, 1, 192000, 0xdef85517 > -0, 54, 54, 1, 192000, 0x7896b5ad > -0, 55, 55, 1, 192000, 0x19fbad39 > -0, 56, 56, 1, 192000, 0x63358748 > -0, 57, 57, 1, 192000, 0xca0196a3 > -0, 58, 58, 1, 192000, 0x0b321da3 > -0, 59, 59, 1, 192000, 0xa07af07e > -0, 60, 60, 1, 192000, 0x21f9310c > -0, 61, 61, 1, 192000, 0x62d59874 > -0, 62, 62, 1, 192000, 0x021227b4 > -0, 63, 63, 1, 192000, 0x01dac0c2 > -0, 64, 64, 1, 192000, 0xf3c33a74 > -0, 65, 65, 1, 192000, 0xeebe83b7 > -0, 66, 66, 1, 192000, 0x9ec77f97 > -0, 67, 67, 1, 192000, 0xc91c2e37 > -0, 68, 68, 1, 192000, 0x7b58751d > -0, 69, 69, 1, 192000, 0xb178dfbb > -0, 70, 70, 1, 192000, 0x2a63b5be > -0, 71, 71, 1, 192000, 0x44a407ac > -0, 72, 72, 1, 192000, 0x9a8e17dd > -0, 73, 73, 1, 192000, 0x5546f4c8 > -0, 74, 74, 1, 192000, 0xeda94586 > -0, 75, 75, 1, 192000, 0x941dfa6b > -0, 76, 76, 1, 192000, 0x9772301d > -0, 77, 77, 1, 192000, 0x8be16b16 > -0, 78, 78, 1, 192000, 0x26dd4496 > -0, 79, 79, 1, 192000, 0x27823797 > -0, 80, 80, 1, 192000, 0xacc914f7 > -0, 81, 81, 1, 192000, 0xcbb72c9b > -0, 82, 82, 1, 192000, 0x4bd3391c > -0, 83, 83, 1, 192000, 0x4e6adbfe > -0, 84, 84, 1, 192000, 0x7a791c75 > -0, 85, 85, 1, 192000, 0xc4f59c94 > -0, 86, 86, 1, 192000, 0xc4f59c94 > -0, 87, 87, 1, 192000, 0x984a4a0b > -0, 88, 88, 1, 192000, 0x3353f31f > -0, 89, 89, 1, 192000, 0xa9d4dc5a > -0, 90, 90, 1, 192000, 0xb33425d0 > -0, 91, 91, 1, 192000, 0x546d768a > -0, 92, 92, 1, 192000, 0xfefbe5c9 > -0, 93, 93, 1, 192000, 0xbd6be61d > -0, 94, 94, 1, 192000, 0xf5792731 > -0, 95, 95, 1, 192000, 0xccde0582 > -0, 96, 96, 1, 192000, 0x857d58ee > -0, 97, 97, 1, 192000, 0xe914ce48 > -0, 98, 98, 1, 192000, 0x1f736298 > -0, 99, 99, 1, 192000, 0xec0b4846 > -0, 100, 100, 1, 192000, 0xe1422624 > -0, 101, 101, 1, 192000, 0x56e2b0e0 > -0, 102, 102, 1, 192000, 0xc4190640 > -0, 103, 103, 1, 192000, 0x7c461977 > -0, 104, 104, 1, 192000, 0x34b1d5e8 > -0, 105, 105, 1, 192000, 0xbdc70f7a > -0, 106, 106, 1, 192000, 0xb466cd8d > -0, 107, 107, 1, 192000, 0x0e86a04c > +0, 0, 0, 1, 192000, 0x7324a772 > +0, 1, 1, 1, 192000, 0x6b48acfe > +0, 2, 2, 1, 192000, 0xb4028a8c > +0, 3, 3, 1, 192000, 0x4ef9ffd9 > +0, 4, 4, 1, 192000, 0xf25fede9 > +0, 5, 5, 1, 192000, 0x4a10f856 > +0, 6, 6, 1, 192000, 0x9bc62bcd > +0, 7, 7, 1, 192000, 0x3e5e42a8 > +0, 8, 8, 1, 192000, 0x68d15693 > +0, 9, 9, 1, 192000, 0xe3f04830 > +0, 10, 10, 1, 192000, 0x369d2c04 > +0, 11, 11, 1, 192000, 0x1ef93f8e > +0, 12, 12, 1, 192000, 0xcfbc6f0d > +0, 13, 13, 1, 192000, 0x391fc3d9 > +0, 14, 14, 1, 192000, 0x20304209 > +0, 15, 15, 1, 192000, 0x4b2671af > +0, 16, 16, 1, 192000, 0x8ad63fa5 > +0, 17, 17, 1, 192000, 0x0581404f > +0, 18, 18, 1, 192000, 0xcad07f30 > +0, 19, 19, 1, 192000, 0xc0ed3a88 > +0, 20, 20, 1, 192000, 0x99d611ff > +0, 21, 21, 1, 192000, 0xc4da2368 > +0, 22, 22, 1, 192000, 0xe5a8bdd6 > +0, 23, 23, 1, 192000, 0x809bc357 > +0, 24, 24, 1, 192000, 0xb2569657 > +0, 25, 25, 1, 192000, 0x356a91e8 > +0, 26, 26, 1, 192000, 0x5ab658cb > +0, 27, 27, 1, 192000, 0xed2148a3 > +0, 28, 28, 1, 192000, 0x0e0e3aac > +0, 29, 29, 1, 192000, 0x08b89165 > +0, 30, 30, 1, 192000, 0x5512caa6 > +0, 31, 31, 1, 192000, 0x3b05f950 > +0, 32, 32, 1, 192000, 0x6d0c364e > +0, 33, 33, 1, 192000, 0x5318a9eb > +0, 34, 34, 1, 192000, 0xc9810eb6 > +0, 35, 35, 1, 192000, 0x910c6a44 > +0, 36, 36, 1, 192000, 0x66fc24e2 > +0, 37, 37, 1, 192000, 0x442c2922 > +0, 38, 38, 1, 192000, 0xca66fb23 > +0, 39, 39, 1, 192000, 0x4c22d6a7 > +0, 40, 40, 1, 192000, 0x74757934 > +0, 41, 41, 1, 192000, 0xddd028a1 > +0, 42, 42, 1, 192000, 0x01952f76 > +0, 43, 43, 1, 192000, 0xfb5d9451 > +0, 44, 44, 1, 192000, 0xcb20a647 > +0, 45, 45, 1, 192000, 0x198684c6 > +0, 46, 46, 1, 192000, 0x372237a7 > +0, 47, 47, 1, 192000, 0x522235ca > +0, 48, 48, 1, 192000, 0x336e7556 > +0, 49, 49, 1, 192000, 0xdd6249b9 > +0, 50, 50, 1, 192000, 0x0bba632c > +0, 51, 51, 1, 192000, 0xbbdb5795 > +0, 52, 52, 1, 192000, 0x61006766 > +0, 53, 53, 1, 192000, 0xe10a5622 > +0, 54, 54, 1, 192000, 0x7aa8b6b8 > +0, 55, 55, 1, 192000, 0x1c0dae44 > +0, 56, 56, 1, 192000, 0x65478853 > +0, 57, 57, 1, 192000, 0xcc1397ae > +0, 58, 58, 1, 192000, 0x0d441eae > +0, 59, 59, 1, 192000, 0xa28cf189 > +0, 60, 60, 1, 192000, 0x240b3217 > +0, 61, 61, 1, 192000, 0x64e7997f > +0, 62, 62, 1, 192000, 0x042428bf > +0, 63, 63, 1, 192000, 0x03ecc1cd > +0, 64, 64, 1, 192000, 0xf5d53b7f > +0, 65, 65, 1, 192000, 0xf0d084c2 > +0, 66, 66, 1, 192000, 0xa0d980a2 > +0, 67, 67, 1, 192000, 0xcb2e2f42 > +0, 68, 68, 1, 192000, 0x7d6a7628 > +0, 69, 69, 1, 192000, 0xb38ae0c6 > +0, 70, 70, 1, 192000, 0x2c75b6c9 > +0, 71, 71, 1, 192000, 0x46b608b7 > +0, 72, 72, 1, 192000, 0x9ca018e8 > +0, 73, 73, 1, 192000, 0x5758f5d3 > +0, 74, 74, 1, 192000, 0xefbb4691 > +0, 75, 75, 1, 192000, 0x962ffb76 > +0, 76, 76, 1, 192000, 0x99843128 > +0, 77, 77, 1, 192000, 0x8df36c21 > +0, 78, 78, 1, 192000, 0x28ef45a1 > +0, 79, 79, 1, 192000, 0x299438a2 > +0, 80, 80, 1, 192000, 0xaedb1602 > +0, 81, 81, 1, 192000, 0xcdc92da6 > +0, 82, 82, 1, 192000, 0x4de53a27 > +0, 83, 83, 1, 192000, 0x507cdd09 > +0, 84, 84, 1, 192000, 0x7c8b1d80 > +0, 85, 85, 1, 192000, 0xc7079d9f > +0, 86, 86, 1, 192000, 0xc7079d9f > +0, 87, 87, 1, 192000, 0x9a5c4b16 > +0, 88, 88, 1, 192000, 0x3565f42a > +0, 89, 89, 1, 192000, 0xabe6dd65 > +0, 90, 90, 1, 192000, 0xb54626db > +0, 91, 91, 1, 192000, 0x567f7795 > +0, 92, 92, 1, 192000, 0x011ce6d4 > +0, 93, 93, 1, 192000, 0xbf7de728 > +0, 94, 94, 1, 192000, 0xf78b283c > +0, 95, 95, 1, 192000, 0xcef0068d > +0, 96, 96, 1, 192000, 0x878f59f9 > +0, 97, 97, 1, 192000, 0xeb26cf53 > +0, 98, 98, 1, 192000, 0x218563a3 > +0, 99, 99, 1, 192000, 0xee1d4951 > +0, 100, 100, 1, 192000, 0xe354272f > +0, 101, 101, 1, 192000, 0x58f4b1eb > +0, 102, 102, 1, 192000, 0xc62b074b > +0, 103, 103, 1, 192000, 0x7e581a82 > +0, 104, 104, 1, 192000, 0x36c3d6f3 > +0, 105, 105, 1, 192000, 0xbfd91085 > +0, 106, 106, 1, 192000, 0xb678ce98 > +0, 107, 107, 1, 192000, 0x1098a157 Will apply this patchset tonight unless there are objections. - Andreas
diff --git a/libavcodec/rl2.c b/libavcodec/rl2.c index 5dedb96266..2464ad59ac 100644 --- a/libavcodec/rl2.c +++ b/libavcodec/rl2.c @@ -91,9 +91,6 @@ static void rl2_rle_decode(Rl2Context *s, const uint8_t *in, int size, break; } - if (len >= out_end - out) - break; - if (s->back_frame) val |= 0x80; else @@ -106,7 +103,7 @@ static void rl2_rle_decode(Rl2Context *s, const uint8_t *in, int size, out += stride_adj; line_end += stride; if (len >= out_end - out) - break; + return; } } } diff --git a/tests/ref/fate/rl2 b/tests/ref/fate/rl2 index 9189822503..a3c6a75d05 100644 --- a/tests/ref/fate/rl2 +++ b/tests/ref/fate/rl2 @@ -3,111 +3,111 @@ #codec_id 0: rawvideo #dimensions 0: 320x200 #sar 0: 0/1 -0, 0, 0, 1, 192000, 0x7112a667 -0, 1, 1, 1, 192000, 0x6936abf3 -0, 2, 2, 1, 192000, 0xb1f08981 -0, 3, 3, 1, 192000, 0x4ce7fece -0, 4, 4, 1, 192000, 0xf04decde -0, 5, 5, 1, 192000, 0x47fef74b -0, 6, 6, 1, 192000, 0x99b42ac2 -0, 7, 7, 1, 192000, 0x3c4c419d -0, 8, 8, 1, 192000, 0x66bf5588 -0, 9, 9, 1, 192000, 0xe1de4725 -0, 10, 10, 1, 192000, 0x348b2af9 -0, 11, 11, 1, 192000, 0x1ce73e83 -0, 12, 12, 1, 192000, 0xcdaa6e02 -0, 13, 13, 1, 192000, 0x370dc2ce -0, 14, 14, 1, 192000, 0x1e1e40fe -0, 15, 15, 1, 192000, 0x491470a4 -0, 16, 16, 1, 192000, 0x88c43e9a -0, 17, 17, 1, 192000, 0x036f3f44 -0, 18, 18, 1, 192000, 0xc8be7e25 -0, 19, 19, 1, 192000, 0xbedb397d -0, 20, 20, 1, 192000, 0x97c410f4 -0, 21, 21, 1, 192000, 0xc2c8225d -0, 22, 22, 1, 192000, 0xe396bccb -0, 23, 23, 1, 192000, 0x7e89c24c -0, 24, 24, 1, 192000, 0xb044954c -0, 25, 25, 1, 192000, 0x335890dd -0, 26, 26, 1, 192000, 0x58a457c0 -0, 27, 27, 1, 192000, 0xeb0f4798 -0, 28, 28, 1, 192000, 0x0bfc39a1 -0, 29, 29, 1, 192000, 0x06a6905a -0, 30, 30, 1, 192000, 0x5300c99b -0, 31, 31, 1, 192000, 0x38f3f845 -0, 32, 32, 1, 192000, 0x6afa3543 -0, 33, 33, 1, 192000, 0x5106a8e0 -0, 34, 34, 1, 192000, 0xc76f0dab -0, 35, 35, 1, 192000, 0x8efa6939 -0, 36, 36, 1, 192000, 0x64ea23d7 -0, 37, 37, 1, 192000, 0x421a2817 -0, 38, 38, 1, 192000, 0xc854fa18 -0, 39, 39, 1, 192000, 0x4a10d59c -0, 40, 40, 1, 192000, 0x72637829 -0, 41, 41, 1, 192000, 0xdbbe2796 -0, 42, 42, 1, 192000, 0xff742e6b -0, 43, 43, 1, 192000, 0xf94b9346 -0, 44, 44, 1, 192000, 0xc90ea53c -0, 45, 45, 1, 192000, 0x177483bb -0, 46, 46, 1, 192000, 0x3510369c -0, 47, 47, 1, 192000, 0x501034bf -0, 48, 48, 1, 192000, 0x315c744b -0, 49, 49, 1, 192000, 0xdb5048ae -0, 50, 50, 1, 192000, 0x09a86221 -0, 51, 51, 1, 192000, 0xb9c9568a -0, 52, 52, 1, 192000, 0x5eee665b -0, 53, 53, 1, 192000, 0xdef85517 -0, 54, 54, 1, 192000, 0x7896b5ad -0, 55, 55, 1, 192000, 0x19fbad39 -0, 56, 56, 1, 192000, 0x63358748 -0, 57, 57, 1, 192000, 0xca0196a3 -0, 58, 58, 1, 192000, 0x0b321da3 -0, 59, 59, 1, 192000, 0xa07af07e -0, 60, 60, 1, 192000, 0x21f9310c -0, 61, 61, 1, 192000, 0x62d59874 -0, 62, 62, 1, 192000, 0x021227b4 -0, 63, 63, 1, 192000, 0x01dac0c2 -0, 64, 64, 1, 192000, 0xf3c33a74 -0, 65, 65, 1, 192000, 0xeebe83b7 -0, 66, 66, 1, 192000, 0x9ec77f97 -0, 67, 67, 1, 192000, 0xc91c2e37 -0, 68, 68, 1, 192000, 0x7b58751d -0, 69, 69, 1, 192000, 0xb178dfbb -0, 70, 70, 1, 192000, 0x2a63b5be -0, 71, 71, 1, 192000, 0x44a407ac -0, 72, 72, 1, 192000, 0x9a8e17dd -0, 73, 73, 1, 192000, 0x5546f4c8 -0, 74, 74, 1, 192000, 0xeda94586 -0, 75, 75, 1, 192000, 0x941dfa6b -0, 76, 76, 1, 192000, 0x9772301d -0, 77, 77, 1, 192000, 0x8be16b16 -0, 78, 78, 1, 192000, 0x26dd4496 -0, 79, 79, 1, 192000, 0x27823797 -0, 80, 80, 1, 192000, 0xacc914f7 -0, 81, 81, 1, 192000, 0xcbb72c9b -0, 82, 82, 1, 192000, 0x4bd3391c -0, 83, 83, 1, 192000, 0x4e6adbfe -0, 84, 84, 1, 192000, 0x7a791c75 -0, 85, 85, 1, 192000, 0xc4f59c94 -0, 86, 86, 1, 192000, 0xc4f59c94 -0, 87, 87, 1, 192000, 0x984a4a0b -0, 88, 88, 1, 192000, 0x3353f31f -0, 89, 89, 1, 192000, 0xa9d4dc5a -0, 90, 90, 1, 192000, 0xb33425d0 -0, 91, 91, 1, 192000, 0x546d768a -0, 92, 92, 1, 192000, 0xfefbe5c9 -0, 93, 93, 1, 192000, 0xbd6be61d -0, 94, 94, 1, 192000, 0xf5792731 -0, 95, 95, 1, 192000, 0xccde0582 -0, 96, 96, 1, 192000, 0x857d58ee -0, 97, 97, 1, 192000, 0xe914ce48 -0, 98, 98, 1, 192000, 0x1f736298 -0, 99, 99, 1, 192000, 0xec0b4846 -0, 100, 100, 1, 192000, 0xe1422624 -0, 101, 101, 1, 192000, 0x56e2b0e0 -0, 102, 102, 1, 192000, 0xc4190640 -0, 103, 103, 1, 192000, 0x7c461977 -0, 104, 104, 1, 192000, 0x34b1d5e8 -0, 105, 105, 1, 192000, 0xbdc70f7a -0, 106, 106, 1, 192000, 0xb466cd8d -0, 107, 107, 1, 192000, 0x0e86a04c +0, 0, 0, 1, 192000, 0x7324a772 +0, 1, 1, 1, 192000, 0x6b48acfe +0, 2, 2, 1, 192000, 0xb4028a8c +0, 3, 3, 1, 192000, 0x4ef9ffd9 +0, 4, 4, 1, 192000, 0xf25fede9 +0, 5, 5, 1, 192000, 0x4a10f856 +0, 6, 6, 1, 192000, 0x9bc62bcd +0, 7, 7, 1, 192000, 0x3e5e42a8 +0, 8, 8, 1, 192000, 0x68d15693 +0, 9, 9, 1, 192000, 0xe3f04830 +0, 10, 10, 1, 192000, 0x369d2c04 +0, 11, 11, 1, 192000, 0x1ef93f8e +0, 12, 12, 1, 192000, 0xcfbc6f0d +0, 13, 13, 1, 192000, 0x391fc3d9 +0, 14, 14, 1, 192000, 0x20304209 +0, 15, 15, 1, 192000, 0x4b2671af +0, 16, 16, 1, 192000, 0x8ad63fa5 +0, 17, 17, 1, 192000, 0x0581404f +0, 18, 18, 1, 192000, 0xcad07f30 +0, 19, 19, 1, 192000, 0xc0ed3a88 +0, 20, 20, 1, 192000, 0x99d611ff +0, 21, 21, 1, 192000, 0xc4da2368 +0, 22, 22, 1, 192000, 0xe5a8bdd6 +0, 23, 23, 1, 192000, 0x809bc357 +0, 24, 24, 1, 192000, 0xb2569657 +0, 25, 25, 1, 192000, 0x356a91e8 +0, 26, 26, 1, 192000, 0x5ab658cb +0, 27, 27, 1, 192000, 0xed2148a3 +0, 28, 28, 1, 192000, 0x0e0e3aac +0, 29, 29, 1, 192000, 0x08b89165 +0, 30, 30, 1, 192000, 0x5512caa6 +0, 31, 31, 1, 192000, 0x3b05f950 +0, 32, 32, 1, 192000, 0x6d0c364e +0, 33, 33, 1, 192000, 0x5318a9eb +0, 34, 34, 1, 192000, 0xc9810eb6 +0, 35, 35, 1, 192000, 0x910c6a44 +0, 36, 36, 1, 192000, 0x66fc24e2 +0, 37, 37, 1, 192000, 0x442c2922 +0, 38, 38, 1, 192000, 0xca66fb23 +0, 39, 39, 1, 192000, 0x4c22d6a7 +0, 40, 40, 1, 192000, 0x74757934 +0, 41, 41, 1, 192000, 0xddd028a1 +0, 42, 42, 1, 192000, 0x01952f76 +0, 43, 43, 1, 192000, 0xfb5d9451 +0, 44, 44, 1, 192000, 0xcb20a647 +0, 45, 45, 1, 192000, 0x198684c6 +0, 46, 46, 1, 192000, 0x372237a7 +0, 47, 47, 1, 192000, 0x522235ca +0, 48, 48, 1, 192000, 0x336e7556 +0, 49, 49, 1, 192000, 0xdd6249b9 +0, 50, 50, 1, 192000, 0x0bba632c +0, 51, 51, 1, 192000, 0xbbdb5795 +0, 52, 52, 1, 192000, 0x61006766 +0, 53, 53, 1, 192000, 0xe10a5622 +0, 54, 54, 1, 192000, 0x7aa8b6b8 +0, 55, 55, 1, 192000, 0x1c0dae44 +0, 56, 56, 1, 192000, 0x65478853 +0, 57, 57, 1, 192000, 0xcc1397ae +0, 58, 58, 1, 192000, 0x0d441eae +0, 59, 59, 1, 192000, 0xa28cf189 +0, 60, 60, 1, 192000, 0x240b3217 +0, 61, 61, 1, 192000, 0x64e7997f +0, 62, 62, 1, 192000, 0x042428bf +0, 63, 63, 1, 192000, 0x03ecc1cd +0, 64, 64, 1, 192000, 0xf5d53b7f +0, 65, 65, 1, 192000, 0xf0d084c2 +0, 66, 66, 1, 192000, 0xa0d980a2 +0, 67, 67, 1, 192000, 0xcb2e2f42 +0, 68, 68, 1, 192000, 0x7d6a7628 +0, 69, 69, 1, 192000, 0xb38ae0c6 +0, 70, 70, 1, 192000, 0x2c75b6c9 +0, 71, 71, 1, 192000, 0x46b608b7 +0, 72, 72, 1, 192000, 0x9ca018e8 +0, 73, 73, 1, 192000, 0x5758f5d3 +0, 74, 74, 1, 192000, 0xefbb4691 +0, 75, 75, 1, 192000, 0x962ffb76 +0, 76, 76, 1, 192000, 0x99843128 +0, 77, 77, 1, 192000, 0x8df36c21 +0, 78, 78, 1, 192000, 0x28ef45a1 +0, 79, 79, 1, 192000, 0x299438a2 +0, 80, 80, 1, 192000, 0xaedb1602 +0, 81, 81, 1, 192000, 0xcdc92da6 +0, 82, 82, 1, 192000, 0x4de53a27 +0, 83, 83, 1, 192000, 0x507cdd09 +0, 84, 84, 1, 192000, 0x7c8b1d80 +0, 85, 85, 1, 192000, 0xc7079d9f +0, 86, 86, 1, 192000, 0xc7079d9f +0, 87, 87, 1, 192000, 0x9a5c4b16 +0, 88, 88, 1, 192000, 0x3565f42a +0, 89, 89, 1, 192000, 0xabe6dd65 +0, 90, 90, 1, 192000, 0xb54626db +0, 91, 91, 1, 192000, 0x567f7795 +0, 92, 92, 1, 192000, 0x011ce6d4 +0, 93, 93, 1, 192000, 0xbf7de728 +0, 94, 94, 1, 192000, 0xf78b283c +0, 95, 95, 1, 192000, 0xcef0068d +0, 96, 96, 1, 192000, 0x878f59f9 +0, 97, 97, 1, 192000, 0xeb26cf53 +0, 98, 98, 1, 192000, 0x218563a3 +0, 99, 99, 1, 192000, 0xee1d4951 +0, 100, 100, 1, 192000, 0xe354272f +0, 101, 101, 1, 192000, 0x58f4b1eb +0, 102, 102, 1, 192000, 0xc62b074b +0, 103, 103, 1, 192000, 0x7e581a82 +0, 104, 104, 1, 192000, 0x36c3d6f3 +0, 105, 105, 1, 192000, 0xbfd91085 +0, 106, 106, 1, 192000, 0xb678ce98 +0, 107, 107, 1, 192000, 0x1098a157
This check is intended to be avoid buffer overflows, yet there are four problems with it: 1. It has an in-built off-by-one error: len == out_end - out is perfectly fine and nothing to worry about. This off-by-one error led to the pixel in the lower-right corner not being set properly for the back frame of the sample from the rl2 FATE-test. This pixel is copied to every frame which is the reason for the update to the reference file of said test. With this patch, the output of the decoder matches the output as captured from the reference decoder* (apart from the fact that said reference somehow lacks the top part of the frame (copied over from the background frame)). 2. Given that the stride of the buffer may be different from the width of the video (despite one pixel taking one byte), there is a second check lateron making the first check redundant (if one returns immediately; a simple break at the second check is not sufficient, because it only exits the inner loop). 3. The check is based around the assumption of the stride being positive (it has this in common with the other check which will be fixed in a future commit). 4. Even after fixing the off-by-one error, the check in question is still triggered by all the non-background frames in the FATE sample as well as by A1100100.RL2. In all these cases, they use len == 255 and val == 128. For videos with background frame this just means "copy from the background frame", which would be done anyway lateron.* Yet for videos without it copying it is necessary to avoid leaving uninitialized parts in the video. *: Available in https://samples.mplayerhq.hu/game-formats/voyeur-rl2/ **: Due to this, the code that copies the rest from the back frame is no longer executed for any of the samples available on the sample server. Given that these are only the files from the demo version of this game, I don't know whether this code is executed for any file in existence or not. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com> --- libavcodec/rl2.c | 5 +- tests/ref/fate/rl2 | 216 ++++++++++++++++++++++----------------------- 2 files changed, 109 insertions(+), 112 deletions(-)