diff mbox series

[FFmpeg-devel,3/5] avcodec/proresenc_kostya: do not write into alpha reserved bitfields

Message ID 20231211190908.1882569-4-u@pkh.me
State New
Headers show
Series [FFmpeg-devel,1/5] avcodec/proresenc_anatoliy: use a compatible bitstream version | expand

Checks

Context Check Description
andriy/configure_x86 warning Failed to apply patch
yinshiyou/configure_loongarch64 warning Failed to apply patch

Commit Message

Clément Bœsch Dec. 11, 2023, 7:06 p.m. UTC
This byte represents 4 reserved bits followed by 4 alpha_channel_type bits.

alpha_channel_type currently has 3 differents defined values: 0 (no
alpha), 1 (8b alpha), and 2 (16b alpha), all the other values are
reserved. This part is correctly written (alpha_bits>>3 does the correct
thing), but the 4 initial bits are reserved.
---
 libavcodec/proresenc_kostya.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Michael Niedermayer Dec. 12, 2023, 11:13 p.m. UTC | #1
On Mon, Dec 11, 2023 at 08:06:53PM +0100, Clément Bœsch wrote:
> This byte represents 4 reserved bits followed by 4 alpha_channel_type bits.
> 
> alpha_channel_type currently has 3 differents defined values: 0 (no
> alpha), 1 (8b alpha), and 2 (16b alpha), all the other values are
> reserved. This part is correctly written (alpha_bits>>3 does the correct
> thing), but the 4 initial bits are reserved.
> ---
>  libavcodec/proresenc_kostya.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

This breaks fate (not just changing the value but actually breaking it)

--- ./tests/ref/vsynth/vsynth1-prores_444_int	2023-12-02 00:46:44.277042842 +0100
+++ tests/data/fate/vsynth1-prores_444_int	2023-12-13 00:11:22.144115839 +0100
@@ -1,4 +1,2 @@
-1d5e484fa8ca08781ef2ed8428963d12 *tests/data/fate/vsynth1-prores_444_int.mov
+d75a2ba07d0642340d040edab6cbb3bb *tests/data/fate/vsynth1-prores_444_int.mov
 9940947 tests/data/fate/vsynth1-prores_444_int.mov
-732ceeb6887524e0aee98762fe50578b *tests/data/fate/vsynth1-prores_444_int.out.rawvideo
-stddev:    2.83 PSNR: 39.08 MAXDIFF:   45 bytes:  7603200/  7603200

thx

[...]
Michael Niedermayer Dec. 12, 2023, 11:16 p.m. UTC | #2
On Wed, Dec 13, 2023 at 12:13:36AM +0100, Michael Niedermayer wrote:
> On Mon, Dec 11, 2023 at 08:06:53PM +0100, Clément Bœsch wrote:
> > This byte represents 4 reserved bits followed by 4 alpha_channel_type bits.
> > 
> > alpha_channel_type currently has 3 differents defined values: 0 (no
> > alpha), 1 (8b alpha), and 2 (16b alpha), all the other values are
> > reserved. This part is correctly written (alpha_bits>>3 does the correct
> > thing), but the 4 initial bits are reserved.
> > ---
> >  libavcodec/proresenc_kostya.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> This breaks fate (not just changing the value but actually breaking it)

This is about the next patch "avcodec/proresenc_anatoliy: do not write into alpha reserved bitfields"
I used a too short search string for the subject i replied to ...
(so issue is real but its the next patch)


> 
> --- ./tests/ref/vsynth/vsynth1-prores_444_int	2023-12-02 00:46:44.277042842 +0100
> +++ tests/data/fate/vsynth1-prores_444_int	2023-12-13 00:11:22.144115839 +0100
> @@ -1,4 +1,2 @@
> -1d5e484fa8ca08781ef2ed8428963d12 *tests/data/fate/vsynth1-prores_444_int.mov
> +d75a2ba07d0642340d040edab6cbb3bb *tests/data/fate/vsynth1-prores_444_int.mov
>  9940947 tests/data/fate/vsynth1-prores_444_int.mov
> -732ceeb6887524e0aee98762fe50578b *tests/data/fate/vsynth1-prores_444_int.out.rawvideo
> -stddev:    2.83 PSNR: 39.08 MAXDIFF:   45 bytes:  7603200/  7603200

[...]
Clément Bœsch Jan. 7, 2024, 6:20 p.m. UTC | #3
On Wed, Dec 13, 2023 at 12:16:56AM +0100, Michael Niedermayer wrote:
> On Wed, Dec 13, 2023 at 12:13:36AM +0100, Michael Niedermayer wrote:
> > On Mon, Dec 11, 2023 at 08:06:53PM +0100, Clément Bœsch wrote:
> > > This byte represents 4 reserved bits followed by 4 alpha_channel_type bits.
> > > 
> > > alpha_channel_type currently has 3 differents defined values: 0 (no
> > > alpha), 1 (8b alpha), and 2 (16b alpha), all the other values are
> > > reserved. This part is correctly written (alpha_bits>>3 does the correct
> > > thing), but the 4 initial bits are reserved.
> > > ---
> > >  libavcodec/proresenc_kostya.c | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > This breaks fate (not just changing the value but actually breaking it)
> 
> This is about the next patch "avcodec/proresenc_anatoliy: do not write into alpha reserved bitfields"
> I used a too short search string for the subject i replied to ...
> (so issue is real but its the next patch)
> 

Just resent a patchset addressing the FATE issue for all the commits
individually (sorry, I only tested with fate-prores initially)

I also fixed "do not write into alpha reserved bitfields" which wasn't
writing the alpha byte unconditionally.

Sorry about the missing v2 in the patchset btw, forgot about it.

[...]
diff mbox series

Patch

diff --git a/libavcodec/proresenc_kostya.c b/libavcodec/proresenc_kostya.c
index 7e660a3708..a71452466e 100644
--- a/libavcodec/proresenc_kostya.c
+++ b/libavcodec/proresenc_kostya.c
@@ -1009,7 +1009,7 @@  static int encode_frame(AVCodecContext *avctx, AVPacket *pkt,
     bytestream_put_byte  (&buf, pic->color_primaries);
     bytestream_put_byte  (&buf, pic->color_trc);
     bytestream_put_byte  (&buf, pic->colorspace);
-    bytestream_put_byte  (&buf, 0x40 | (ctx->alpha_bits >> 3));
+    bytestream_put_byte  (&buf, ctx->alpha_bits >> 3);
     bytestream_put_byte  (&buf, 0);             // reserved
     if (ctx->quant_sel != QUANT_MAT_DEFAULT) {
         bytestream_put_byte  (&buf, 0x03);      // matrix flags - both matrices are present