| Message ID | 1566973334-19107-1-git-send-email-linjie.fu@intel.com |
|---|---|
| State | New |
| Headers | show |
> Am 28.08.2019 um 08:22 schrieb Linjie Fu <linjie.fu@intel.com>: > > Add support for packed 4:4:4 pixel format AYUV. > > It is the format that VAAPI/QSV uses when coping with 4:4:4 > surfaces. > Alpha channel will be set to default value for HEVC REXT > hw decode. What is the „default value“? Carl Eugen
> -----Original Message----- > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf > Of Carl Eugen Hoyos > Sent: Wednesday, August 28, 2019 14:51 > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel format > > > > > Am 28.08.2019 um 08:22 schrieb Linjie Fu <linjie.fu@intel.com>: > > > > Add support for packed 4:4:4 pixel format AYUV. > > > > It is the format that VAAPI/QSV uses when coping with 4:4:4 > > surfaces. > > > Alpha channel will be set to default value for HEVC REXT > > hw decode. > > What is the „default value“? > Default zero. (and it should be 255 for display)
Am Mi., 28. Aug. 2019 um 09:34 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>: > > > -----Original Message----- > > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf > > Of Carl Eugen Hoyos > > Sent: Wednesday, August 28, 2019 14:51 > > To: FFmpeg development discussions and patches <ffmpeg- > > devel@ffmpeg.org> > > Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel format > > > > > > > > > Am 28.08.2019 um 08:22 schrieb Linjie Fu <linjie.fu@intel.com>: > > > > > > Add support for packed 4:4:4 pixel format AYUV. > > > > > > It is the format that VAAPI/QSV uses when coping with 4:4:4 > > > surfaces. > > > > > Alpha channel will be set to default value for HEVC REXT > > > hw decode. > > > > What is the „default value“? > > > > Default zero. (and it should be 255 for display) If this is not a bug in the driver, there is no alpha channel that can be used. Rename it to 0YUV and remove the alpha usage. Carl Eugen
> -----Original Message----- > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf > Of Carl Eugen Hoyos > Sent: Thursday, August 29, 2019 05:17 > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel format > > Am Mi., 28. Aug. 2019 um 09:34 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>: > > > > > -----Original Message----- > > > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On > Behalf > > > Of Carl Eugen Hoyos > > > Sent: Wednesday, August 28, 2019 14:51 > > > To: FFmpeg development discussions and patches <ffmpeg- > > > devel@ffmpeg.org> > > > Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel > format > > > > > > > > > > > > > Am 28.08.2019 um 08:22 schrieb Linjie Fu <linjie.fu@intel.com>: > > > > > > > > Add support for packed 4:4:4 pixel format AYUV. > > > > > > > > It is the format that VAAPI/QSV uses when coping with 4:4:4 > > > > surfaces. > > > > > > > Alpha channel will be set to default value for HEVC REXT > > > > hw decode. > > > > > > What is the „default value“? > > > > > > > Default zero. (and it should be 255 for display) > > If this is not a bug in the driver, there is no alpha channel that can be used. > Rename it to 0YUV and remove the alpha usage. > From the pixel format level, adding AYUV format is for a more fundamental/common usage. It is valid no matter the alpha data is 0 or 255 or specific value to indicate the transparency. And the hardware decode result is an AYUV format with fixed alpha values, so it should still be an AYUV format IMHO. - linjie
Am 29.08.2019 um 03:34 schrieb Fu, Linjie <linjie.fu@intel.com>: >> -----Original Message----- >> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf >> Of Carl Eugen Hoyos >> Sent: Thursday, August 29, 2019 05:17 >> To: FFmpeg development discussions and patches <ffmpeg- >> devel@ffmpeg.org> >> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel format >> >>>> Am Mi., 28. Aug. 2019 um 09:34 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>: >>>> >>>> -----Original Message----- >>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On >> Behalf >>>> Of Carl Eugen Hoyos >>>> Sent: Wednesday, August 28, 2019 14:51 >>>> To: FFmpeg development discussions and patches <ffmpeg- >>>> devel@ffmpeg.org> >>>> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel >> format >>>> >>>> >>>> >>>>> Am 28.08.2019 um 08:22 schrieb Linjie Fu <linjie.fu@intel.com>: >>>>> >>>>> Add support for packed 4:4:4 pixel format AYUV. >>>>> >>>>> It is the format that VAAPI/QSV uses when coping with 4:4:4 >>>>> surfaces. >>>> >>>>> Alpha channel will be set to default value for HEVC REXT >>>>> hw decode. >>>> >>>> What is the „default value“? >>> >>> Default zero. (and it should be 255 for display) >> >> If this is not a bug in the driver, there is no alpha channel that can be used. >> Rename it to 0YUV and remove the alpha usage. > > From the pixel format level, adding AYUV format is for a more fundamental/common usage. Yes, but the new pixel format will only be accepted if it is needed for hardware compatibility. If the hardware only supports 0YUV, it cannot be used to argue in favour of a new AYUV format. Carl Eugen
On 8/29/19, Carl Eugen Hoyos <ceffmpeg@gmail.com> wrote: > > > Am 29.08.2019 um 03:34 schrieb Fu, Linjie <linjie.fu@intel.com>: > >>> -----Original Message----- >>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf >>> Of Carl Eugen Hoyos >>> Sent: Thursday, August 29, 2019 05:17 >>> To: FFmpeg development discussions and patches <ffmpeg- >>> devel@ffmpeg.org> >>> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel >>> format >>> >>>>> Am Mi., 28. Aug. 2019 um 09:34 Uhr schrieb Fu, Linjie >>>>> <linjie.fu@intel.com>: >>>>> >>>>> -----Original Message----- >>>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On >>> Behalf >>>>> Of Carl Eugen Hoyos >>>>> Sent: Wednesday, August 28, 2019 14:51 >>>>> To: FFmpeg development discussions and patches <ffmpeg- >>>>> devel@ffmpeg.org> >>>>> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel >>> format >>>>> >>>>> >>>>> >>>>>> Am 28.08.2019 um 08:22 schrieb Linjie Fu <linjie.fu@intel.com>: >>>>>> >>>>>> Add support for packed 4:4:4 pixel format AYUV. >>>>>> >>>>>> It is the format that VAAPI/QSV uses when coping with 4:4:4 >>>>>> surfaces. >>>>> >>>>>> Alpha channel will be set to default value for HEVC REXT >>>>>> hw decode. >>>>> >>>>> What is the „default value“? >>>> >>>> Default zero. (and it should be 255 for display) >>> >>> If this is not a bug in the driver, there is no alpha channel that can be >>> used. >>> Rename it to 0YUV and remove the alpha usage. >> >> From the pixel format level, adding AYUV format is for a more >> fundamental/common usage. > > Yes, but the new pixel format will only be accepted if it is needed for > hardware compatibility. > If the hardware only supports 0YUV, it cannot be used to argue in favour of > a new AYUV format. Nope. > > Carl Eugen > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On Thu, Aug 29, 2019 at 1:47 PM Carl Eugen Hoyos <ceffmpeg@gmail.com> wrote: > > > > Am 29.08.2019 um 03:34 schrieb Fu, Linjie <linjie.fu@intel.com>: > > >> -----Original Message----- > >> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf > >> Of Carl Eugen Hoyos > >> Sent: Thursday, August 29, 2019 05:17 > >> To: FFmpeg development discussions and patches <ffmpeg- > >> devel@ffmpeg.org> > >> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel format > >> > >>>> Am Mi., 28. Aug. 2019 um 09:34 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>: > >>>> > >>>> -----Original Message----- > >>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On > >> Behalf > >>>> Of Carl Eugen Hoyos > >>>> Sent: Wednesday, August 28, 2019 14:51 > >>>> To: FFmpeg development discussions and patches <ffmpeg- > >>>> devel@ffmpeg.org> > >>>> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel > >> format > >>>> > >>>> > >>>> > >>>>> Am 28.08.2019 um 08:22 schrieb Linjie Fu <linjie.fu@intel.com>: > >>>>> > >>>>> Add support for packed 4:4:4 pixel format AYUV. > >>>>> > >>>>> It is the format that VAAPI/QSV uses when coping with 4:4:4 > >>>>> surfaces. > >>>> > >>>>> Alpha channel will be set to default value for HEVC REXT > >>>>> hw decode. > >>>> > >>>> What is the „default value“? > >>> > >>> Default zero. (and it should be 255 for display) > >> > >> If this is not a bug in the driver, there is no alpha channel that can be used. > >> Rename it to 0YUV and remove the alpha usage. > > > > From the pixel format level, adding AYUV format is for a more fundamental/common usage. > > Yes, but the new pixel format will only be accepted if it is needed for hardware compatibility. > If the hardware only supports 0YUV, it cannot be used to argue in favour of a new AYUV format. > AYUV is a standard format used on Windows and documented by Microsoft as the recommended 4:4:4 8-bit format for all purposes. Its hardly a niche format. Its perfectly appropriate to support it. - Hendrik
> Am 29.08.2019 um 14:30 schrieb Hendrik Leppkes <h.leppkes@gmail.com>: > >> On Thu, Aug 29, 2019 at 1:47 PM Carl Eugen Hoyos <ceffmpeg@gmail.com> wrote: >> >> >> >> Am 29.08.2019 um 03:34 schrieb Fu, Linjie <linjie.fu@intel.com>: >> >>>> -----Original Message----- >>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf >>>> Of Carl Eugen Hoyos >>>> Sent: Thursday, August 29, 2019 05:17 >>>> To: FFmpeg development discussions and patches <ffmpeg- >>>> devel@ffmpeg.org> >>>> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel format >>>> >>>>>> Am Mi., 28. Aug. 2019 um 09:34 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>: >>>>>> >>>>>> -----Original Message----- >>>>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On >>>> Behalf >>>>>> Of Carl Eugen Hoyos >>>>>> Sent: Wednesday, August 28, 2019 14:51 >>>>>> To: FFmpeg development discussions and patches <ffmpeg- >>>>>> devel@ffmpeg.org> >>>>>> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel >>>> format >>>>>> >>>>>> >>>>>> >>>>>>> Am 28.08.2019 um 08:22 schrieb Linjie Fu <linjie.fu@intel.com>: >>>>>>> >>>>>>> Add support for packed 4:4:4 pixel format AYUV. >>>>>>> >>>>>>> It is the format that VAAPI/QSV uses when coping with 4:4:4 >>>>>>> surfaces. >>>>>> >>>>>>> Alpha channel will be set to default value for HEVC REXT >>>>>>> hw decode. >>>>>> >>>>>> What is the „default value“? >>>>> >>>>> Default zero. (and it should be 255 for display) >>>> >>>> If this is not a bug in the driver, there is no alpha channel that can be used. >>>> Rename it to 0YUV and remove the alpha usage. >>> >>> From the pixel format level, adding AYUV format is for a more fundamental/common usage. >> >> Yes, but the new pixel format will only be accepted if it is needed for hardware compatibility. >> If the hardware only supports 0YUV, it cannot be used to argue in favour of a new AYUV format. >> > > AYUV is a standard format used on Windows and documented by Microsoft > as the recommended 4:4:4 8-bit format for all purposes. Its hardly a > niche format. Its perfectly appropriate to support it. For which use case? Carl Eugen
> Am 29.08.2019 um 14:30 schrieb Hendrik Leppkes <h.leppkes@gmail.com>: > >> On Thu, Aug 29, 2019 at 1:47 PM Carl Eugen Hoyos <ceffmpeg@gmail.com> wrote: >> >> >> >> Am 29.08.2019 um 03:34 schrieb Fu, Linjie <linjie.fu@intel.com>: >> >>>> -----Original Message----- >>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf >>>> Of Carl Eugen Hoyos >>>> Sent: Thursday, August 29, 2019 05:17 >>>> To: FFmpeg development discussions and patches <ffmpeg- >>>> devel@ffmpeg.org> >>>> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel format >>>> >>>>>> Am Mi., 28. Aug. 2019 um 09:34 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>: >>>>>> >>>>>> -----Original Message----- >>>>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On >>>> Behalf >>>>>> Of Carl Eugen Hoyos >>>>>> Sent: Wednesday, August 28, 2019 14:51 >>>>>> To: FFmpeg development discussions and patches <ffmpeg- >>>>>> devel@ffmpeg.org> >>>>>> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel >>>> format >>>>>> >>>>>> >>>>>> >>>>>>> Am 28.08.2019 um 08:22 schrieb Linjie Fu <linjie.fu@intel.com>: >>>>>>> >>>>>>> Add support for packed 4:4:4 pixel format AYUV. >>>>>>> >>>>>>> It is the format that VAAPI/QSV uses when coping with 4:4:4 >>>>>>> surfaces. >>>>>> >>>>>>> Alpha channel will be set to default value for HEVC REXT >>>>>>> hw decode. >>>>>> >>>>>> What is the „default value“? >>>>> >>>>> Default zero. (and it should be 255 for display) >>>> >>>> If this is not a bug in the driver, there is no alpha channel that can be used. >>>> Rename it to 0YUV and remove the alpha usage. >>> >>> From the pixel format level, adding AYUV format is for a more fundamental/common usage. >> >> Yes, but the new pixel format will only be accepted if it is needed for hardware compatibility. >> If the hardware only supports 0YUV, it cannot be used to argue in favour of a new AYUV format. >> > > AYUV is a standard format used on Windows and documented by Microsoft > as the recommended 4:4:4 8-bit Sorry, I missed this: This is not the format suggested in this posted patch. (It is the format I suggested.) > format for all purposes. Its hardly a > niche format. Its perfectly appropriate to support it. Carl Eugen
> -----Original Message----- > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf > Of Carl Eugen Hoyos > Sent: Thursday, August 29, 2019 21:31 > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel format > > > > > Am 29.08.2019 um 14:30 schrieb Hendrik Leppkes <h.leppkes@gmail.com>: > > > >> On Thu, Aug 29, 2019 at 1:47 PM Carl Eugen Hoyos <ceffmpeg@gmail.com> > wrote: > >> > >> > >> > >> Am 29.08.2019 um 03:34 schrieb Fu, Linjie <linjie.fu@intel.com>: > >> > >>>> -----Original Message----- > >>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On > Behalf > >>>> Of Carl Eugen Hoyos > >>>> Sent: Thursday, August 29, 2019 05:17 > >>>> To: FFmpeg development discussions and patches <ffmpeg- > >>>> devel@ffmpeg.org> > >>>> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel > format > >>>> > >>>>>> Am Mi., 28. Aug. 2019 um 09:34 Uhr schrieb Fu, Linjie > <linjie.fu@intel.com>: > >>>>>> > >>>>>> -----Original Message----- > >>>>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] > On > >>>> Behalf > >>>>>> Of Carl Eugen Hoyos > >>>>>> Sent: Wednesday, August 28, 2019 14:51 > >>>>>> To: FFmpeg development discussions and patches <ffmpeg- > >>>>>> devel@ffmpeg.org> > >>>>>> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV > pixel > >>>> format > >>>>>> > >>>>>> > >>>>>> > >>>>>>> Am 28.08.2019 um 08:22 schrieb Linjie Fu <linjie.fu@intel.com>: > >>>>>>> > >>>>>>> Add support for packed 4:4:4 pixel format AYUV. > >>>>>>> > >>>>>>> It is the format that VAAPI/QSV uses when coping with 4:4:4 > >>>>>>> surfaces. > >>>>>> > >>>>>>> Alpha channel will be set to default value for HEVC REXT > >>>>>>> hw decode. > >>>>>> > >>>>>> What is the „default value“? > >>>>> > >>>>> Default zero. (and it should be 255 for display) > >>>> > >>>> If this is not a bug in the driver, there is no alpha channel that can be > used. > >>>> Rename it to 0YUV and remove the alpha usage. > >>> > >>> From the pixel format level, adding AYUV format is for a more > fundamental/common usage. > >> > >> Yes, but the new pixel format will only be accepted if it is needed for > hardware compatibility. > >> If the hardware only supports 0YUV, it cannot be used to argue in favour > of a new AYUV format. > >> > > > > AYUV is a standard format used on Windows and documented by Microsoft > > > as the recommended 4:4:4 8-bit > > Sorry, I missed this: > This is not the format suggested in this posted patch. > (It is the format I suggested.) > > > format for all purposes. Its hardly a > > niche format. Its perfectly appropriate to support it. Got the concern. If we need a reason to support this standard pixel format apart from the hardware compatibility, it could be some requirements like CSC from ARGB to YUV and don't want to lose the alpha channel information: https://stackoverflow.com/questions/6023565/argb-to-yuv-color-spaces-in-net "How to convert image beetween ARGB and YUV in c#? I found formulas describing RGB to YUV conversion. So what should I do with alpha channel? " Thus AYUV will make more sense. - linjie
Am Do., 29. Aug. 2019 um 16:16 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>:
> Thus AYUV will make more sense.
If your hardware decoding does not produce valid alpha data (0xFF
for opaque, 0x00 for completely transparent), you cannot use an
FFmpeg pix_fmt that defines alpha for decoding.
If you define a pix_fmt that does not contain alpha information (as
needed for your hardware decoder), please do not use "A" in its
name.
Carl Eugen
> -----Original Message----- > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf > Of Carl Eugen Hoyos > Sent: Friday, August 30, 2019 00:12 > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel format > > Am Do., 29. Aug. 2019 um 16:16 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>: > > > Thus AYUV will make more sense. > > If your hardware decoding does not produce valid alpha data (0xFF > for opaque, 0x00 for completely transparent), you cannot use an > FFmpeg pix_fmt that defines alpha for decoding. > If you define a pix_fmt that does not contain alpha information (as > needed for your hardware decoder), please do not use "A" in its > name. The hardware decode output is in packed 4:4:4:4 8 bit(32 bits in total). Apart from Y/U/V channel, there is an additional need for a channel to store the zero byte in surfaces(nb_components = 4). And it is designed in hardware/driver for VA_FourCC_AYUV and AYUV format. It's not quite proper to use something like "0YUV" directly which indicates that there is no Alpha channel data like"RGB0" or "0RGB"(nb_components = 3). "0YUV": YUV0/YUV0/... "RGB0":RGB/RGB/... It's true that we could only map Y/U/V data from surface to frame in ffmpeg and use a 3-components pixel format like "0YUV", but IMHO it's kind of improper because the output from hardware/driver is forced changed in Application level even if user didn't call for a CSC: - hardware/driver output (in AYUV): YUV0/YUV0/... - ffmpeg output (in 0YUV): YUV/YUV/... Please feel free to offer your opinions on this question(AYUV or 0YUV) and let me know how to push it step-further. Thanks, Linjie
On 31/08/2019 05:22, Fu, Linjie wrote: >> -----Original Message----- >> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf >> Of Carl Eugen Hoyos >> Sent: Friday, August 30, 2019 00:12 >> To: FFmpeg development discussions and patches <ffmpeg- >> devel@ffmpeg.org> >> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel format >> >> Am Do., 29. Aug. 2019 um 16:16 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>: >> >>> Thus AYUV will make more sense. >> >> If your hardware decoding does not produce valid alpha data (0xFF >> for opaque, 0x00 for completely transparent), you cannot use an >> FFmpeg pix_fmt that defines alpha for decoding. >> If you define a pix_fmt that does not contain alpha information (as >> needed for your hardware decoder), please do not use "A" in its >> name. > > The hardware decode output is in packed 4:4:4:4 8 bit(32 bits in total). Apart> from Y/U/V channel, there is an additional need for a channel to store the > zero byte in surfaces(nb_components = 4). And it is designed in hardware/driver > for VA_FourCC_AYUV and AYUV format. > > It's not quite proper to use something like "0YUV" directly which indicates that > there is no Alpha channel data like"RGB0" or "0RGB"(nb_components = 3). > > "0YUV": YUV0/YUV0/... > "RGB0":RGB/RGB/... I think you've misread what RGB0 is. The component step is 4, not 3 (see libavutil/pixdesc.c) - the format with a step of 3 is RGB24. RGBA: R G B A R G B A ... RGB0: R G B 0 R G B 0 ... RGB24: R G B R G B R G ... Following the same pattern we would have: AYUV: A Y U V A Y U V ... 0YUV: 0 Y U V 0 Y U V ... To me it looks like 0YUV is exactly what you want here. - Mark
> -----Original Message----- > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf > Of Mark Thompson > Sent: Saturday, August 31, 2019 23:10 > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel format > > On 31/08/2019 05:22, Fu, Linjie wrote: > >> -----Original Message----- > >> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On > Behalf > >> Of Carl Eugen Hoyos > >> Sent: Friday, August 30, 2019 00:12 > >> To: FFmpeg development discussions and patches <ffmpeg- > >> devel@ffmpeg.org> > >> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel > format > >> > >> Am Do., 29. Aug. 2019 um 16:16 Uhr schrieb Fu, Linjie > <linjie.fu@intel.com>: > >> > >>> Thus AYUV will make more sense. > >> > >> If your hardware decoding does not produce valid alpha data (0xFF > >> for opaque, 0x00 for completely transparent), you cannot use an > >> FFmpeg pix_fmt that defines alpha for decoding. > >> If you define a pix_fmt that does not contain alpha information (as > >> needed for your hardware decoder), please do not use "A" in its > >> name. > > > > The hardware decode output is in packed 4:4:4:4 8 bit(32 bits in total). Apart> > from Y/U/V channel, there is an additional need for a channel to store the > > zero byte in surfaces(nb_components = 4). And it is designed in > hardware/driver > > for VA_FourCC_AYUV and AYUV format. > > > > It's not quite proper to use something like "0YUV" directly which indicates > that > > there is no Alpha channel data like"RGB0" or "0RGB"(nb_components = 3). > > > > "0YUV": YUV0/YUV0/... > > "RGB0":RGB/RGB/... > > I think you've misread what RGB0 is. The component step is 4, not 3 (see > libavutil/pixdesc.c) - the format with a step of 3 is RGB24. > > RGBA: R G B A R G B A ... > RGB0: R G B 0 R G B 0 ... > RGB24: R G B R G B R G ... Thanks for pointing out this. Just took the nb_components into account and misread the step info at first. > Following the same pattern we would have: > > AYUV: A Y U V A Y U V ... > 0YUV: 0 Y U V 0 Y U V ... > > To me it looks like 0YUV is exactly what you want here. Will update the patch if 0YUV is more acceptable. And thanks Carl. - linjie
Am Sa., 31. Aug. 2019 um 19:25 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>: > > > -----Original Message----- > > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf > > Of Mark Thompson > > Sent: Saturday, August 31, 2019 23:10 > > To: ffmpeg-devel@ffmpeg.org > > Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel format > > > > On 31/08/2019 05:22, Fu, Linjie wrote: > > >> -----Original Message----- > > >> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On > > Behalf > > >> Of Carl Eugen Hoyos > > >> Sent: Friday, August 30, 2019 00:12 > > >> To: FFmpeg development discussions and patches <ffmpeg- > > >> devel@ffmpeg.org> > > >> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel > > format > > >> > > >> Am Do., 29. Aug. 2019 um 16:16 Uhr schrieb Fu, Linjie > > <linjie.fu@intel.com>: > > >> > > >>> Thus AYUV will make more sense. > > >> > > >> If your hardware decoding does not produce valid alpha data (0xFF > > >> for opaque, 0x00 for completely transparent), you cannot use an > > >> FFmpeg pix_fmt that defines alpha for decoding. > > >> If you define a pix_fmt that does not contain alpha information (as > > >> needed for your hardware decoder), please do not use "A" in its > > >> name. > > > > > > The hardware decode output is in packed 4:4:4:4 8 bit(32 bits in total). Apart> > > from Y/U/V channel, there is an additional need for a channel to store the > > > zero byte in surfaces(nb_components = 4). And it is designed in > > hardware/driver > > > for VA_FourCC_AYUV and AYUV format. > > > > > > It's not quite proper to use something like "0YUV" directly which indicates > > that > > > there is no Alpha channel data like"RGB0" or "0RGB"(nb_components = 3). > > > > > > "0YUV": YUV0/YUV0/... > > > "RGB0":RGB/RGB/... > > > > I think you've misread what RGB0 is. The component step is 4, not 3 (see > > libavutil/pixdesc.c) - the format with a step of 3 is RGB24. > > > > RGBA: R G B A R G B A ... > > RGB0: R G B 0 R G B 0 ... > > RGB24: R G B R G B R G ... > > Thanks for pointing out this. > Just took the nb_components into account and misread the step info at first. > > > Following the same pattern we would have: > > > > AYUV: A Y U V A Y U V ... > > 0YUV: 0 Y U V 0 Y U V ... > > > > To me it looks like 0YUV is exactly what you want here. > > Will update the patch if 0YUV is more acceptable. Could you double-check if there is no bug in the decoder? I ask because transparency is possible with hevc and the relevant Windows format does support transparency from how I read the specification. Carl Eugen
> -----Original Message----- > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf > Of Carl Eugen Hoyos > Sent: Sunday, September 1, 2019 01:57 > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel format > > Am Sa., 31. Aug. 2019 um 19:25 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>: > > > > > -----Original Message----- > > > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On > Behalf > > > Of Mark Thompson > > > Sent: Saturday, August 31, 2019 23:10 > > > To: ffmpeg-devel@ffmpeg.org > > > Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel > format > > > > > > On 31/08/2019 05:22, Fu, Linjie wrote: > > > >> -----Original Message----- > > > >> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On > > > Behalf > > > >> Of Carl Eugen Hoyos > > > >> Sent: Friday, August 30, 2019 00:12 > > > >> To: FFmpeg development discussions and patches <ffmpeg- > > > >> devel@ffmpeg.org> > > > >> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel > > > format > > > >> > > > >> Am Do., 29. Aug. 2019 um 16:16 Uhr schrieb Fu, Linjie > > > <linjie.fu@intel.com>: > > > >> > > > >>> Thus AYUV will make more sense. > > > >> > > > >> If your hardware decoding does not produce valid alpha data (0xFF > > > >> for opaque, 0x00 for completely transparent), you cannot use an > > > >> FFmpeg pix_fmt that defines alpha for decoding. > > > >> If you define a pix_fmt that does not contain alpha information (as > > > >> needed for your hardware decoder), please do not use "A" in its > > > >> name. > > > > > > > > The hardware decode output is in packed 4:4:4:4 8 bit(32 bits in total). > Apart> > > > from Y/U/V channel, there is an additional need for a channel to store the > > > > zero byte in surfaces(nb_components = 4). And it is designed in > > > hardware/driver > > > > for VA_FourCC_AYUV and AYUV format. > > > > > > > > It's not quite proper to use something like "0YUV" directly which > indicates > > > that > > > > there is no Alpha channel data like"RGB0" or "0RGB"(nb_components = > 3). > > > > > > > > "0YUV": YUV0/YUV0/... > > > > "RGB0":RGB/RGB/... > > > > > > I think you've misread what RGB0 is. The component step is 4, not 3 (see > > > libavutil/pixdesc.c) - the format with a step of 3 is RGB24. > > > > > > RGBA: R G B A R G B A ... > > > RGB0: R G B 0 R G B 0 ... > > > RGB24: R G B R G B R G ... > > > > Thanks for pointing out this. > > Just took the nb_components into account and misread the step info at > first. > > > > > Following the same pattern we would have: > > > > > > AYUV: A Y U V A Y U V ... > > > 0YUV: 0 Y U V 0 Y U V ... > > > > > > To me it looks like 0YUV is exactly what you want here. > > > > Will update the patch if 0YUV is more acceptable. > > Could you double-check if there is no bug in the decoder? > I ask because transparency is possible with hevc and the > relevant Windows format does support transparency from > how I read the specification. > Double confirmed. The alpha channel is configurable on HCP_SURFACE_STATE command, currently driver(iHD) set zero as default value. "0YUV" is good enough, "AYUV" could be hold on. - linjie
> -----Original Message----- > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf > Of Carl Eugen Hoyos > Sent: Sunday, September 1, 2019 01:57 > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel format > > Am Sa., 31. Aug. 2019 um 19:25 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>: > > > > > -----Original Message----- > > > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On > Behalf > > > Of Mark Thompson > > > Sent: Saturday, August 31, 2019 23:10 > > > To: ffmpeg-devel@ffmpeg.org > > > Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel > format > > > > > > On 31/08/2019 05:22, Fu, Linjie wrote: > > > >> -----Original Message----- > > > >> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On > > > Behalf > > > >> Of Carl Eugen Hoyos > > > >> Sent: Friday, August 30, 2019 00:12 > > > >> To: FFmpeg development discussions and patches <ffmpeg- > > > >> devel@ffmpeg.org> > > > >> Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: add AYUV pixel > > > format > > > >> > > > >> Am Do., 29. Aug. 2019 um 16:16 Uhr schrieb Fu, Linjie > > > <linjie.fu@intel.com>: > > > >> > > > >>> Thus AYUV will make more sense. > > > >> > > > >> If your hardware decoding does not produce valid alpha data (0xFF > > > >> for opaque, 0x00 for completely transparent), you cannot use an > > > >> FFmpeg pix_fmt that defines alpha for decoding. > > > >> If you define a pix_fmt that does not contain alpha information (as > > > >> needed for your hardware decoder), please do not use "A" in its > > > >> name. > > > > > > > > The hardware decode output is in packed 4:4:4:4 8 bit(32 bits in total). > Apart> > > > from Y/U/V channel, there is an additional need for a channel to store the > > > > zero byte in surfaces(nb_components = 4). And it is designed in > > > hardware/driver > > > > for VA_FourCC_AYUV and AYUV format. > > > > > > > > It's not quite proper to use something like "0YUV" directly which > indicates > > > that > > > > there is no Alpha channel data like"RGB0" or "0RGB"(nb_components = > 3). > > > > > > > > "0YUV": YUV0/YUV0/... > > > > "RGB0":RGB/RGB/... > > > > > > I think you've misread what RGB0 is. The component step is 4, not 3 (see > > > libavutil/pixdesc.c) - the format with a step of 3 is RGB24. > > > > > > RGBA: R G B A R G B A ... > > > RGB0: R G B 0 R G B 0 ... > > > RGB24: R G B R G B R G ... > > > > Thanks for pointing out this. > > Just took the nb_components into account and misread the step info at > first. > > > > > Following the same pattern we would have: > > > > > > AYUV: A Y U V A Y U V ... > > > 0YUV: 0 Y U V 0 Y U V ... > > > > > > To me it looks like 0YUV is exactly what you want here. > > > > Will update the patch if 0YUV is more acceptable. > > Could you double-check if there is no bug in the decoder? > I ask because transparency is possible with hevc and the > relevant Windows format does support transparency from > how I read the specification. > Hi, Rethink about this, hardware decoder should support a valid alpha data. Default 0x00 or 0xFF for alpha channel doesn't make sense in HEVC Rext decoding. Filed an issue in media driver: https://github.com/intel/media-driver/issues/719 Will rebase and resend the patch set for QSV HEVC REXT decoding, basing on AYUV,Y410, Y210. Thanks, Linjie
diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index b97b066..d217000 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -205,6 +205,19 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { { 0, 4, 1, 0, 8, 3, 7, 2 }, /* V */ }, }, + [AV_PIX_FMT_AYUV] = { + .name = "ayuv", + .nb_components = 4, + .log2_chroma_w = 0, + .log2_chroma_h = 0, + .comp = { + { 0, 4, 1, 0, 8, 3, 7, 2 }, /* Y */ + { 0, 4, 2, 0, 8, 3, 7, 3 }, /* U */ + { 0, 4, 3, 0, 8, 3, 7, 4 }, /* V */ + { 0, 4, 0, 0, 8, 3, 7, 1 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_ALPHA, + }, [AV_PIX_FMT_RGB24] = { .name = "rgb24", .nb_components = 3, diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h index 8b54c94..adbaec8 100644 --- a/libavutil/pixfmt.h +++ b/libavutil/pixfmt.h @@ -348,6 +348,8 @@ enum AVPixelFormat { AV_PIX_FMT_NV24, ///< planar YUV 4:4:4, 24bpp, 1 plane for Y and 1 plane for the UV components, which are interleaved (first byte U and the following byte V) AV_PIX_FMT_NV42, ///< as above, but U and V bytes are swapped + AV_PIX_FMT_AYUV, ///< packed YUV 4:4:4, 32bpp, A Y Cb Cr, + AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions }; diff --git a/libavutil/tests/pixfmt_best.c b/libavutil/tests/pixfmt_best.c index 53f7264..2939e48 100644 --- a/libavutil/tests/pixfmt_best.c +++ b/libavutil/tests/pixfmt_best.c @@ -91,6 +91,7 @@ int main(void) TEST(AV_PIX_FMT_YUVA420P, AV_PIX_FMT_YUV420P); TEST(AV_PIX_FMT_YUVA422P, AV_PIX_FMT_YUV422P); TEST(AV_PIX_FMT_YUVA444P, AV_PIX_FMT_YUV444P); + TEST(AV_PIX_FMT_AYUV, AV_PIX_FMT_YUV444P); TEST(AV_PIX_FMT_AYUV64, AV_PIX_FMT_YUV444P16); TEST(AV_PIX_FMT_RGBA, AV_PIX_FMT_RGB24); TEST(AV_PIX_FMT_ABGR, AV_PIX_FMT_RGB24); diff --git a/libavutil/version.h b/libavutil/version.h index ecc6a7c..658a508 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 56 -#define LIBAVUTIL_VERSION_MINOR 33 +#define LIBAVUTIL_VERSION_MINOR 34 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \
Add support for packed 4:4:4 pixel format AYUV. It is the format that VAAPI/QSV uses when coping with 4:4:4 surfaces. Alpha channel will be set to default value for HEVC REXT hw decode. Signed-off-by: Linjie Fu <linjie.fu@intel.com> --- libavutil/pixdesc.c | 13 +++++++++++++ libavutil/pixfmt.h | 2 ++ libavutil/tests/pixfmt_best.c | 1 + libavutil/version.h | 2 +- 4 files changed, 17 insertions(+), 1 deletion(-)