| Message ID | 20200619015248.21873-1-fei.w.wang@intel.com |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel,v1,1/9] lavu/pix_fmt: add P012 pixel format | expand |
| Context | Check | Description |
|---|---|---|
| andriy/default | pending | |
| andriy/make | success | Make finished |
| andriy/make_fate | success | Make fate finished |
On Fri, Jun 19, 2020 at 3:53 AM Fei Wang <fei.w.wang@intel.com> wrote: > > P012 is 12bit planner format which is similar to NV12. It using two > bytes to store 12bit valid data and 4bit zero in LSB. This format > will be used for hardware decode/encode in VAAPI and QSV. > P012 is not required, you can just use P016. All these P* formats have the advantage of being aligned to MSB, which means the bitdepth does not matter for their layout. Instead you can use P016 and if you must know, look at a bitdepth value separately. - Hendrik
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Hendrik Leppkes > Sent: Friday, June 19, 2020 3:21 PM > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v1 1/9] lavu/pix_fmt: add P012 pixel > format > > On Fri, Jun 19, 2020 at 3:53 AM Fei Wang <fei.w.wang@intel.com> wrote: > > > > P012 is 12bit planner format which is similar to NV12. It using two > > bytes to store 12bit valid data and 4bit zero in LSB. This format will > > be used for hardware decode/encode in VAAPI and QSV. > > > > P012 is not required, you can just use P016. All these P* formats have the > advantage of being aligned to MSB, which means the bitdepth does not matter > for their layout. > Instead you can use P016 and if you must know, look at a bitdepth value > separately. If using P016 instead of P012 for 12bit clips, that means for 16bit clips we also need to use P016, which will bring the conflict with current VAAPI logic. It need to resolve same av format map to different RT format and VA format. @Mark Thompson, how about your opinion of this? The other reason that I used P012 is that P010 also existed in ffempg, but not replaced by P016. Fei > > - Hendrik > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org > with subject "unsubscribe".
Am Fr., 19. Juni 2020 um 10:27 Uhr schrieb Wang, Fei W <fei.w.wang@intel.com>: > > > > > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > Hendrik Leppkes > > Sent: Friday, June 19, 2020 3:21 PM > > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> > > Subject: Re: [FFmpeg-devel] [PATCH v1 1/9] lavu/pix_fmt: add P012 pixel > > format > > > > On Fri, Jun 19, 2020 at 3:53 AM Fei Wang <fei.w.wang@intel.com> wrote: > > > > > > P012 is 12bit planner format which is similar to NV12. It using two > > > bytes to store 12bit valid data and 4bit zero in LSB. This format will > > > be used for hardware decode/encode in VAAPI and QSV. > > > > > > > P012 is not required, you can just use P016. All these P* formats have the > > advantage of being aligned to MSB, which means the bitdepth does not matter > > for their layout. > > Instead you can use P016 and if you must know, look at a bitdepth value > > separately. > If using P016 instead of P012 for 12bit clips, that means for 16bit clips we also > need to use P016, which will bring the conflict with current VAAPI logic. Then fix the logic, it is easy to detect how many significant bits the data contains. Carl Eugen
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Carl > Eugen Hoyos > Sent: Saturday, June 20, 2020 12:59 AM > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v1 1/9] lavu/pix_fmt: add P012 pixel > format > > Am Fr., 19. Juni 2020 um 10:27 Uhr schrieb Wang, Fei W > <fei.w.wang@intel.com>: > > > > > > > > > -----Original Message----- > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > > Hendrik Leppkes > > > Sent: Friday, June 19, 2020 3:21 PM > > > To: FFmpeg development discussions and patches > > > <ffmpeg-devel@ffmpeg.org> > > > Subject: Re: [FFmpeg-devel] [PATCH v1 1/9] lavu/pix_fmt: add P012 > > > pixel format > > > > > > On Fri, Jun 19, 2020 at 3:53 AM Fei Wang <fei.w.wang@intel.com> wrote: > > > > > > > > P012 is 12bit planner format which is similar to NV12. It using > > > > two bytes to store 12bit valid data and 4bit zero in LSB. This > > > > format will be used for hardware decode/encode in VAAPI and QSV. > > > > > > > > > > P012 is not required, you can just use P016. All these P* formats > > > have the advantage of being aligned to MSB, which means the bitdepth > > > does not matter for their layout. > > > Instead you can use P016 and if you must know, look at a bitdepth > > > value separately. > > If using P016 instead of P012 for 12bit clips, that means for 16bit > > clips we also need to use P016, which will bring the conflict with current > VAAPI logic. > > Then fix the logic, it is easy to detect how many significant bits the data > contains. Yes, it is not hard to refine the logic, and before the action I'd like to listen From Mark Thompson, the maintainer of VAAPI to reach an agreement. Kindly Ping @Mark Thompson > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org > with subject "unsubscribe".
On 19/06/2020 08:21, Hendrik Leppkes wrote: > On Fri, Jun 19, 2020 at 3:53 AM Fei Wang <fei.w.wang@intel.com> wrote: >> >> P012 is 12bit planner format which is similar to NV12. It using two >> bytes to store 12bit valid data and 4bit zero in LSB. This format >> will be used for hardware decode/encode in VAAPI and QSV. >> > > P012 is not required, you can just use P016. All these P* formats > have the advantage of being aligned to MSB, which means the bitdepth > does not matter for their layout. > Instead you can use P016 and if you must know, look at a bitdepth > value separately. Tracking it separately does not seem fun - it looks to me like it would require adding a new bit depth field to AVFrame. FFmpeg has always used pixfmt as defining both the memory layout and which bits are used in that (so, for example, ARGB and 0RGB are not the same thing), unlike most of the graphics APIs which tend to define those two separately. - Mark
Am So., 28. Juni 2020 um 23:01 Uhr schrieb Mark Thompson <sw@jkqxz.net>: > FFmpeg has always used pixfmt as defining both the memory layout > and which bits are used in that (so, for example, ARGB and 0RGB > are not the same thing) But they have the same bitdepth per component... Carl Eugen
Agree with Mark. P012 and P016 have different significant bits, we should use different pixfmts, otherwise an extra field in AVFrame is needed for bit depth. BTW there are the YUV420P variants for 10 / 12 / 14 / 16 bit in FFmpeg, it would be better to follow FFmpeg's style to introduce P012 format instead of reusing P016. Thanks Haihao > Am So., 28. Juni 2020 um 23:01 Uhr schrieb Mark Thompson <sw@jkqxz.net>: > > > FFmpeg has always used pixfmt as defining both the memory layout > > and which bits are used in that (so, for example, ARGB and 0RGB > > are not the same thing) > > But they have the same bitdepth per component... > > Carl Eugen > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Xiang, > Haihao > Sent: Tuesday, June 30, 2020 15:34 > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v1 1/9] lavu/pix_fmt: add P012 pixel > format > > > Am So., 28. Juni 2020 um 23:01 Uhr schrieb Mark Thompson <sw@jkqxz.net>: > > > > > FFmpeg has always used pixfmt as defining both the memory layout > > > and which bits are used in that (so, for example, ARGB and 0RGB > > > are not the same thing) > > > > But they have the same bitdepth per component... > > > Agree with Mark. P012 and P016 have different significant bits, we should use > different pixfmts, otherwise an extra field in AVFrame is needed for bit depth. > > BTW there are the YUV420P variants for 10 / 12 / 14 / 16 bit in FFmpeg, it > would > be better to follow FFmpeg's style to introduce P012 format instead of reusing > P016. Sorry for picking up this old thread. We'd like to add the support for 12bit decoding / encoding in VAAPI and QSV. Is there any other concern if adding P012 in FFmpeg ? Thanks Haihao
Am Fr., 21. Jan. 2022 um 06:56 Uhr schrieb Xiang, Haihao <haihao.xiang-at-intel.com@ffmpeg.org>: > > > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Xiang, > > Haihao > > Sent: Tuesday, June 30, 2020 15:34 > > To: ffmpeg-devel@ffmpeg.org > > Subject: Re: [FFmpeg-devel] [PATCH v1 1/9] lavu/pix_fmt: add P012 pixel > > format > > > > > Am So., 28. Juni 2020 um 23:01 Uhr schrieb Mark Thompson <sw@jkqxz.net>: > > > > > > > FFmpeg has always used pixfmt as defining both the memory layout > > > > and which bits are used in that (so, for example, ARGB and 0RGB > > > > are not the same thing) > > > > > > But they have the same bitdepth per component... > > > Agree with Mark. P012 and P016 have different significant bits, we should use > > different pixfmts, otherwise an extra field in AVFrame is needed for bit depth. > > > > BTW there are the YUV420P variants for 10 / 12 / 14 / 16 bit in FFmpeg, it > > would be better to follow FFmpeg's style to introduce P012 format instead > > of reusing P016. I was under the impression that YUV420P12 is much more different from YUV420P16 than P012 from P016: Did I misunderstand? (Reading the thread again, I don't think I did) > Sorry for picking up this old thread. > > We'd like to add the support for 12bit decoding / encoding in VAAPI and QSV. > Is there any other concern if adding P012 in FFmpeg ? Did you already explain why you cannot use P016 or in which situation it would create a different output? Carl Eugen
On Sun, 2022-01-23 at 01:06 +0100, Carl Eugen Hoyos wrote: > Am Fr., 21. Jan. 2022 um 06:56 Uhr schrieb Xiang, Haihao > <haihao.xiang-at-intel.com@ffmpeg.org>: > > > > > -----Original Message----- > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Xiang, > > > Haihao > > > Sent: Tuesday, June 30, 2020 15:34 > > > To: ffmpeg-devel@ffmpeg.org > > > Subject: Re: [FFmpeg-devel] [PATCH v1 1/9] lavu/pix_fmt: add P012 pixel > > > format > > > > > > > Am So., 28. Juni 2020 um 23:01 Uhr schrieb Mark Thompson <sw@jkqxz.net>: > > > > > > > > > FFmpeg has always used pixfmt as defining both the memory layout > > > > > and which bits are used in that (so, for example, ARGB and 0RGB > > > > > are not the same thing) > > > > > > > > But they have the same bitdepth per component... > > > Agree with Mark. P012 and P016 have different significant bits, we should > > > use > > > different pixfmts, otherwise an extra field in AVFrame is needed for bit > > > depth. > > > > > > BTW there are the YUV420P variants for 10 / 12 / 14 / 16 bit in FFmpeg, it > > > would be better to follow FFmpeg's style to introduce P012 format instead > > > of reusing P016. > > I was under the impression that YUV420P12 is much more different from > YUV420P16 than P012 from P016: Did I misunderstand? > (Reading the thread again, I don't think I did) I understand YUV420P12 is a LSB format, so YUV420P12 can't be taken as YUV420P16. The style here means FFmpeg uses pixfmt for both memory layout and bit depth. > > > Sorry for picking up this old thread. > > > > We'd like to add the support for 12bit decoding / encoding in VAAPI and QSV. > > Is there any other concern if adding P012 in FFmpeg ? > > Did you already explain why you cannot use P016 or in which situation > it would create a different output? $ ffmpeg -hwaccel vaapi -f rawvideo -pix_fmt p016 -s 1920x1080 -i input.yuv -vf "hwupload,format=vaapi" -c:v hevc_vaapi -f null - If using P016, how will we know the input is 12bit indeed, not 14/16bit if hevc_vaapi may support both 12bit and 14/16bit inputs in the future ? Will we add a new option to ffmpeg to specify the bit depth and a new flag in AVFrame ? Thanks Haihao > > Carl Eugen > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Am Mi., 26. Jan. 2022 um 09:33 Uhr schrieb Xiang, Haihao <haihao.xiang-at-intel.com@ffmpeg.org>: > > Did you already explain why you cannot use P016 or in which situation > > it would create a different output? > > $ ffmpeg -hwaccel vaapi -f rawvideo -pix_fmt p016 -s 1920x1080 -i input.yuv > -vf "hwupload,format=vaapi" -c:v hevc_vaapi -f null - > > If using P016, how will we know the input is 12bit indeed Thank you for explaining this, at least I didn't realize this reason earlier! I hope we can find another solution than to add an additional pix_fmt. I believe I had suggested an additional property bit_depth for raw video a long time ago. Carl Eugen
diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index 8274713226..38297e2e83 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -2147,6 +2147,30 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { }, .flags = AV_PIX_FMT_FLAG_PLANAR | AV_PIX_FMT_FLAG_BE, }, + [AV_PIX_FMT_P012LE] = { + .name = "p012le", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 1, + .comp = { + { 0, 2, 0, 4, 12, 1, 11, 1 }, /* Y */ + { 1, 4, 0, 4, 12, 3, 11, 1 }, /* U */ + { 1, 4, 2, 4, 12, 3, 11, 3 }, /* V */ + }, + .flags = AV_PIX_FMT_FLAG_PLANAR, + }, + [AV_PIX_FMT_P012BE] = { + .name = "p012be", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 1, + .comp = { + { 0, 2, 0, 4, 12, 1, 11, 1 }, /* Y */ + { 1, 4, 0, 4, 12, 3, 11, 1 }, /* U */ + { 1, 4, 2, 4, 12, 3, 11, 3 }, /* V */ + }, + .flags = AV_PIX_FMT_FLAG_PLANAR | AV_PIX_FMT_FLAG_BE, + }, [AV_PIX_FMT_P016LE] = { .name = "p016le", .nb_components = 3, diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h index a46acf3c5e..261e1ea352 100644 --- a/libavutil/pixfmt.h +++ b/libavutil/pixfmt.h @@ -360,6 +360,10 @@ enum AVPixelFormat { AV_PIX_FMT_X2RGB10LE, ///< packed RGB 10:10:10, 30bpp, (msb)2X 10R 10G 10B(lsb), little-endian, X=unused/undefined AV_PIX_FMT_X2RGB10BE, ///< packed RGB 10:10:10, 30bpp, (msb)2X 10R 10G 10B(lsb), big-endian, X=unused/undefined + + AV_PIX_FMT_P012LE, ///< like NV12, with 12bpp per component, little-endian + AV_PIX_FMT_P012BE, ///< like NV12, with 12bpp per component, big-endian + AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions }; @@ -450,6 +454,7 @@ enum AVPixelFormat { #define AV_PIX_FMT_Y210 AV_PIX_FMT_NE(Y210BE, Y210LE) #define AV_PIX_FMT_X2RGB10 AV_PIX_FMT_NE(X2RGB10BE, X2RGB10LE) +#define AV_PIX_FMT_P012 AV_PIX_FMT_NE(P012BE, P012LE) /** * Chromaticity coordinates of the source primaries. diff --git a/libavutil/version.h b/libavutil/version.h index 3ce9b1831e..a63f79feb1 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 56 -#define LIBAVUTIL_VERSION_MINOR 55 +#define LIBAVUTIL_VERSION_MINOR 56 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ diff --git a/tests/ref/fate/sws-pixdesc-query b/tests/ref/fate/sws-pixdesc-query index c3cccfa492..f738a3ad74 100644 --- a/tests/ref/fate/sws-pixdesc-query +++ b/tests/ref/fate/sws-pixdesc-query @@ -57,6 +57,8 @@ isNBPS: nv20le p010be p010le + p012be + p012le x2rgb10be x2rgb10le xyz12be @@ -137,6 +139,7 @@ isBE: grayf32be nv20be p010be + p012be p016be rgb444be rgb48be @@ -188,6 +191,8 @@ isYUV: nv42 p010be p010le + p012be + p012le p016be p016le uyvy422 @@ -282,6 +287,8 @@ isPlanarYUV: nv42 p010be p010le + p012be + p012le p016be p016le yuv410p @@ -365,6 +372,8 @@ isSemiPlanarYUV: nv42 p010be p010le + p012be + p012le p016be p016le @@ -740,6 +749,8 @@ Planar: nv42 p010be p010le + p012be + p012le p016be p016le yuv410p
P012 is 12bit planner format which is similar to NV12. It using two bytes to store 12bit valid data and 4bit zero in LSB. This format will be used for hardware decode/encode in VAAPI and QSV. Signed-off-by: Fei Wang <fei.w.wang@intel.com> --- libavutil/pixdesc.c | 24 ++++++++++++++++++++++++ libavutil/pixfmt.h | 5 +++++ libavutil/version.h | 2 +- tests/ref/fate/sws-pixdesc-query | 11 +++++++++++ 4 files changed, 41 insertions(+), 1 deletion(-)