| Message ID | 1575470593-7732-1-git-send-email-linjie.fu@intel.com |
|---|---|
| State | New |
| Headers | show |
On 2019-12-04 15:43, Linjie Fu wrote: > Previously, media driver provided planar format(like 420 8 bit), > but for HEVC Range Extension (422/444 8/10 bit), the decoded image > is produced in packed format because Windows expects it. > > Add some packed pixel formats for hardware decode support in VAAPI > and QSV: > > 4:2:2 10 bit: Y210 > 4:4:4 8 bit: 0YUV > 4:4:4 10 bit: Y410 > > + [AV_PIX_FMT_Y410LE] = { > + .name = "y410le", > + .nb_components = 4, > + .log2_chroma_w = 0, > + .log2_chroma_h = 0, > + .comp = { > + { 0, 32, 10, 0, 10, 31, 9, 11 }, /* Y */ > + { 0, 32, 0, 0, 10, 31, 9, 1 }, /* U */ > + { 0, 32, 20, 0, 10, 31, 9, 21 }, /* V */ > + { 0, 32, 30, 0, 2, 31, 1, 31 }, /* A */ > + }, > + .flags = AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_BITSTREAM, > + }, > diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h > index d78e863..a163350 100644 > --- a/libavutil/pixfmt.h > +++ b/libavutil/pixfmt.h > @@ -348,6 +348,12 @@ enum AVPixelFormat { > AV_PIX_FMT_NV24, ///< planar YUV 4:4:4, 24bpp, 1 plane for Y and 1 plane for the UV components, which are interleaved (first byte U and the following byte V) > AV_PIX_FMT_NV42, ///< as above, but U and V bytes are swapped > > + AV_PIX_FMT_Y210BE, ///< packed YUV 4:2:2, 32bpp, Y0 Cb Y1 Cr, big-endian > + AV_PIX_FMT_Y210LE, ///< packed YUV 4:2:2, 32bpp, Y0 Cb Y1 Cr, little-endian > + AV_PIX_FMT_0YUV, ///< packed YUV 4:4:4, 32bpp, X Y Cb Cr, X=unused/undefined > + AV_PIX_FMT_Y410LE, ///< packed YUV 4:4:4, 32bpp, Cr Y Cb A, little-endian > + AV_PIX_FMT_Y410BE, ///< packed YUV 4:4:4, 32bpp, Cr Y Cb A, big-endian > + > AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions > }; > I will ask again. From > http://ffmpeg.org/pipermail/ffmpeg-devel/2019-June/245929.html > Why am I suspicious that at least one of those is a re-ordered v210? I > seem to recall that we rejected adding v210 to this list. Either they > don't belong in this list or they don't belong because libavcodec has a > proper decoder (at least for v210). > > This might be the thread I was remembering but March seems too recent >> https://ffmpeg.org/pipermail/ffmpeg-devel/2019-March/241549.html > > No real conclusion was reached there. > > Do bit-packed formats belong in an AVPixelFormat? Despite what was said last time I do believe this is packed. I have taken a little time to actually understand these magic number structs. y410 is clearly packed like v210. Look at the those offsets: 0, 10, 20, 30. Packed into a 32-bit word. Flagged with AV_PIX_FMT_FLAG_BITSTREAM. How is that any different to v210? Can you address a single sample in that 1 plane format without using shifts and bit-wise ands? Isn't that the definition of packed? I do not mean interleaved. Okay, y410 is a little better in that it is 444 so the sample order does not change through 6 word cycle. Is that the key difference? Do bit-packed formats belong in an AVPixelFormat? If yes then I do not object to this patch or any others like this. If no then why is this not rejected? Does the AV_PIX_FMT_FLAG_BITSTREAM flag mean they do belong? I admit I haven't seen this before so maybe I should shut up and not send this email.
On 12/5/19, James Darnley <james.darnley@gmail.com> wrote: > On 2019-12-04 15:43, Linjie Fu wrote: >> Previously, media driver provided planar format(like 420 8 bit), >> but for HEVC Range Extension (422/444 8/10 bit), the decoded image >> is produced in packed format because Windows expects it. >> >> Add some packed pixel formats for hardware decode support in VAAPI >> and QSV: >> >> 4:2:2 10 bit: Y210 >> 4:4:4 8 bit: 0YUV >> 4:4:4 10 bit: Y410 >> > >> + [AV_PIX_FMT_Y410LE] = { >> + .name = "y410le", >> + .nb_components = 4, >> + .log2_chroma_w = 0, >> + .log2_chroma_h = 0, >> + .comp = { >> + { 0, 32, 10, 0, 10, 31, 9, 11 }, /* Y */ >> + { 0, 32, 0, 0, 10, 31, 9, 1 }, /* U */ >> + { 0, 32, 20, 0, 10, 31, 9, 21 }, /* V */ >> + { 0, 32, 30, 0, 2, 31, 1, 31 }, /* A */ >> + }, >> + .flags = AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_BITSTREAM, >> + }, > > > >> diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h >> index d78e863..a163350 100644 >> --- a/libavutil/pixfmt.h >> +++ b/libavutil/pixfmt.h >> @@ -348,6 +348,12 @@ enum AVPixelFormat { >> AV_PIX_FMT_NV24, ///< planar YUV 4:4:4, 24bpp, 1 plane for Y and >> 1 plane for the UV components, which are interleaved (first byte U and the >> following byte V) >> AV_PIX_FMT_NV42, ///< as above, but U and V bytes are swapped >> >> + AV_PIX_FMT_Y210BE, ///< packed YUV 4:2:2, 32bpp, Y0 Cb Y1 Cr, >> big-endian >> + AV_PIX_FMT_Y210LE, ///< packed YUV 4:2:2, 32bpp, Y0 Cb Y1 Cr, >> little-endian >> + AV_PIX_FMT_0YUV, ///< packed YUV 4:4:4, 32bpp, X Y Cb Cr, >> X=unused/undefined >> + AV_PIX_FMT_Y410LE, ///< packed YUV 4:4:4, 32bpp, Cr Y Cb A, >> little-endian >> + AV_PIX_FMT_Y410BE, ///< packed YUV 4:4:4, 32bpp, Cr Y Cb A, >> big-endian >> + >> AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS >> if you want to link with shared libav* because the number of formats might >> differ between versions >> }; >> > > I will ask again. From >> http://ffmpeg.org/pipermail/ffmpeg-devel/2019-June/245929.html > >> Why am I suspicious that at least one of those is a re-ordered v210? I >> seem to recall that we rejected adding v210 to this list. Either they >> don't belong in this list or they don't belong because libavcodec has a >> proper decoder (at least for v210). >> >> This might be the thread I was remembering but March seems too recent >>> https://ffmpeg.org/pipermail/ffmpeg-devel/2019-March/241549.html >> >> No real conclusion was reached there. >> >> Do bit-packed formats belong in an AVPixelFormat? > > Despite what was said last time I do believe this is packed. I have > taken a little time to actually understand these magic number structs. > > y410 is clearly packed like v210. Look at the those offsets: 0, 10, 20, > 30. Packed into a 32-bit word. Flagged with AV_PIX_FMT_FLAG_BITSTREAM. > > How is that any different to v210? Can you address a single sample in > that 1 plane format without using shifts and bit-wise ands? Isn't that > the definition of packed? I do not mean interleaved. > > Okay, y410 is a little better in that it is 444 so the sample order does > not change through 6 word cycle. Is that the key difference? > > > > Do bit-packed formats belong in an AVPixelFormat? > > If yes then I do not object to this patch or any others like this. > > If no then why is this not rejected? Please summon Technical Committee, they sure will know the answer ;-) > > > > Does the AV_PIX_FMT_FLAG_BITSTREAM flag mean they do belong? I admit I > haven't seen this before so maybe I should shut up and not send this email. > > >
On 05/12/2019 11:55, James Darnley wrote: > On 2019-12-04 15:43, Linjie Fu wrote: >> Previously, media driver provided planar format(like 420 8 bit), >> but for HEVC Range Extension (422/444 8/10 bit), the decoded image >> is produced in packed format because Windows expects it. >> >> Add some packed pixel formats for hardware decode support in VAAPI >> and QSV: >> >> 4:2:2 10 bit: Y210 >> 4:4:4 8 bit: 0YUV >> 4:4:4 10 bit: Y410 >> > >> + [AV_PIX_FMT_Y410LE] = { >> + .name = "y410le", >> + .nb_components = 4, >> + .log2_chroma_w = 0, >> + .log2_chroma_h = 0, >> + .comp = { >> + { 0, 32, 10, 0, 10, 31, 9, 11 }, /* Y */ >> + { 0, 32, 0, 0, 10, 31, 9, 1 }, /* U */ >> + { 0, 32, 20, 0, 10, 31, 9, 21 }, /* V */ >> + { 0, 32, 30, 0, 2, 31, 1, 31 }, /* A */ >> + }, >> + .flags = AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_BITSTREAM, >> + }, > > > >> diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h >> index d78e863..a163350 100644 >> --- a/libavutil/pixfmt.h >> +++ b/libavutil/pixfmt.h >> @@ -348,6 +348,12 @@ enum AVPixelFormat { >> AV_PIX_FMT_NV24, ///< planar YUV 4:4:4, 24bpp, 1 plane for Y and 1 plane for the UV components, which are interleaved (first byte U and the following byte V) >> AV_PIX_FMT_NV42, ///< as above, but U and V bytes are swapped >> >> + AV_PIX_FMT_Y210BE, ///< packed YUV 4:2:2, 32bpp, Y0 Cb Y1 Cr, big-endian >> + AV_PIX_FMT_Y210LE, ///< packed YUV 4:2:2, 32bpp, Y0 Cb Y1 Cr, little-endian >> + AV_PIX_FMT_0YUV, ///< packed YUV 4:4:4, 32bpp, X Y Cb Cr, X=unused/undefined >> + AV_PIX_FMT_Y410LE, ///< packed YUV 4:4:4, 32bpp, Cr Y Cb A, little-endian >> + AV_PIX_FMT_Y410BE, ///< packed YUV 4:4:4, 32bpp, Cr Y Cb A, big-endian >> + >> AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions >> }; >> > > I will ask again. From >> http://ffmpeg.org/pipermail/ffmpeg-devel/2019-June/245929.html > >> Why am I suspicious that at least one of those is a re-ordered v210? I >> seem to recall that we rejected adding v210 to this list. Either they >> don't belong in this list or they don't belong because libavcodec has a >> proper decoder (at least for v210). >> >> This might be the thread I was remembering but March seems too recent >>> https://ffmpeg.org/pipermail/ffmpeg-devel/2019-March/241549.html >> >> No real conclusion was reached there. >> >> Do bit-packed formats belong in an AVPixelFormat? > > Despite what was said last time I do believe this is packed. I have > taken a little time to actually understand these magic number structs. > > y410 is clearly packed like v210. Look at the those offsets: 0, 10, 20, > 30. Packed into a 32-bit word. Flagged with AV_PIX_FMT_FLAG_BITSTREAM. > > How is that any different to v210? Can you address a single sample in > that 1 plane format without using shifts and bit-wise ands? Isn't that > the definition of packed? I do not mean interleaved. > > Okay, y410 is a little better in that it is 444 so the sample order does > not change through 6 word cycle. Is that the key difference? > > > > Do bit-packed formats belong in an AVPixelFormat? > > If yes then I do not object to this patch or any others like this. > > If no then why is this not rejected? > > > > Does the AV_PIX_FMT_FLAG_BITSTREAM flag mean they do belong? I admit I > haven't seen this before so maybe I should shut up and not send this email. The current precedent would say that it doesn't belong in AVPixelFormat, but it is still an in-memory representation of actual pixels that we need to be able to refer to. If it can't be used directly, then it should probably be added as an opaque format in order to be able to refer to it when using hardware codecs which produce exactly this (ending up with something which looks like the exsting hardware formats - they can't be used normally and have specific rules for each one, but they are still entries in AVPixelFormat). Personally I do think that this sort of format should probably be allowed (and v210 similarly), but there are ways around the problem if there is consensus against it. - Mark
diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index 05dd4a1..5fa934f 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -205,6 +205,66 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { { 0, 4, 1, 0, 8, 3, 7, 2 }, /* V */ }, }, + [AV_PIX_FMT_Y210LE] = { + .name = "y210le", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 0, + .comp = { + { 0, 4, 0, 6, 10, 3, 9, 1 }, /* Y */ + { 0, 8, 2, 6, 10, 7, 9, 3 }, /* U */ + { 0, 8, 6, 6, 10, 7, 9, 7 }, /* V */ + }, + }, + [AV_PIX_FMT_Y210BE] = { + .name = "y210be", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 0, + .comp = { + { 0, 4, 0, 6, 10, 3, 9, 1 }, /* Y */ + { 0, 8, 2, 6, 10, 7, 9, 3 }, /* U */ + { 0, 8, 6, 6, 10, 7, 9, 7 }, /* V */ + }, + .flags = AV_PIX_FMT_FLAG_BE, + }, + [AV_PIX_FMT_0YUV] = { + .name = "0yuv", + .nb_components = 3, + .log2_chroma_w = 0, + .log2_chroma_h = 0, + .comp = { + { 0, 4, 1, 0, 8, 3, 7, 2 }, /* Y */ + { 0, 4, 2, 0, 8, 3, 7, 3 }, /* U */ + { 0, 4, 3, 0, 8, 3, 7, 4 }, /* V */ + }, + }, + [AV_PIX_FMT_Y410LE] = { + .name = "y410le", + .nb_components = 4, + .log2_chroma_w = 0, + .log2_chroma_h = 0, + .comp = { + { 0, 32, 10, 0, 10, 31, 9, 11 }, /* Y */ + { 0, 32, 0, 0, 10, 31, 9, 1 }, /* U */ + { 0, 32, 20, 0, 10, 31, 9, 21 }, /* V */ + { 0, 32, 30, 0, 2, 31, 1, 31 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_BITSTREAM, + }, + [AV_PIX_FMT_Y410BE] = { + .name = "y410be", + .nb_components = 4, + .log2_chroma_w = 0, + .log2_chroma_h = 0, + .comp = { + { 0, 32, 10, 0, 10, 31, 9, 11 }, /* Y */ + { 0, 32, 0, 0, 10, 31, 9, 1 }, /* U */ + { 0, 32, 20, 0, 10, 31, 9, 21 }, /* V */ + { 0, 32, 30, 0, 2, 31, 1, 31 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_BITSTREAM | AV_PIX_FMT_FLAG_BE, + }, [AV_PIX_FMT_RGB24] = { .name = "rgb24", .nb_components = 3, diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h index d78e863..a163350 100644 --- a/libavutil/pixfmt.h +++ b/libavutil/pixfmt.h @@ -348,6 +348,12 @@ enum AVPixelFormat { AV_PIX_FMT_NV24, ///< planar YUV 4:4:4, 24bpp, 1 plane for Y and 1 plane for the UV components, which are interleaved (first byte U and the following byte V) AV_PIX_FMT_NV42, ///< as above, but U and V bytes are swapped + AV_PIX_FMT_Y210BE, ///< packed YUV 4:2:2, 32bpp, Y0 Cb Y1 Cr, big-endian + AV_PIX_FMT_Y210LE, ///< packed YUV 4:2:2, 32bpp, Y0 Cb Y1 Cr, little-endian + AV_PIX_FMT_0YUV, ///< packed YUV 4:4:4, 32bpp, X Y Cb Cr, X=unused/undefined + AV_PIX_FMT_Y410LE, ///< packed YUV 4:4:4, 32bpp, Cr Y Cb A, little-endian + AV_PIX_FMT_Y410BE, ///< packed YUV 4:4:4, 32bpp, Cr Y Cb A, big-endian + AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions }; @@ -436,6 +442,9 @@ enum AVPixelFormat { #define AV_PIX_FMT_P010 AV_PIX_FMT_NE(P010BE, P010LE) #define AV_PIX_FMT_P016 AV_PIX_FMT_NE(P016BE, P016LE) +#define AV_PIX_FMT_Y210 AV_PIX_FMT_NE(Y210BE, Y210LE) +#define AV_PIX_FMT_Y410 AV_PIX_FMT_NE(Y410BE, Y410LE) + /** * Chromaticity coordinates of the source primaries. * These values match the ones defined by ISO/IEC 23001-8_2013 ยง 7.1. diff --git a/libavutil/version.h b/libavutil/version.h index e181633..4de0fa1 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,8 +79,8 @@ */ #define LIBAVUTIL_VERSION_MAJOR 56 -#define LIBAVUTIL_VERSION_MINOR 36 -#define LIBAVUTIL_VERSION_MICRO 101 +#define LIBAVUTIL_VERSION_MINOR 37 +#define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ LIBAVUTIL_VERSION_MINOR, \ diff --git a/tests/ref/fate/sws-pixdesc-query b/tests/ref/fate/sws-pixdesc-query index e234922..d0efa92 100644 --- a/tests/ref/fate/sws-pixdesc-query +++ b/tests/ref/fate/sws-pixdesc-query @@ -59,6 +59,10 @@ isNBPS: p010le xyz12be xyz12le + y210be + y210le + y410be + y410le yuv420p10be yuv420p10le yuv420p12be @@ -140,6 +144,8 @@ isBE: rgb565be rgba64be xyz12be + y210be + y410be ya16be yuv420p10be yuv420p12be @@ -171,6 +177,7 @@ isBE: yuva444p9be isYUV: + 0yuv ayuv64be ayuv64le nv12 @@ -188,6 +195,10 @@ isYUV: uyyvyy411 xyz12be xyz12le + y210be + y210le + y410be + y410le ya16be ya16le ya8 @@ -599,6 +610,8 @@ ALPHA: rgb32_1 rgba64be rgba64le + y410be + y410le ya16be ya16le ya8 @@ -631,6 +644,7 @@ ALPHA: Packed: 0bgr 0rgb + 0yuv ayuv64be ayuv64le bayer_bggr16be @@ -686,6 +700,10 @@ Packed: uyyvyy411 xyz12be xyz12le + y210be + y210le + y410be + y410le ya16be ya16le ya8
Previously, media driver provided planar format(like 420 8 bit), but for HEVC Range Extension (422/444 8/10 bit), the decoded image is produced in packed format because Windows expects it. Add some packed pixel formats for hardware decode support in VAAPI and QSV: 4:2:2 10 bit: Y210 4:4:4 8 bit: 0YUV 4:4:4 10 bit: Y410 Add query support in fate and update the minor version. Signed-off-by: Linjie Fu <linjie.fu@intel.com> --- libavutil/pixdesc.c | 60 ++++++++++++++++++++++++++++++++++++++++ libavutil/pixfmt.h | 9 ++++++ libavutil/version.h | 4 +-- tests/ref/fate/sws-pixdesc-query | 18 ++++++++++++ 4 files changed, 89 insertions(+), 2 deletions(-)