| Message ID | 20190511183157.27909-2-philipl@overt.org |
|---|---|
| State | New |
| Headers | show |
Am Sa., 11. Mai 2019 um 20:32 Uhr schrieb Philip Langdale <philipl@overt.org>: > > These are the 4:4:4 variants of the semi-planar NV12/NV21 formats. > > These formats are not used much, so we've never had a reason to add > them until now. VDPAU recently added support HEVC 4:4:4 content > and when you use the OpenGL interop, the returned surfaces are in I wonder if there is an "only" missing somewhere. > NV24 format, so we need the pixel format for media players, even > if there's no direct use within ffmpeg. Thank you! > Separately, there are apparently webcams that use NV24, but I've > never seen one. [...] > diff --git a/libavutil/version.h b/libavutil/version.h > index c0968de621..4922e267cc 100644 > --- a/libavutil/version.h > +++ b/libavutil/version.h > @@ -80,7 +80,7 @@ > > #define LIBAVUTIL_VERSION_MAJOR 56 > #define LIBAVUTIL_VERSION_MINOR 26 > -#define LIBAVUTIL_VERSION_MICRO 101 > +#define LIBAVUTIL_VERSION_MICRO 102 We typically raise minor for new pix_fmts, resetting micro was forgotten the last times afaict. Carl Eugen
> -----Original Message----- > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf > Of Carl Eugen Hoyos > Sent: Sunday, May 12, 2019 06:25 > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 1/3] avutil: Add NV24 and NV42 pixel > formats > > Am Sa., 11. Mai 2019 um 20:32 Uhr schrieb Philip Langdale > <philipl@overt.org>: > > > > These are the 4:4:4 variants of the semi-planar NV12/NV21 formats. > > > > These formats are not used much, so we've never had a reason to add > > them until now. VDPAU recently added support HEVC 4:4:4 content > > and when you use the OpenGL interop, the returned surfaces are in > > I wonder if there is an "only" missing somewhere. > > > NV24 format, so we need the pixel format for media players, even > > if there's no direct use within ffmpeg. > > Thank you! > > > Separately, there are apparently webcams that use NV24, but I've > > never seen one. > > [...] > > > diff --git a/libavutil/version.h b/libavutil/version.h > > index c0968de621..4922e267cc 100644 > > --- a/libavutil/version.h > > +++ b/libavutil/version.h > > @@ -80,7 +80,7 @@ > > > > #define LIBAVUTIL_VERSION_MAJOR 56 > > #define LIBAVUTIL_VERSION_MINOR 26 > > -#define LIBAVUTIL_VERSION_MICRO 101 > > +#define LIBAVUTIL_VERSION_MICRO 102 > > We typically raise minor for new pix_fmts, resetting > micro was forgotten the last times afaict. > Hi, I'm working on adding support for HEVC 4:2:2/ 4:4:4, 8/10 bit, in vaapi decoding. And I'm aslo thinking of add some new pix_fmt for packed FourCC: 444 8 bit : AYUV 444 10 bit: Y410 422 10 bit: Y210 So if I'd like to add these new pix_fmts, some modification is necessary too as far as I got from relevant threads: 1. add support in swscale for being able to convert to and from a format, including up and down scale; 2. add test.(maybe a query is needed too) 3. bump a new minor version. Any other concerns should be followed? I'm trying to make sure I understood this correctly. Thanks in advance. Linjie
Am So., 12. Mai 2019 um 05:38 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>: > I'm working on adding support for HEVC 4:2:2/ 4:4:4, 8/10 bit, in vaapi decoding. > And I'm aslo thinking of add some new pix_fmt for packed FourCC: > 444 8 bit : AYUV > 444 10 bit: Y410 > 422 10 bit: Y210 Do I understand correctly that for 420 8 bit, vaapi produces the decoded image in a planar format but for 444 8 bit (and 10 bit), you decided that a packed format makes more sense? (Aren't luma and chroma stored independently in hevc?) Note that no matter what you do with the decoded image, there will be a (measurable) performance hit because of the needed transformation into something (planar) that other filters and encoders understand. Carl Eugen
> -----Original Message----- > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf > Of Carl Eugen Hoyos > Sent: Sunday, May 12, 2019 18:45 > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 1/3] avutil: Add NV24 and NV42 pixel > formats > > Am So., 12. Mai 2019 um 05:38 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>: > > > I'm working on adding support for HEVC 4:2:2/ 4:4:4, 8/10 bit, in vaapi > decoding. > > And I'm aslo thinking of add some new pix_fmt for packed FourCC: > > 444 8 bit : AYUV > > 444 10 bit: Y410 > > 422 10 bit: Y210 > > Do I understand correctly that for 420 8 bit, vaapi produces the > decoded image in a planar format but for 444 8 bit (and 10 bit), > you decided that a packed format makes more sense? > (Aren't luma and chroma stored independently in hevc?) > It's true that for 420 8 bit, the decoded image is in planar format. But as to 444 8 bit(and 10 bit), as far as I know in media driver , the decoded image is produced in packed format. See: https://github.com/intel/media-driver/blob/8773ef5e82e8a151e1bcfef317f7808f319c91c2/media_driver/linux/gen11/ddi/media_libva_caps_g11.cpp#L842 else if(profile == VAProfileHEVCMain444 || profile == VAProfileVP9Profile1) { attribs[i].type = VASurfaceAttribPixelFormat; attribs[i].value.type = VAGenericValueTypeInteger; attribs[i].flags = VA_SURFACE_ATTRIB_GETTABLE | VA_SURFACE_ATTRIB_SETTABLE; attribs[i].value.value.i = VA_FOURCC_AYUV; i++; } > Note that no matter what you do with the decoded image, there > will be a (measurable) performance hit because of the needed > transformation into something (planar) that other filters and > encoders understand. Agree, the directly decoded format is better. And it's also one of the reasons I'd like to add these packed formats. Linjie
diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index fe38344d73..b97b0665b0 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -2320,6 +2320,30 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { }, .flags = AV_PIX_FMT_FLAG_PLANAR | AV_PIX_FMT_FLAG_ALPHA, }, + [AV_PIX_FMT_NV24] = { + .name = "nv24", + .nb_components = 3, + .log2_chroma_w = 0, + .log2_chroma_h = 0, + .comp = { + { 0, 1, 0, 0, 8, 0, 7, 1 }, /* Y */ + { 1, 2, 0, 0, 8, 1, 7, 1 }, /* U */ + { 1, 2, 1, 0, 8, 1, 7, 2 }, /* V */ + }, + .flags = AV_PIX_FMT_FLAG_PLANAR, + }, + [AV_PIX_FMT_NV42] = { + .name = "nv42", + .nb_components = 3, + .log2_chroma_w = 0, + .log2_chroma_h = 0, + .comp = { + { 0, 1, 0, 0, 8, 0, 7, 1 }, /* Y */ + { 1, 2, 1, 0, 8, 1, 7, 2 }, /* U */ + { 1, 2, 0, 0, 8, 1, 7, 1 }, /* V */ + }, + .flags = AV_PIX_FMT_FLAG_PLANAR, + }, }; #if FF_API_PLUS1_MINUS1 FF_ENABLE_DEPRECATION_WARNINGS diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h index 24d1b7e415..8b54c9415b 100644 --- a/libavutil/pixfmt.h +++ b/libavutil/pixfmt.h @@ -345,6 +345,9 @@ enum AVPixelFormat { AV_PIX_FMT_YUVA444P12BE, ///< planar YUV 4:4:4,36bpp, (1 Cr & Cb sample per 1x1 Y samples), 12b alpha, big-endian AV_PIX_FMT_YUVA444P12LE, ///< planar YUV 4:4:4,36bpp, (1 Cr & Cb sample per 1x1 Y samples), 12b alpha, little-endian + AV_PIX_FMT_NV24, ///< planar YUV 4:4:4, 24bpp, 1 plane for Y and 1 plane for the UV components, which are interleaved (first byte U and the following byte V) + AV_PIX_FMT_NV42, ///< as above, but U and V bytes are swapped + AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions }; diff --git a/libavutil/tests/pixfmt_best.c b/libavutil/tests/pixfmt_best.c index e98fcc19a5..53f7264207 100644 --- a/libavutil/tests/pixfmt_best.c +++ b/libavutil/tests/pixfmt_best.c @@ -76,6 +76,7 @@ int main(void) TEST(AV_PIX_FMT_P010, AV_PIX_FMT_YUV420P10); TEST(AV_PIX_FMT_P016, AV_PIX_FMT_YUV420P16); TEST(AV_PIX_FMT_NV16, AV_PIX_FMT_YUV422P); + TEST(AV_PIX_FMT_NV24, AV_PIX_FMT_YUV444P); TEST(AV_PIX_FMT_YUYV422, AV_PIX_FMT_YUV422P); TEST(AV_PIX_FMT_UYVY422, AV_PIX_FMT_YUV422P); TEST(AV_PIX_FMT_BGR565, AV_PIX_FMT_RGB565); diff --git a/libavutil/version.h b/libavutil/version.h index c0968de621..4922e267cc 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -80,7 +80,7 @@ #define LIBAVUTIL_VERSION_MAJOR 56 #define LIBAVUTIL_VERSION_MINOR 26 -#define LIBAVUTIL_VERSION_MICRO 101 +#define LIBAVUTIL_VERSION_MICRO 102 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ LIBAVUTIL_VERSION_MINOR, \
These are the 4:4:4 variants of the semi-planar NV12/NV21 formats. These formats are not used much, so we've never had a reason to add them until now. VDPAU recently added support HEVC 4:4:4 content and when you use the OpenGL interop, the returned surfaces are in NV24 format, so we need the pixel format for media players, even if there's no direct use within ffmpeg. Separately, there are apparently webcams that use NV24, but I've never seen one. Signed-off-by: Philip Langdale <philipl@overt.org> --- libavutil/pixdesc.c | 24 ++++++++++++++++++++++++ libavutil/pixfmt.h | 3 +++ libavutil/tests/pixfmt_best.c | 1 + libavutil/version.h | 2 +- 4 files changed, 29 insertions(+), 1 deletion(-)