| Message ID | 20220814213313.37948-3-philipl@overt.org |
|---|---|
| State | New |
| Headers | show |
| Series | V2: VAAPI: Add high bit depth encode/decode support | expand |
| Context | Check | Description |
|---|---|---|
| yinshiyou/make_loongarch64 | success | Make finished |
| yinshiyou/make_fate_loongarch64 | success | Make fate finished |
| andriy/make_x86 | fail | Make failed |
On Sun, Aug 14, 2022 at 02:33:12PM -0700, Philip Langdale wrote: > With the necessary pixel formats defined, we can now expose support for > the remaining 10/12bit combinations that VAAPI can handle. > > Specifically, we are adding support for: > > * HEVC > ** 12bit 420 > ** 10bit 422 > ** 12bit 422 > ** 10bit 444 > ** 12bit 444 > > * VP9 > ** 10bit 422 > ** 10bit 444 > > These obviously require actual hardware support to be usable, but where > that exists, it is now enabled. > > I had to make some adjustments to the encode logic for matching bit > depth as the existing code assumed that the picture depth and the pixel > format depth were always the same, which is not true for 12bit content > which uses 16bit pixel formats. breaks build on ubuntu x86-64, assuming i did not miss any patch make CC libavutil/hwcontext_vaapi.o libavutil/hwcontext_vaapi.c:103:9: error: ‘VA_RT_FORMAT_YUV420_12’ undeclared here (not in a function); did you mean ‘VA_RT_FORMAT_YUV420’? VA_RT_FORMAT_ ## rt, \ ^ libavutil/hwcontext_vaapi.c:137:5: note: in expansion of macro ‘MAP’ MAP(P016, YUV420_12, P016, 0), ^~~ ffbuild/common.mak:81: recipe for target 'libavutil/hwcontext_vaapi.o' failed make: *** [libavutil/hwcontext_vaapi.o] Error 1 [...]
On Tue, 16 Aug 2022 00:10:49 +0200 Michael Niedermayer <michael@niedermayer.cc> wrote: > On Sun, Aug 14, 2022 at 02:33:12PM -0700, Philip Langdale wrote: > > With the necessary pixel formats defined, we can now expose support > > for the remaining 10/12bit combinations that VAAPI can handle. > > > > Specifically, we are adding support for: > > > > * HEVC > > ** 12bit 420 > > ** 10bit 422 > > ** 12bit 422 > > ** 10bit 444 > > ** 12bit 444 > > > > * VP9 > > ** 10bit 422 > > ** 10bit 444 > > > > These obviously require actual hardware support to be usable, but > > where that exists, it is now enabled. > > > > I had to make some adjustments to the encode logic for matching bit > > depth as the existing code assumed that the picture depth and the > > pixel format depth were always the same, which is not true for > > 12bit content which uses 16bit pixel formats. > > breaks build on ubuntu x86-64, assuming i did not miss any patch > make > CC libavutil/hwcontext_vaapi.o > libavutil/hwcontext_vaapi.c:103:9: error: ‘VA_RT_FORMAT_YUV420_12’ > undeclared here (not in a function); did you mean > ‘VA_RT_FORMAT_YUV420’? VA_RT_FORMAT_ ## rt, \ ^ > libavutil/hwcontext_vaapi.c:137:5: note: in expansion of macro ‘MAP’ > MAP(P016, YUV420_12, P016, 0), > ^~~ > ffbuild/common.mak:81: recipe for target > 'libavutil/hwcontext_vaapi.o' failed make: *** > [libavutil/hwcontext_vaapi.o] Error 1 > I guess there's probably a libva version dependency I need to guard for. What version do you have installed? Thanks, --phil
On Mon, Aug 15, 2022 at 03:23:29PM -0700, Philip Langdale wrote: > On Tue, 16 Aug 2022 00:10:49 +0200 > Michael Niedermayer <michael@niedermayer.cc> wrote: > > > On Sun, Aug 14, 2022 at 02:33:12PM -0700, Philip Langdale wrote: > > > With the necessary pixel formats defined, we can now expose support > > > for the remaining 10/12bit combinations that VAAPI can handle. > > > > > > Specifically, we are adding support for: > > > > > > * HEVC > > > ** 12bit 420 > > > ** 10bit 422 > > > ** 12bit 422 > > > ** 10bit 444 > > > ** 12bit 444 > > > > > > * VP9 > > > ** 10bit 422 > > > ** 10bit 444 > > > > > > These obviously require actual hardware support to be usable, but > > > where that exists, it is now enabled. > > > > > > I had to make some adjustments to the encode logic for matching bit > > > depth as the existing code assumed that the picture depth and the > > > pixel format depth were always the same, which is not true for > > > 12bit content which uses 16bit pixel formats. > > > > breaks build on ubuntu x86-64, assuming i did not miss any patch > > make > > CC libavutil/hwcontext_vaapi.o > > libavutil/hwcontext_vaapi.c:103:9: error: ‘VA_RT_FORMAT_YUV420_12’ > > undeclared here (not in a function); did you mean > > ‘VA_RT_FORMAT_YUV420’? VA_RT_FORMAT_ ## rt, \ ^ > > libavutil/hwcontext_vaapi.c:137:5: note: in expansion of macro ‘MAP’ > > MAP(P016, YUV420_12, P016, 0), > > ^~~ > > ffbuild/common.mak:81: recipe for target > > 'libavutil/hwcontext_vaapi.o' failed make: *** > > [libavutil/hwcontext_vaapi.o] Error 1 > > > > I guess there's probably a libva version dependency I need to guard > for. What version do you have installed? ii libva-dev:amd64 2.1.0-3 amd64 Video Acceleration (VA) API for Linux -- development files ii libva-drm2:amd64 2.1.0-3 amd64 Video Acceleration (VA) API for Linux -- DRM runtime ii libva-glx2:amd64 2.1.0-3 amd64 Video Acceleration (VA) API for Linux -- GLX runtime ii libva-wayland2:amd64 2.1.0-3 amd64 Video Acceleration (VA) API for Linux -- Wayland runtime ii libva-x11-2:amd64 2.1.0-3 amd64 Video Acceleration (VA) API for Linux -- X11 runtime ii libva2:amd64 2.1.0-3 amd64 Video Acceleration (VA) API for Linux -- runtime thx [...]
On Tue, 16 Aug 2022 19:29:24 +0200 Michael Niedermayer <michael@niedermayer.cc> wrote: > > > CC libavutil/hwcontext_vaapi.o > > > libavutil/hwcontext_vaapi.c:103:9: error: ‘VA_RT_FORMAT_YUV420_12’ > > > undeclared here (not in a function); did you mean > > > ‘VA_RT_FORMAT_YUV420’? VA_RT_FORMAT_ ## rt, \ ^ > > > libavutil/hwcontext_vaapi.c:137:5: note: in expansion of macro > > > ‘MAP’ MAP(P016, YUV420_12, P016, 0), > > > ^~~ > > > ffbuild/common.mak:81: recipe for target > > > 'libavutil/hwcontext_vaapi.o' failed make: *** > > > [libavutil/hwcontext_vaapi.o] Error 1 > > > > > > > I guess there's probably a libva version dependency I need to guard > > for. What version do you have installed? > > ii libva-dev:amd64 > 2.1.0-3 amd64 That's older than what I have, although I'd have thought it would be new enough. Regardless, I've included the appropriate #ifdef guard in the latest version of the patchset. Thanks, --phil
On Tue, 16 Aug 2022 12:52:52 -0700 Philip Langdale <philipl@overt.org> wrote: > On Tue, 16 Aug 2022 19:29:24 +0200 > Michael Niedermayer <michael@niedermayer.cc> wrote: > > > > > CC libavutil/hwcontext_vaapi.o > > > > libavutil/hwcontext_vaapi.c:103:9: error: > > > > ‘VA_RT_FORMAT_YUV420_12’ undeclared here (not in a function); > > > > did you mean ‘VA_RT_FORMAT_YUV420’? VA_RT_FORMAT_ ## rt, \ ^ > > > > libavutil/hwcontext_vaapi.c:137:5: note: in expansion of macro > > > > ‘MAP’ MAP(P016, YUV420_12, P016, 0), > > > > ^~~ > > > > ffbuild/common.mak:81: recipe for target > > > > 'libavutil/hwcontext_vaapi.o' failed make: *** > > > > [libavutil/hwcontext_vaapi.o] Error 1 > > > > > > > > > > I guess there's probably a libva version dependency I need to > > > guard for. What version do you have installed? > > > > ii libva-dev:amd64 > > 2.1.0-3 amd64 > > That's older than what I have, although I'd have thought it would be > new enough. Regardless, I've included the appropriate #ifdef guard in > the latest version of the patchset. > > Thanks, Ooops. I thought that said 2.10.0, not 2.1.0. 2.1.0 is ancient (2018) but it turns out even 2.10.0 is too old. For the record it requires 2.13.0. --phil
On Tue, Aug 16, 2022 at 12:55:38PM -0700, Philip Langdale wrote: > On Tue, 16 Aug 2022 12:52:52 -0700 > Philip Langdale <philipl@overt.org> wrote: > > > On Tue, 16 Aug 2022 19:29:24 +0200 > > Michael Niedermayer <michael@niedermayer.cc> wrote: > > > > > > > CC libavutil/hwcontext_vaapi.o > > > > > libavutil/hwcontext_vaapi.c:103:9: error: > > > > > ‘VA_RT_FORMAT_YUV420_12’ undeclared here (not in a function); > > > > > did you mean ‘VA_RT_FORMAT_YUV420’? VA_RT_FORMAT_ ## rt, \ ^ > > > > > libavutil/hwcontext_vaapi.c:137:5: note: in expansion of macro > > > > > ‘MAP’ MAP(P016, YUV420_12, P016, 0), > > > > > ^~~ > > > > > ffbuild/common.mak:81: recipe for target > > > > > 'libavutil/hwcontext_vaapi.o' failed make: *** > > > > > [libavutil/hwcontext_vaapi.o] Error 1 > > > > > > > > > > > > > I guess there's probably a libva version dependency I need to > > > > guard for. What version do you have installed? > > > > > > ii libva-dev:amd64 > > > 2.1.0-3 amd64 > > > > That's older than what I have, although I'd have thought it would be > > new enough. Regardless, I've included the appropriate #ifdef guard in > > the latest version of the patchset. > > > > Thanks, > > Ooops. I thought that said 2.10.0, not 2.1.0. 2.1.0 is ancient (2018) > but it turns out even 2.10.0 is too old. For the record it requires > 2.13.0. its not ancient its ubuntu LTS ;) or rather whats in it [...]
diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c index f1be8af2cd..1a895800a6 100644 --- a/libavcodec/hevcdec.c +++ b/libavcodec/hevcdec.c @@ -481,11 +481,19 @@ static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps) #endif case AV_PIX_FMT_YUV420P12: case AV_PIX_FMT_YUV444P12: +#if CONFIG_HEVC_VAAPI_HWACCEL + *fmt++ = AV_PIX_FMT_VAAPI; +#endif #if CONFIG_HEVC_VDPAU_HWACCEL *fmt++ = AV_PIX_FMT_VDPAU; #endif #if CONFIG_HEVC_NVDEC_HWACCEL *fmt++ = AV_PIX_FMT_CUDA; +#endif + break; + case AV_PIX_FMT_YUV422P12: +#if CONFIG_HEVC_VAAPI_HWACCEL + *fmt++ = AV_PIX_FMT_VAAPI; #endif break; } diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c index bc2d3ed803..9ba7dfd886 100644 --- a/libavcodec/vaapi_decode.c +++ b/libavcodec/vaapi_decode.c @@ -262,16 +262,28 @@ static const struct { MAP(YUY2, YUYV422), #ifdef VA_FOURCC_Y210 MAP(Y210, Y210), +#endif +#ifdef VA_FOURCC_Y216 + MAP(Y216, Y216), #endif // 4:4:0 MAP(422V, YUV440P), // 4:4:4 MAP(444P, YUV444P), MAP(AYUV, VUYA), +#ifdef VA_FOURCC_Y410 + MAP(Y410, Y410), +#endif +#ifdef VA_FOURCC_Y416 + MAP(Y416, Y416), +#endif // 4:2:0 10-bit #ifdef VA_FOURCC_P010 MAP(P010, P010), #endif +#ifdef VA_FOURCC_P016 + MAP(P016, P016), +#endif #ifdef VA_FOURCC_I010 MAP(I010, YUV420P10), #endif @@ -415,6 +427,7 @@ static const struct { #if VA_CHECK_VERSION(0, 39, 0) MAP(VP9, VP9_1, VP9Profile1 ), MAP(VP9, VP9_2, VP9Profile2 ), + MAP(VP9, VP9_3, VP9Profile3 ), #endif #if VA_CHECK_VERSION(1, 8, 0) MAP(AV1, AV1_MAIN, AV1Profile0), diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index f13daa5cff..1cf11c4cd5 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1305,7 +1305,11 @@ static const VAAPIEncodeRTFormat vaapi_encode_rt_formats[] = { { "YUV420", VA_RT_FORMAT_YUV420, 8, 3, 1, 1 }, { "YUV422", VA_RT_FORMAT_YUV422, 8, 3, 1, 0 }, #if VA_CHECK_VERSION(1, 2, 0) + { "YUV420_12", VA_RT_FORMAT_YUV420_12, 12, 3, 1, 1 }, { "YUV422_10", VA_RT_FORMAT_YUV422_10, 10, 3, 1, 0 }, + { "YUV422_12", VA_RT_FORMAT_YUV422_12, 12, 3, 1, 0 }, + { "YUV444_10", VA_RT_FORMAT_YUV444_10, 10, 4, 0, 0 }, + { "YUV444_12", VA_RT_FORMAT_YUV444_12, 12, 4, 0, 0 }, #endif { "YUV444", VA_RT_FORMAT_YUV444, 8, 3, 0, 0 }, { "AYUV", VA_RT_FORMAT_YUV444, 8, 4, 0, 0 }, @@ -1342,7 +1346,7 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) VAConfigAttrib rt_format_attr; const VAAPIEncodeRTFormat *rt_format; const char *profile_string, *entrypoint_string; - int i, j, n, depth, err; + int i, j, n, depth, logical_depth, err; if (ctx->low_power) { @@ -1365,7 +1369,9 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) } depth = desc->comp[0].depth; for (i = 1; i < desc->nb_components; i++) { - if (desc->comp[i].depth != depth) { + // We do not apply this depth requirement to the fourth component as + // that will be the alpha channel when present, which can be smaller. + if (i < 3 && desc->comp[i].depth != depth) { av_log(avctx, AV_LOG_ERROR, "Invalid input pixfmt (%s).\n", desc->name); return AVERROR(EINVAL); @@ -1391,8 +1397,11 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) av_assert0(ctx->codec->profiles); for (i = 0; (ctx->codec->profiles[i].av_profile != FF_PROFILE_UNKNOWN); i++) { + profile = &ctx->codec->profiles[i]; - if (depth != profile->depth || + // Look for a 16bit format if the profile depth is > 10bit + logical_depth = profile->depth > 10 ? 16 : profile->depth; + if (depth != logical_depth || desc->nb_components != profile->nb_components) continue; if (desc->nb_components > 1 && @@ -1476,7 +1485,9 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) for (i = 0; i < FF_ARRAY_ELEMS(vaapi_encode_rt_formats); i++) { rt_format = &vaapi_encode_rt_formats[i]; - if (rt_format->depth == depth && + // Look for a 16bit pixel format if the rt format depth is > 10bit + logical_depth = rt_format->depth > 10 ? 16 : rt_format->depth; + if (logical_depth == depth && rt_format->nb_components == profile->nb_components && rt_format->log2_chroma_w == profile->log2_chroma_w && rt_format->log2_chroma_h == profile->log2_chroma_h) diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 1de323af78..1a65fb9ebf 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -283,7 +283,9 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) return AVERROR(EINVAL); } } - bit_depth = desc->comp[0].depth; + // Bit depth must be taken from the profile, as the pixel format will be + // 16bit for >= 12bit content. + bit_depth = ctx->profile->depth; // VPS @@ -1276,10 +1278,14 @@ static const VAAPIEncodeProfile vaapi_encode_h265_profiles[] = { { FF_PROFILE_HEVC_REXT, 10, 3, 1, 1, VAProfileHEVCMain10 }, #endif #if VA_CHECK_VERSION(1, 2, 0) + { FF_PROFILE_HEVC_REXT, 12, 3, 1, 1, VAProfileHEVCMain12 }, { FF_PROFILE_HEVC_REXT, 8, 3, 1, 0, VAProfileHEVCMain422_10 }, { FF_PROFILE_HEVC_REXT, 10, 3, 1, 0, VAProfileHEVCMain422_10 }, - // Four channels because this uses the AYUV format which has Alpha + { FF_PROFILE_HEVC_REXT, 12, 3, 1, 0, VAProfileHEVCMain422_12 }, + // Four channels because these use formats which have Alpha { FF_PROFILE_HEVC_REXT, 8, 4, 0, 0, VAProfileHEVCMain444 }, + { FF_PROFILE_HEVC_REXT, 10, 4, 0, 0, VAProfileHEVCMain444_10 }, + { FF_PROFILE_HEVC_REXT, 12, 4, 0, 0, VAProfileHEVCMain444_12 }, #endif { FF_PROFILE_UNKNOWN } }; diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index 9b455e10c9..e6c7f01f11 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -228,9 +228,11 @@ static av_cold int vaapi_encode_vp9_configure(AVCodecContext *avctx) static const VAAPIEncodeProfile vaapi_encode_vp9_profiles[] = { { FF_PROFILE_VP9_0, 8, 3, 1, 1, VAProfileVP9Profile0 }, - // Four channels because this uses the AYUV format which has Alpha + // Four channels because this uses a format which has Alpha { FF_PROFILE_VP9_1, 8, 4, 0, 0, VAProfileVP9Profile1 }, { FF_PROFILE_VP9_2, 10, 3, 1, 1, VAProfileVP9Profile2 }, + // Four channels because this uses a format which has Alpha + { FF_PROFILE_VP9_3, 10, 4, 0, 0, VAProfileVP9Profile3 }, { FF_PROFILE_UNKNOWN } }; diff --git a/libavcodec/vaapi_hevc.c b/libavcodec/vaapi_hevc.c index d82975979a..20fb36adfa 100644 --- a/libavcodec/vaapi_hevc.c +++ b/libavcodec/vaapi_hevc.c @@ -567,15 +567,24 @@ VAProfile ff_vaapi_parse_hevc_rext_profile(AVCodecContext *avctx) } #if VA_CHECK_VERSION(1, 2, 0) - if (!strcmp(profile->name, "Main 4:2:2 10") || + if (!strcmp(profile->name, "Main 12") || + !strcmp(profile->name, "Main 12 Intra")) + return VAProfileHEVCMain12; + else if (!strcmp(profile->name, "Main 4:2:2 10") || !strcmp(profile->name, "Main 4:2:2 10 Intra")) return VAProfileHEVCMain422_10; + else if (!strcmp(profile->name, "Main 4:2:2 12") || + !strcmp(profile->name, "Main 4:2:2 12 Intra")) + return VAProfileHEVCMain422_12; else if (!strcmp(profile->name, "Main 4:4:4") || !strcmp(profile->name, "Main 4:4:4 Intra")) return VAProfileHEVCMain444; else if (!strcmp(profile->name, "Main 4:4:4 10") || !strcmp(profile->name, "Main 4:4:4 10 Intra")) return VAProfileHEVCMain444_10; + else if (!strcmp(profile->name, "Main 4:4:4 12") || + !strcmp(profile->name, "Main 4:4:4 12 Intra")) + return VAProfileHEVCMain444_12; #else av_log(avctx, AV_LOG_WARNING, "HEVC profile %s is " "not supported with this VA version.\n", profile->name); diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 2ee5145727..c3e79907fd 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -121,6 +121,9 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { MAP(YUY2, YUV422, YUYV422, 0), #ifdef VA_FOURCC_Y210 MAP(Y210, YUV422_10, Y210, 0), +#endif +#ifdef VA_FOURCC_Y216 + MAP(Y216, YUV422_12, Y216, 0), #endif MAP(411P, YUV411, YUV411P, 0), MAP(422V, YUV422, YUV440P, 0), @@ -129,6 +132,9 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { MAP(Y800, YUV400, GRAY8, 0), #ifdef VA_FOURCC_P010 MAP(P010, YUV420_10BPP, P010, 0), +#endif +#ifdef VA_FOURCC_P016 + MAP(P016, YUV420_12, P016, 0), #endif MAP(BGRA, RGB32, BGRA, 0), MAP(BGRX, RGB32, BGR0, 0), @@ -143,6 +149,12 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { #ifdef VA_FOURCC_X2R10G10B10 MAP(X2R10G10B10, RGB32_10, X2RGB10, 0), #endif +#ifdef VA_FOURCC_Y410 + MAP(Y410, YUV444_10, Y410, 0), +#endif +#ifdef VA_FOURCC_Y416 + MAP(Y416, YUV444_12, Y416, 0), +#endif }; #undef MAP
With the necessary pixel formats defined, we can now expose support for the remaining 10/12bit combinations that VAAPI can handle. Specifically, we are adding support for: * HEVC ** 12bit 420 ** 10bit 422 ** 12bit 422 ** 10bit 444 ** 12bit 444 * VP9 ** 10bit 422 ** 10bit 444 These obviously require actual hardware support to be usable, but where that exists, it is now enabled. I had to make some adjustments to the encode logic for matching bit depth as the existing code assumed that the picture depth and the pixel format depth were always the same, which is not true for 12bit content which uses 16bit pixel formats. Signed-off-by: Philip Langdale <philipl@overt.org> --- libavcodec/hevcdec.c | 8 ++++++++ libavcodec/vaapi_decode.c | 13 +++++++++++++ libavcodec/vaapi_encode.c | 19 +++++++++++++++---- libavcodec/vaapi_encode_h265.c | 10 ++++++++-- libavcodec/vaapi_encode_vp9.c | 4 +++- libavcodec/vaapi_hevc.c | 11 ++++++++++- libavutil/hwcontext_vaapi.c | 12 ++++++++++++ 7 files changed, 69 insertions(+), 8 deletions(-)