[FFmpeg-devel,v2,1/4] lavc/vaapi_encode: add EQUAL_MULTI_ROWS support for slice structure

VA_ENC_SLICE_STRUCTURE_EQUAL_MULTI_ROWS is added to in the latest
libva (1.8.0) which matches the hardware behaviour:

/** \brief Driver supports any number of rows per slice but they must be the same
*       for all slices except for the last one, which must be equal or smaller
*       to the previous slices. */

And VA_ENC_SLICE_STRUCTURE_EQUAL_ROWS is kind of deprecated for iHD
since it's somehow introduced in [1] however misleading from what we
actually handle, and one row per slice would not get a good quality.

Caps query support in driver is WIP, and this would fix the multi slice
encoding for VAEntrypointEncSliceLP.

[1]<https://github.com/intel/libva/commit/0e6d5441f19bdc674b4da3169d614d10fd644778>

Signed-off-by: Linjie Fu <linjie.fu@intel.com>
---
 libavcodec/vaapi_encode.c | 3 +++
 1 file changed, 3 insertions(+)

On Tue, May 12, 2020 at 9:49 PM Linjie Fu <linjie.fu@intel.com> wrote:
>
> VA_ENC_SLICE_STRUCTURE_EQUAL_MULTI_ROWS is added to in the latest
> libva (1.8.0) which matches the hardware behaviour:
>
> /** \brief Driver supports any number of rows per slice but they must be the same
> *       for all slices except for the last one, which must be equal or smaller
> *       to the previous slices. */
>
> And VA_ENC_SLICE_STRUCTURE_EQUAL_ROWS is kind of deprecated for iHD
> since it's somehow introduced in [1] however misleading from what we
> actually handle, and one row per slice would not get a good quality.
>
> Caps query support in driver is WIP, and this would fix the multi slice
> encoding for VAEntrypointEncSliceLP.
>
> [1]<https://github.com/intel/libva/commit/0e6d5441f19bdc674b4da3169d614d10fd644778>
>
> Signed-off-by: Linjie Fu <linjie.fu@intel.com>
> ---
>  libavcodec/vaapi_encode.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
> index cb05ebd..234618a 100644
> --- a/libavcodec/vaapi_encode.c
> +++ b/libavcodec/vaapi_encode.c
> @@ -1888,6 +1888,9 @@ static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx)
>          req_slices = avctx->slices;
>      }
>      if (slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_ROWS ||
> +#if VA_CHECK_VERSION(1, 8, 0)
> +        slice_structure & VA_ENC_SLICE_STRUCTURE_EQUAL_MULTI_ROWS ||
> +#endif
>          slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_MACROBLOCKS) {
>          ctx->nb_slices  = req_slices;
>          ctx->slice_size = ctx->slice_block_rows / ctx->nb_slices;
> --
> 2.7.4

Full support is provided in libva[1] and media-driver[2], and we've
observed it works for multi slice encoding for AVC. (300+ cases)

Prefer to apply this soon with commit message refined.

- Linjie

[1] https://github.com/intel/libva/commit/0e6d5441f19bdc674b4da3169d614d10fd644778
[2] https://github.com/intel/media-driver/commit/c63d4b1dbed4ed186e4c4ea67a3e3601367d9a8e

On Thu, Jun 18, 2020 at 1:36 PM Linjie Fu <linjie.justin.fu@gmail.com> wrote:
>
> On Tue, May 12, 2020 at 9:49 PM Linjie Fu <linjie.fu@intel.com> wrote:
> >
> > VA_ENC_SLICE_STRUCTURE_EQUAL_MULTI_ROWS is added to in the latest
> > libva (1.8.0) which matches the hardware behaviour:
> >
> > /** \brief Driver supports any number of rows per slice but they must be the same
> > *       for all slices except for the last one, which must be equal or smaller
> > *       to the previous slices. */
> >
> > And VA_ENC_SLICE_STRUCTURE_EQUAL_ROWS is kind of deprecated for iHD
> > since it's somehow introduced in [1] however misleading from what we
> > actually handle, and one row per slice would not get a good quality.
> >
> > Caps query support in driver is WIP, and this would fix the multi slice
> > encoding for VAEntrypointEncSliceLP.
> >
> > [1]<https://github.com/intel/libva/commit/0e6d5441f19bdc674b4da3169d614d10fd644778>
> >
> > Signed-off-by: Linjie Fu <linjie.fu@intel.com>
> > ---
> >  libavcodec/vaapi_encode.c | 3 +++
> >  1 file changed, 3 insertions(+)
> >
> > diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
> > index cb05ebd..234618a 100644
> > --- a/libavcodec/vaapi_encode.c
> > +++ b/libavcodec/vaapi_encode.c
> > @@ -1888,6 +1888,9 @@ static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx)
> >          req_slices = avctx->slices;
> >      }
> >      if (slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_ROWS ||
> > +#if VA_CHECK_VERSION(1, 8, 0)
> > +        slice_structure & VA_ENC_SLICE_STRUCTURE_EQUAL_MULTI_ROWS ||
> > +#endif
> >          slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_MACROBLOCKS) {
> >          ctx->nb_slices  = req_slices;
> >          ctx->slice_size = ctx->slice_block_rows / ctx->nb_slices;
> > --
> > 2.7.4
>
> Full support is provided in libva[1] and media-driver[2], and we've
> observed it works for multi slice encoding for AVC. (300+ cases)
>
> Prefer to apply this soon with commit message refined.
>
Applied, thx.
Prefer to apply the reset of this patch set soon to support tile encoding.

- Linjie

On 19/07/2020 08:00, Linjie Fu wrote:
> On Thu, Jun 18, 2020 at 1:36 PM Linjie Fu <linjie.justin.fu@gmail.com> wrote:
>>
>> On Tue, May 12, 2020 at 9:49 PM Linjie Fu <linjie.fu@intel.com> wrote:
>>>
>>> VA_ENC_SLICE_STRUCTURE_EQUAL_MULTI_ROWS is added to in the latest
>>> libva (1.8.0) which matches the hardware behaviour:
>>>
>>> /** \brief Driver supports any number of rows per slice but they must be the same
>>> *       for all slices except for the last one, which must be equal or smaller
>>> *       to the previous slices. */
>>>
>>> And VA_ENC_SLICE_STRUCTURE_EQUAL_ROWS is kind of deprecated for iHD
>>> since it's somehow introduced in [1] however misleading from what we
>>> actually handle, and one row per slice would not get a good quality.
>>>
>>> Caps query support in driver is WIP, and this would fix the multi slice
>>> encoding for VAEntrypointEncSliceLP.
>>>
>>> [1]<https://github.com/intel/libva/commit/0e6d5441f19bdc674b4da3169d614d10fd644778>
>>>
>>> Signed-off-by: Linjie Fu <linjie.fu@intel.com>
>>> ---
>>>   libavcodec/vaapi_encode.c | 3 +++
>>>   1 file changed, 3 insertions(+)
>>>
>>> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
>>> index cb05ebd..234618a 100644
>>> --- a/libavcodec/vaapi_encode.c
>>> +++ b/libavcodec/vaapi_encode.c
>>> @@ -1888,6 +1888,9 @@ static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx)
>>>           req_slices = avctx->slices;
>>>       }
>>>       if (slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_ROWS ||
>>> +#if VA_CHECK_VERSION(1, 8, 0)
>>> +        slice_structure & VA_ENC_SLICE_STRUCTURE_EQUAL_MULTI_ROWS ||
>>> +#endif
>>>           slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_MACROBLOCKS) {
>>>           ctx->nb_slices  = req_slices;
>>>           ctx->slice_size = ctx->slice_block_rows / ctx->nb_slices;
>>> --
>>> 2.7.4
>>
>> Full support is provided in libva[1] and media-driver[2], and we've
>> observed it works for multi slice encoding for AVC. (300+ cases)
>>
>> Prefer to apply this soon with commit message refined.
>>
> Applied, thx.
Treating EQUAL_MULTI_ROWS in the same way as the arbitrary-size cases is just wrong.

Consider 9 rows, 4 slices - we pick 4 slices with sizes { 3, 2, 2, 2 }, which EQUAL_MULTI_ROWS does not allow.

It isn't possible to split the frame into 4 slices at all with the EQUAL_MULTI_ROWS structure - the closest options are 3 slices with sizes { 3, 3, 3 } or 5 slices with sizes { 2, 2, 2, 2, 1 }.

You can see the same problem at 1080p with five slices.

Thanks,

- Mark

> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Mark Thompson
> Sent: Tuesday, July 28, 2020 05:52
> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/4] lavc/vaapi_encode: add
> EQUAL_MULTI_ROWS support for slice structure
> 
> On 19/07/2020 08:00, Linjie Fu wrote:
> > On Thu, Jun 18, 2020 at 1:36 PM Linjie Fu <linjie.justin.fu@gmail.com>
> wrote:
> >>
> >> On Tue, May 12, 2020 at 9:49 PM Linjie Fu <linjie.fu@intel.com> wrote:
> >>>
> >>> VA_ENC_SLICE_STRUCTURE_EQUAL_MULTI_ROWS is added to in the
> latest
> >>> libva (1.8.0) which matches the hardware behaviour:
> >>>
> >>> /** \brief Driver supports any number of rows per slice but they must
> be the same
> >>> *       for all slices except for the last one, which must be equal or smaller
> >>> *       to the previous slices. */
> >>>
> >>> And VA_ENC_SLICE_STRUCTURE_EQUAL_ROWS is kind of deprecated
> for iHD
> >>> since it's somehow introduced in [1] however misleading from what we
> >>> actually handle, and one row per slice would not get a good quality.
> >>>
> >>> Caps query support in driver is WIP, and this would fix the multi slice
> >>> encoding for VAEntrypointEncSliceLP.
> >>>
> >>>
> [1]<https://github.com/intel/libva/commit/0e6d5441f19bdc674b4da3169d61
> 4d10fd644778>
> >>>
> >>> Signed-off-by: Linjie Fu <linjie.fu@intel.com>
> >>> ---
> >>>   libavcodec/vaapi_encode.c | 3 +++
> >>>   1 file changed, 3 insertions(+)
> >>>
> >>> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
> >>> index cb05ebd..234618a 100644
> >>> --- a/libavcodec/vaapi_encode.c
> >>> +++ b/libavcodec/vaapi_encode.c
> >>> @@ -1888,6 +1888,9 @@ static av_cold int
> vaapi_encode_init_slice_structure(AVCodecContext *avctx)
> >>>           req_slices = avctx->slices;
> >>>       }
> >>>       if (slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_ROWS
> ||
> >>> +#if VA_CHECK_VERSION(1, 8, 0)
> >>> +        slice_structure &
> VA_ENC_SLICE_STRUCTURE_EQUAL_MULTI_ROWS ||
> >>> +#endif
> >>>           slice_structure &
> VA_ENC_SLICE_STRUCTURE_ARBITRARY_MACROBLOCKS) {
> >>>           ctx->nb_slices  = req_slices;
> >>>           ctx->slice_size = ctx->slice_block_rows / ctx->nb_slices;
> >>> --
> >>> 2.7.4
> >>
> >> Full support is provided in libva[1] and media-driver[2], and we've
> >> observed it works for multi slice encoding for AVC. (300+ cases)
> >>
> >> Prefer to apply this soon with commit message refined.
> >>
> > Applied, thx.
> Treating EQUAL_MULTI_ROWS in the same way as the arbitrary-size cases is
> just wrong.
> 
> Consider 9 rows, 4 slices - we pick 4 slices with sizes { 3, 2, 2, 2 }, which
> EQUAL_MULTI_ROWS does not allow.
> 
> It isn't possible to split the frame into 4 slices at all with the
> EQUAL_MULTI_ROWS structure - the closest options are 3 slices with sizes { 3,
> 3, 3 } or 5 slices with sizes { 2, 2, 2, 2, 1 }.
> 
> You can see the same problem at 1080p with five slices.

Umm.. saw the conflicts with the definition/notation in LIBVA.

Actually, I doubt whether Hardware still have such limitation which needs the last
slice to be no larger than the previous. In the case of 9 rows, 4 slices, the current
implementation of rounding works well. {3, 2, 2, 2} .

Had sent a mail to the media-driver/libva scrum team to double confirm this, thx.

- Linjie