diff mbox series

[FFmpeg-devel,V2,08/10] libavutil/hwcontext_vulkan: fix wrong offset of plane

Message ID 20210831072430.303090-8-wenbin.chen@intel.com
State New
Headers show
Series [FFmpeg-devel,V2,01/10] libavfilter/vulkan: Fix problem when device have queue_count greater than 1
Related show

Checks

Context Check Description
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished
andriy/make_ppc success Make finished
andriy/make_fate_ppc success Make fate finished

Commit Message

Wenbin Chen Aug. 31, 2021, 7:24 a.m. UTC
According to spec, if we use VkBindImagePlaneMemoryInfo to bind image
we mush create image with disjoint flag.
The offset in subresourcelayout is relative to the base address of
the plane, but the offset in drm is relative to the drm objectis so
I think this offset should be 0.
Also, when I import vaapi frame to vulkan I got broken frame, and
setting plane_data->offset to 0 makes command works.

Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
---
 libavutil/hwcontext_vulkan.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

Comments

Lynne Aug. 31, 2021, 11:33 a.m. UTC | #1
31 Aug 2021, 09:24 by wenbin.chen@intel.com:

> According to spec, if we use VkBindImagePlaneMemoryInfo to bind image
> we mush create image with disjoint flag.
> The offset in subresourcelayout is relative to the base address of
> the plane, but the offset in drm is relative to the drm objectis so
> I think this offset should be 0.
> Also, when I import vaapi frame to vulkan I got broken frame, and
> setting plane_data->offset to 0 makes command works.
>
> Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
>

I'm not too sure about that. I'll need to test the whole import/export code on AMD first,
and that may take some time.
Lynne Aug. 31, 2021, 11:41 a.m. UTC | #2
31 Aug 2021, 09:24 by wenbin.chen@intel.com:

> According to spec, if we use VkBindImagePlaneMemoryInfo to bind image
> we mush create image with disjoint flag.
> The offset in subresourcelayout is relative to the base address of
> the plane, but the offset in drm is relative to the drm objectis so
> I think this offset should be 0.
> Also, when I import vaapi frame to vulkan I got broken frame, and
> setting plane_data->offset to 0 makes command works.
>
> Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
>

I don't quite get the point of this. Is there some situation where
clients may want to change the modifier of the image they're allocating,
like maybe use a different modifier when allocating images that would
be used as framebuffers to be presented?
Wenbin Chen Sept. 1, 2021, 1:40 a.m. UTC | #3
> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Lynne
> Sent: Tuesday, August 31, 2021 7:42 PM
> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [V2 08/10] libavutil/hwcontext_vulkan: fix wrong
> offset of plane
> 
> 31 Aug 2021, 09:24 by wenbin.chen@intel.com:
> 
> > According to spec, if we use VkBindImagePlaneMemoryInfo to bind image
> > we mush create image with disjoint flag.
> > The offset in subresourcelayout is relative to the base address of
> > the plane, but the offset in drm is relative to the drm objectis so
> > I think this offset should be 0.
> > Also, when I import vaapi frame to vulkan I got broken frame, and
> > setting plane_data->offset to 0 makes command works.
> >
> > Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
> >
> 
> I don't quite get the point of this. Is there some situation where
> clients may want to change the modifier of the image they're allocating,
> like maybe use a different modifier when allocating images that would
> be used as framebuffers to be presented?

Maybe you reply to the wrong patch? I assume you want to reply to the 9th patch.
" specify the modifier to create VKImage". 
I tried on the latest intel driver. The log recommend to use linear or drmModifier to create image.
Also if I use optimal to create image, the export drm modifier is unknown.
Wenbin Chen Sept. 6, 2021, 1:26 a.m. UTC | #4
> -----Original Message-----
> From: Lynne <dev@lynne.ee>
> Sent: Wednesday, September 1, 2021 7:33 PM
> To: Chen, Wenbin <wenbin.chen@intel.com>
> Subject: RE: [FFmpeg-devel] [V2 08/10] libavutil/hwcontext_vulkan: fix wrong
> offset of plane
> 
> 1 Sept 2021, 03:40 by wenbin.chen@intel.com:
> 
> >
> >
> >> -----Original Message-----
> >> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> >> Lynne
> >> Sent: Tuesday, August 31, 2021 7:42 PM
> >> To: FFmpeg development discussions and patches <ffmpeg-
> >> devel@ffmpeg.org>
> >> Subject: Re: [FFmpeg-devel] [V2 08/10] libavutil/hwcontext_vulkan: fix
> wrong
> >> offset of plane
> >>
> >> 31 Aug 2021, 09:24 by wenbin.chen@intel.com:
> >>
> >> > According to spec, if we use VkBindImagePlaneMemoryInfo to bind
> image
> >> > we mush create image with disjoint flag.
> >> > The offset in subresourcelayout is relative to the base address of
> >> > the plane, but the offset in drm is relative to the drm objectis so
> >> > I think this offset should be 0.
> >> > Also, when I import vaapi frame to vulkan I got broken frame, and
> >> > setting plane_data->offset to 0 makes command works.
> >> >
> >> > Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
> >> >
> >>
> >> I don't quite get the point of this. Is there some situation where
> >> clients may want to change the modifier of the image they're allocating,
> >> like maybe use a different modifier when allocating images that would
> >> be used as framebuffers to be presented?
> >>
> >
> > Maybe you reply to the wrong patch? I assume you want to reply to the 9th
> patch.
> > " specify the modifier to create VKImage".
> > I tried on the latest intel driver. The log recommend to use linear or
> drmModifier to create image.
> > Also if I use optimal to create image, the export drm modifier is unknown.
> >
> 
> Linear is almost never optimal. Could you post a link to the recommendation?
> The DRM modifier is known for optimal, there's a Vulkan API to retrieve
> image modifiers.

This restriction is for VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT.

I get recommendation information from here.
https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/intel/vulkan/anv_formats.c#L1412
If I use optimal to create VkImage, the modifier I get from VkGetImageDrmFormatModifierPropertiesExt() 
is DRM_FORMAT_MOD_INVALID.
I tested this on Intel platform, and I see AMD has this restriction as well.
https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd/vulkan/radv_formats.c#L1530
diff mbox series

Patch

diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c
index 4983518a77..3a639c997b 100644
--- a/libavutil/hwcontext_vulkan.c
+++ b/libavutil/hwcontext_vulkan.c
@@ -2382,7 +2382,9 @@  static int vulkan_map_from_drm_frame_desc(AVHWFramesContext *hwfc, AVVkFrame **f
             .extent.depth          = 1,
             .mipLevels             = 1,
             .arrayLayers           = 1,
-            .flags                 = VK_IMAGE_CREATE_ALIAS_BIT,
+            .flags                 = VK_IMAGE_CREATE_ALIAS_BIT |
+                                     (has_modifiers && planes > 1) ? VK_IMAGE_CREATE_DISJOINT_BIT :
+                                     0,
             .tiling                = f->tiling,
             .initialLayout         = VK_IMAGE_LAYOUT_UNDEFINED, /* specs say so */
             .usage                 = frames_hwctx->usage,
@@ -2397,7 +2399,7 @@  static int vulkan_map_from_drm_frame_desc(AVHWFramesContext *hwfc, AVVkFrame **f
                      hwfc->sw_format, src->width, src->height, i);
 
         for (int j = 0; j < planes; j++) {
-            plane_data[j].offset     = desc->layers[i].planes[j].offset;
+            plane_data[j].offset     = 0;
             plane_data[j].rowPitch   = desc->layers[i].planes[j].pitch;
             plane_data[j].size       = 0; /* The specs say so for all 3 */
             plane_data[j].arrayPitch = 0;