[FFmpeg-devel] lavc/amfenc: DXVA2 textures support implementation by AMF encoder

Submitted by Alexander Kravchenko on April 13, 2018, 9:21 a.m.

Details

Message ID 20180413092112.20468-1-akravchenko188@gmail.com
State New
Headers show

Commit Message

Alexander Kravchenko April 13, 2018, 9:21 a.m.
This patch contains DXVA2 textures support implementation by AMF encoder (in addition of D3D11 textures)

Samples of usage:
DXVA2 decoder -> dxva2_vld texture -> AMF Encoder:
ffmpeg -hwaccel dxva2 -hwaccel_output_format dxva2_vld -extra_hw_frames 16 -i input.mp4 -an -c:v h264_amf out.mkv

D3D11va decoder -> d3d11 texture -> AMF Encoder:
ffmpeg -hwaccel d3d11va -hwaccel_output_format d3d11 -extra_hw_frames 16 -i input.mp4 -an -c:v h264_amf out.mkv

---
Sending updated patch (Fixes according Mark's review):
> > ---
>    ^
> (When adding commentary which isn't part of the commit message to an email please place it after this line so that it doesn't end up in the commit message.)
Done here, hopefully correctly

> >      { AV_PIX_FMT_D3D11,      AMF_SURFACE_NV12 },
> > +    { AV_PIX_FMT_DXVA2_VLD,  AMF_SURFACE_NV12 },
> 
> As with D3D11, this isn't necessarily true.  This was ignored before, but do you have any plan for how P010 (and others?) will be handled here?
removed HW types from format map, and added logic reading pixel format from avctx->sw_pix_fmt in case if avctx->pix_fmt is HWACCEL type

> +static void get_dx9_device_from_devmgr(IDirect3DDeviceManager9 *devmgr, IDirect3DDevice9 **device, void *avcl) {
> ...
> Might be cleaner using an error return rather than the null device?
Fixed

> Everything using D3D9 types needs to be inside CONFIG_DXVA2
Fixed

> Passing NULL here will make this case succeed in cases where it shouldn't, I think?
Agree, fixed

> Tbh I don't think this fallback case should exist at all, it should just fail.
> Is there any use-case for having it?  The user passed a DXVA2 frames context on a 
> non-AMD device and expects it to work with that hardware input, this fallback makes 
> it kindof work with at least two copies in a way which is likely to be very slow.  
> Even if the user does want to do that, it would be better for them to do it explicitly 
> to ensure that they aware of the problem.  (We don't automatically do this in any other case.)
Agree, fixed

> Spurious whitespace.
Fixed in changed blocks/functions

> Tested on Windows 7, works well.  
> Unlike with D3D11 the OpenCL interop works properly as well, 
> so e.g. -vf 'hwmap=derive_device=opencl,convolution_opencl=0 1 0 1 -4 1 0 1 0,hwmap=derive_device=dxva2:reverse=1:extra_hw_frames=16' as encoder input works too.
Could you send the samples (or link if they are published, I will add to my tests and will check OpenCL interop with D3D11)



 libavcodec/amfenc.c | 158 +++++++++++++++++++++++++++++++++++++++++-----------
 1 file changed, 126 insertions(+), 32 deletions(-)

Comments

Mark Thompson April 14, 2018, 4:15 p.m.
On 13/04/18 10:21, Alexander Kravchenko wrote:
> 
> This patch contains DXVA2 textures support implementation by AMF encoder (in addition of D3D11 textures)
> 
> Samples of usage:
> DXVA2 decoder -> dxva2_vld texture -> AMF Encoder:
> ffmpeg -hwaccel dxva2 -hwaccel_output_format dxva2_vld -extra_hw_frames 16 -i input.mp4 -an -c:v h264_amf out.mkv
> 
> D3D11va decoder -> d3d11 texture -> AMF Encoder:
> ffmpeg -hwaccel d3d11va -hwaccel_output_format d3d11 -extra_hw_frames 16 -i input.mp4 -an -c:v h264_amf out.mkv
> 
> ---
> Sending updated patch (Fixes according Mark's review):
>>> ---
>>    ^
>> (When adding commentary which isn't part of the commit message to an email please place it after this line so that it doesn't end up in the commit message.)
> Done here, hopefully correctly
> 
>>>      { AV_PIX_FMT_D3D11,      AMF_SURFACE_NV12 },
>>> +    { AV_PIX_FMT_DXVA2_VLD,  AMF_SURFACE_NV12 },
>>
>> As with D3D11, this isn't necessarily true.  This was ignored before, but do you have any plan for how P010 (and others?) will be handled here?
> removed HW types from format map, and added logic reading pixel format from avctx->sw_pix_fmt in case if avctx->pix_fmt is HWACCEL type
> 
>> +static void get_dx9_device_from_devmgr(IDirect3DDeviceManager9 *devmgr, IDirect3DDevice9 **device, void *avcl) {
>> ...
>> Might be cleaner using an error return rather than the null device?
> Fixed
> 
>> Everything using D3D9 types needs to be inside CONFIG_DXVA2
> Fixed
> 
>> Passing NULL here will make this case succeed in cases where it shouldn't, I think?
> Agree, fixed
> 
>> Tbh I don't think this fallback case should exist at all, it should just fail.
>> Is there any use-case for having it?  The user passed a DXVA2 frames context on a 
>> non-AMD device and expects it to work with that hardware input, this fallback makes 
>> it kindof work with at least two copies in a way which is likely to be very slow.  
>> Even if the user does want to do that, it would be better for them to do it explicitly 
>> to ensure that they aware of the problem.  (We don't automatically do this in any other case.)
> Agree, fixed
> 
>> Spurious whitespace.
> Fixed in changed blocks/functions

I've sent a new set containing this patch as 3 and 4 (I split out the format check part, since that doesn't have anything to do with DXVA2), which hopefully makes the initialisation and surface mapping setup cleaner.  Would you like to look at that and comment?  Quite a bit of stuff got moved around in the merge.

>> Tested on Windows 7, works well.  
>> Unlike with D3D11 the OpenCL interop works properly as well, 
>> so e.g. -vf 'hwmap=derive_device=opencl,convolution_opencl=0 1 0 1 -4 1 0 1 0,hwmap=derive_device=dxva2:reverse=1:extra_hw_frames=16' as encoder input works too.
> Could you send the samples (or link if they are published, I will add to my tests and will check OpenCL interop with D3D11)

Use any OpenCL filter with mapping to/from DXVA2.  For example:

./ffmpeg_g -y -hwaccel dxva2 -hwaccel_output_format dxva2_vld -i input.mkv -an -vf 'hwmap=derive_device=opencl,convolution_opencl=0 1 0 1 -4 1 0 1 0,hwmap=derive_device=dxva2:reverse=1:extra_hw_frames=16' -c:v h264_amf output.mkv

(Applies an edge-detect convolution on the luma plane.)

The source filter is also usable with a little trickiness to get the right setup:

./ffmpeg_g -y -init_hw_device dxva2=d3d -init_hw_device opencl=cl@d3d -filter_hw_device cl -filter_complex 'openclsrc=source=sierpinski.cl:kernel=sierpinski_carpet:size=1920x1080:format=nv12,hwmap=derive_device=dxva2:reverse=1:extra_hw_frames=16' -c:v h264_amf output.mkv

(Using the sierpinski carpet example from the documentation.)

Note that OpenCL <-> D3D11 won't work on AMD for normal video surfaces (NV12) because there is no support for multiple-plane textures, so it's only going to work with DXVA2 currently.  Intel has an extension ("cl_intel_d3d11_nv12_media_sharing") which adds a simple hack overloading the subresource index and therefore it is usable on Intel GPUs, but other vendors don't have that.

(There should probably be a wiki page on all of this.  I've never got around to writing it.)

Thanks,

- Mark
Alexander Kravchenko April 14, 2018, 9:01 p.m.
> -----Original Message-----
> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf Of Mark Thompson
> Sent: Saturday, April 14, 2018 7:15 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH] lavc/amfenc: DXVA2 textures support implementation by AMF encoder
> 
> 
> I've sent a new set containing this patch as 3 and 4 (I split out the format check part, since that doesn't have anything to do with
> DXVA2), which hopefully makes the initialisation and surface mapping setup cleaner.  Would you like to look at that and comment?
> Quite a bit of stuff got moved around in the merge.
> 


Hi Mark,
I briefly read the patches, they look good
But I tried to apply them locally
The first one failed
1) I cloned ffmpeg
2) pasted patch text to file p1.patch
3) git apply p1.patch

error: patch failed: libavcodec/amfenc.c:152
error: libavcodec/amfenc.c: patch does not apply

it look like patch expexts "amf_load_library(AVCodecContext *avctx)" at line 152
@@ -152,10 +152,30 @@ static int amf_load_library(AVCodecContext *avctx)

In github
https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/amfenc.c
it is on line 155

did I miss something?

Thanks,
Alexander
Alexander Kravchenko April 14, 2018, 9:44 p.m.
> -----Original Message-----
> From: Alexander Kravchenko [mailto:akravchenko188@gmail.com]
> Sent: Sunday, April 15, 2018 12:02 AM
> To: 'FFmpeg development discussions and patches' <ffmpeg-devel@ffmpeg.org>
> Subject: RE: [FFmpeg-devel] [PATCH] lavc/amfenc: DXVA2 textures support implementation by AMF encoder
> 
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf
> > Of Mark Thompson
> > Sent: Saturday, April 14, 2018 7:15 PM
> > To: ffmpeg-devel@ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH] lavc/amfenc: DXVA2 textures
> > support implementation by AMF encoder
> >
> >
> > I've sent a new set containing this patch as 3 and 4 (I split out the
> > format check part, since that doesn't have anything to do with DXVA2), which hopefully makes the initialisation and surface mapping
> setup cleaner.  Would you like to look at that and comment?
> > Quite a bit of stuff got moved around in the merge.
> >
> 
> 
> Hi Mark,
> I briefly read the patches, they look good But I tried to apply them locally The first one failed
> 1) I cloned ffmpeg
> 2) pasted patch text to file p1.patch
> 3) git apply p1.patch
> 
> error: patch failed: libavcodec/amfenc.c:152
> error: libavcodec/amfenc.c: patch does not apply
> 
> it look like patch expexts "amf_load_library(AVCodecContext *avctx)" at line 152 @@ -152,10 +152,30 @@ static int
> amf_load_library(AVCodecContext *avctx)
> 
> In github
> https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/amfenc.c
> it is on line 155
> 
> did I miss something?
> 
> Thanks,
> Alexander
> 

I managed to apply patch, patch was broken by Far Manager Editor.
Pasting patch text to nano solved the applying problem.
Ruiling Song April 19, 2018, 2:18 a.m.
> Note that OpenCL <-> D3D11 won't work on AMD for normal video surfaces

> (NV12) because there is no support for multiple-plane textures, so it's only going

> to work with DXVA2 currently.  Intel has an extension

> ("cl_intel_d3d11_nv12_media_sharing") which adds a simple hack overloading

> the subresource index and therefore it is usable on Intel GPUs, but other vendors

> don't have that.


For OpenCL NV12 support, I think we can use two separate images as arguments,
one image for Y plane, and another image for UV plane.
I think AMD OpenCL should support (CL_RG + CL_UNORM_INT8), right?
So, we can get same behavior across different OpenCL vendors.

Ruiling
> 

> (There should probably be a wiki page on all of this.  I've never got around to

> writing it.)

> 

> Thanks,

> 

> - Mark

> _______________________________________________

> ffmpeg-devel mailing list

> ffmpeg-devel@ffmpeg.org

> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Patch hide | download patch | download mbox

diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c
index b9418b6791..7cdf17a972 100644
--- a/libavcodec/amfenc.c
+++ b/libavcodec/amfenc.c
@@ -24,6 +24,9 @@ 
 #if CONFIG_D3D11VA
 #include "libavutil/hwcontext_d3d11va.h"
 #endif
+#if CONFIG_DXVA2
+#include "libavutil/hwcontext_dxva2.h"
+#endif
 #include "libavutil/mem.h"
 #include "libavutil/pixdesc.h"
 #include "libavutil/time.h"
@@ -50,6 +53,9 @@  const enum AVPixelFormat ff_amf_pix_fmts[] = {
     AV_PIX_FMT_YUV420P,
 #if CONFIG_D3D11VA
     AV_PIX_FMT_D3D11,
+#endif
+#if CONFIG_DXVA2
+    AV_PIX_FMT_DXVA2_VLD,
 #endif
     AV_PIX_FMT_NONE
 };
@@ -68,7 +74,6 @@  static const FormatMap format_map[] =
     { AV_PIX_FMT_GRAY8,      AMF_SURFACE_GRAY8 },
     { AV_PIX_FMT_YUV420P,    AMF_SURFACE_YUV420P },
     { AV_PIX_FMT_YUYV422,    AMF_SURFACE_YUY2 },
-    { AV_PIX_FMT_D3D11,      AMF_SURFACE_NV12 },
 };
 
 
@@ -152,6 +157,26 @@  static int amf_load_library(AVCodecContext *avctx)
     return 0;
 }
 
+#if CONFIG_DXVA2
+static HRESULT get_dx9_device_from_devmgr(IDirect3DDeviceManager9 *devmgr, IDirect3DDevice9 **device, void *avcl)
+{
+    HRESULT hr;
+    HANDLE device_handle;
+
+    if (SUCCEEDED(hr = devmgr->lpVtbl->OpenDeviceHandle(devmgr, &device_handle))) {
+        if (SUCCEEDED(hr = devmgr->lpVtbl->LockDevice(devmgr, device_handle, device, FALSE))) {
+            devmgr->lpVtbl->UnlockDevice(devmgr, device_handle, FALSE);
+        } else {
+            av_log(avcl, AV_LOG_INFO, "Failed to lock device handle for Direct3D9 device: %lx.\n", (unsigned long)hr);
+        }
+        devmgr->lpVtbl->CloseDeviceHandle(devmgr, device_handle);
+    } else {
+        av_log(avcl, AV_LOG_INFO, "Failed to open device handle for Direct3D9 device: %lx.\n", (unsigned long)hr);
+    }
+    return hr;
+}
+#endif
+
 static int amf_init_context(AVCodecContext *avctx)
 {
     AmfContext         *ctx = avctx->priv_data;
@@ -177,34 +202,61 @@  static int amf_init_context(AVCodecContext *avctx)
     res = ctx->factory->pVtbl->CreateContext(ctx->factory, &ctx->context);
     AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext() failed with error %d\n", res);
     // try to reuse existing DX device
-#if CONFIG_D3D11VA
     if (avctx->hw_frames_ctx) {
-        AVHWFramesContext *device_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data;
-        if (device_ctx->device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) {
-            if (amf_av_to_amf_format(device_ctx->sw_format) != AMF_SURFACE_UNKNOWN) {
-                if (device_ctx->device_ctx->hwctx) {
-                    AVD3D11VADeviceContext *device_d3d11 = (AVD3D11VADeviceContext *)device_ctx->device_ctx->hwctx;
+        AVHWFramesContext *frames_ctx = (AVHWFramesContext *)avctx->hw_frames_ctx->data;
+        if (amf_av_to_amf_format(frames_ctx->sw_format) != AMF_SURFACE_UNKNOWN) {
+            if (frames_ctx->device_ctx->hwctx) {
+#if CONFIG_D3D11VA
+                if (frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) {
+                    AVD3D11VADeviceContext *device_d3d11 = (AVD3D11VADeviceContext *)frames_ctx->device_ctx->hwctx;
                     res = ctx->context->pVtbl->InitDX11(ctx->context, device_d3d11->device, AMF_DX11_1);
                     if (res == AMF_OK) {
                         ctx->hw_frames_ctx = av_buffer_ref(avctx->hw_frames_ctx);
                         if (!ctx->hw_frames_ctx) {
                             return AVERROR(ENOMEM);
                         }
-                        if (device_ctx->initial_pool_size > 0)
-                            ctx->hwsurfaces_in_queue_max = device_ctx->initial_pool_size - 1;
+                        if (frames_ctx->initial_pool_size > 0)
+                            ctx->hwsurfaces_in_queue_max = frames_ctx->initial_pool_size - 1;
                     } else {
-                        if(res == AMF_NOT_SUPPORTED)
-                            av_log(avctx, AV_LOG_INFO, "avctx->hw_frames_ctx has D3D11 device which doesn't have D3D11VA interface, switching to default\n");
+                        if (res == AMF_NOT_SUPPORTED)
+                            av_log(avctx, AV_LOG_INFO, "avctx->hw_frames_ctx has D3D11 device which doesn't have D3D11VA interface\n");
                         else
-                            av_log(avctx, AV_LOG_INFO, "avctx->hw_frames_ctx has non-AMD device, switching to default\n");
+                            av_log(avctx, AV_LOG_INFO, "avctx->hw_frames_ctx has non-AMD device\n");
+                        return AVERROR(ENODEV);
                     }
                 }
-            } else {
-                av_log(avctx, AV_LOG_INFO, "avctx->hw_frames_ctx has format not uspported by AMF, switching to default\n");
+#endif
+#if CONFIG_DXVA2
+                if (frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_DXVA2) {
+                    AVDXVA2DeviceContext *device_dxva2 = (AVDXVA2DeviceContext *)frames_ctx->device_ctx->hwctx;
+                    IDirect3DDevice9 *device_dx9 = NULL;
+                    if(SUCCEEDED(get_dx9_device_from_devmgr(device_dxva2->devmgr, &device_dx9, avctx))) {
+                        res = ctx->context->pVtbl->InitDX9(ctx->context, device_dx9);
+                        device_dx9->lpVtbl->Release(device_dx9);
+                        if (res == AMF_OK) {
+                            ctx->hw_frames_ctx = av_buffer_ref(avctx->hw_frames_ctx);
+                            if (!ctx->hw_frames_ctx) {
+                                return AVERROR(ENOMEM);
+                            }
+                            if (frames_ctx->initial_pool_size > 0)
+                                ctx->hwsurfaces_in_queue_max = frames_ctx->initial_pool_size - 1;
+                        } else {
+                            if (res == AMF_NOT_SUPPORTED)
+                                av_log(avctx, AV_LOG_INFO, "avctx->hw_frames_ctx has D3D device which doesn't have DXVA2 interface\n");
+                            else
+                                av_log(avctx, AV_LOG_INFO, "avctx->hw_frames_ctx has non-AMD device\n");
+                            return AVERROR(ENODEV);
+                        }
+                    }
+                }
+#endif
             }
+        } else {
+            av_log(avctx, AV_LOG_INFO, "avctx->hw_frames_ctx has format not uspported by AMF\n");
         }
     } else if (avctx->hw_device_ctx) {
-        AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)(avctx->hw_device_ctx->data);
+        AVHWDeviceContext *device_ctx = (AVHWDeviceContext *)(avctx->hw_device_ctx->data);
+#if CONFIG_D3D11VA
         if (device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) {
             if (device_ctx->hwctx) {
                 AVD3D11VADeviceContext *device_d3d11 = (AVD3D11VADeviceContext *)device_ctx->hwctx;
@@ -216,19 +268,42 @@  static int amf_init_context(AVCodecContext *avctx)
                     }
                 } else {
                     if (res == AMF_NOT_SUPPORTED)
-                        av_log(avctx, AV_LOG_INFO, "avctx->hw_device_ctx has D3D11 device which doesn't have D3D11VA interface, switching to default\n");
+                        av_log(avctx, AV_LOG_INFO, "avctx->hw_device_ctx has D3D11 device which doesn't have D3D11VA interface\n");
                     else
-                        av_log(avctx, AV_LOG_INFO, "avctx->hw_device_ctx has non-AMD device, switching to default\n");
+                        av_log(avctx, AV_LOG_INFO, "avctx->hw_device_ctx has non-AMD device\n");
+                    return AVERROR(ENODEV);
+                }
+            }
+        }
+#endif        
+#if CONFIG_DXVA2
+        if (device_ctx->type == AV_HWDEVICE_TYPE_DXVA2) {
+            AVDXVA2DeviceContext *device_dxva2 = (AVDXVA2DeviceContext *)device_ctx->hwctx;
+            IDirect3DDevice9 *device_dx9 = NULL;
+            if(SUCCEEDED(get_dx9_device_from_devmgr(device_dxva2->devmgr, &device_dx9, avctx))) {
+                res = ctx->context->pVtbl->InitDX9(ctx->context, device_dx9);
+                device_dx9->lpVtbl->Release(device_dx9);
+                if (res == AMF_OK) {
+                    ctx->hw_device_ctx = av_buffer_ref(avctx->hw_device_ctx);
+                    if (!ctx->hw_device_ctx) {
+                        return AVERROR(ENOMEM);
+                    }
+                } else {
+                    if (res == AMF_NOT_SUPPORTED)
+                        av_log(avctx, AV_LOG_INFO, "avctx->hw_device_ctx has D3D device which doesn't have DXVA2 interface\n");
+                    else
+                        av_log(avctx, AV_LOG_INFO, "avctx->hw_device_ctx has non-AMD device\n");
+                    return AVERROR(ENODEV);
                 }
             }
         }
-    }
 #endif
+    }
     if (!ctx->hw_frames_ctx && !ctx->hw_device_ctx) {
         res = ctx->context->pVtbl->InitDX11(ctx->context, NULL, AMF_DX11_1);
         if (res != AMF_OK) {
             res = ctx->context->pVtbl->InitDX9(ctx->context, NULL);
-            AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "InitDX9() failed with error %d\n", res);
+            AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENODEV), "InitDX9() failed with error %d\n", res);
         }
     }
     return 0;
@@ -239,6 +314,7 @@  static int amf_init_encoder(AVCodecContext *avctx)
     AmfContext          *ctx = avctx->priv_data;
     const wchar_t       *codec_id = NULL;
     AMF_RESULT           res = AMF_OK;
+    enum AVPixelFormat   pix_fmt;
 
     switch (avctx->codec->id) {
         case AV_CODEC_ID_H264:
@@ -252,8 +328,13 @@  static int amf_init_encoder(AVCodecContext *avctx)
     }
     AMF_RETURN_IF_FALSE(ctx, codec_id != NULL, AVERROR(EINVAL), "Codec %d is not supported\n", avctx->codec->id);
 
-    ctx->format = amf_av_to_amf_format(avctx->pix_fmt);
-    AMF_RETURN_IF_FALSE(ctx, ctx->format != AMF_SURFACE_UNKNOWN, AVERROR(EINVAL), "Format %d is not supported\n", avctx->pix_fmt);
+    if(is_hwaccel_pix_fmt(avctx->pix_fmt)) {
+        pix_fmt = avctx->sw_pix_fmt;
+    } else {
+        pix_fmt = avctx->pix_fmt;
+    }
+    ctx->format = amf_av_to_amf_format(pix_fmt);
+    AMF_RETURN_IF_FALSE(ctx, ctx->format != AMF_SURFACE_UNKNOWN, AVERROR(EINVAL), "Format %d is not supported\n", pix_fmt);
 
     res = ctx->factory->pVtbl->CreateComponent(ctx->factory, ctx->context, codec_id, &ctx->encoder);
     AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_ENCODER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", codec_id, res);
@@ -555,22 +636,35 @@  int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame)
             // check if the same hw_frames_ctx as used in initialization
             (ctx->hw_frames_ctx && frame->hw_frames_ctx->data == ctx->hw_frames_ctx->data) ||
             // check if the same hw_device_ctx as used in initialization
-            (ctx->hw_device_ctx && ((AVHWFramesContext*)frame->hw_frames_ctx->data)->device_ctx ==
-            (AVHWDeviceContext*)ctx->hw_device_ctx->data)
+            (ctx->hw_device_ctx && ((AVHWFramesContext *)frame->hw_frames_ctx->data)->device_ctx ==
+            (AVHWDeviceContext *)ctx->hw_device_ctx->data)
         )) {
             AMFBuffer *frame_ref_storage_buffer;
-
 #if CONFIG_D3D11VA
-            static const GUID AMFTextureArrayIndexGUID = { 0x28115527, 0xe7c3, 0x4b66, { 0x99, 0xd3, 0x4f, 0x2a, 0xe6, 0xb4, 0x7f, 0xaf } };
-            ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; // actual texture
-            int index = (int)(size_t)frame->data[1]; // index is a slice in texture array is - set to tell AMF which slice to use
-            texture->lpVtbl->SetPrivateData(texture, &AMFTextureArrayIndexGUID, sizeof(index), &index);
+            if (((AVHWFramesContext *)frame->hw_frames_ctx->data)->device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) {
+                static const GUID AMFTextureArrayIndexGUID = { 0x28115527, 0xe7c3, 0x4b66, { 0x99, 0xd3, 0x4f, 0x2a, 0xe6, 0xb4, 0x7f, 0xaf } };
+
+                ID3D11Texture2D *texture = (ID3D11Texture2D *)frame->data[0]; // actual texture
+                int index = (int)(size_t)frame->data[1]; // index is a slice in texture array is - set to tell AMF which slice to use
+                texture->lpVtbl->SetPrivateData(texture, &AMFTextureArrayIndexGUID, sizeof(index), &index);
 
-            res = ctx->context->pVtbl->CreateSurfaceFromDX11Native(ctx->context, texture, &surface, NULL); // wrap to AMF surface
-            AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX11Native() failed  with error %d\n", res);
+                res = ctx->context->pVtbl->CreateSurfaceFromDX11Native(ctx->context, texture, &surface, NULL); // wrap to AMF surface
+                AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX11Native() failed  with error %d\n", res);
 
-            // input HW surfaces can be vertically aligned by 16; tell AMF the real size
-            surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height);
+                // input HW surfaces can be vertically aligned by 16; tell AMF the real size
+                surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height);
+            }
+#endif
+#if CONFIG_DXVA2
+            if (((AVHWFramesContext *)frame->hw_frames_ctx->data)->device_ctx->type == AV_HWDEVICE_TYPE_DXVA2) {
+                IDirect3DSurface9 *texture = (IDirect3DSurface9 *)frame->data[3]; // actual texture
+
+                res = ctx->context->pVtbl->CreateSurfaceFromDX9Native(ctx->context, texture, &surface, NULL); // wrap to AMF surface
+                AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX9Native() failed  with error %d\n", res);
+
+                // input HW surfaces can be vertically aligned by 16; tell AMF the real size
+                surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height);
+            }
 #endif
 
             frame_ref_storage_buffer = amf_create_buffer_with_frame_ref(frame, ctx->context);