diff mbox series

[FFmpeg-devel,V2] libavutil/hwcontext_qsv: fix a bug for mapping qsv frame to vaapi

Message ID 20210913062222.3221575-1-wenbin.chen@intel.com
State Accepted
Commit f2891fbdeddd9049ba03b600d6c93a1ab732df66
Headers show
Series [FFmpeg-devel,V2] libavutil/hwcontext_qsv: fix a bug for mapping qsv frame to vaapi
Related show

Checks

Context Check Description
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished
andriy/make_ppc success Make finished
andriy/make_fate_ppc success Make fate finished

Commit Message

Chen, Wenbin Sept. 13, 2021, 6:22 a.m. UTC
Command below failed.
ffmpeg -v verbose -init_hw_device vaapi=va:/dev/dri/renderD128
-init_hw_device qsv=qs@va -hwaccel qsv -hwaccel_device qs
-filter_hw_device va -c:v h264_qsv
-i 1080P.264 -vf "hwmap,format=vaapi" -c:v h264_vaapi output.264

Cause: Assign pair->first directly to data[3] in vaapi frame.
pair->first is *VASurfaceID while data[3] in vaapi frame is
VASurfaceID. I fix this line of code. Now the command above works.

Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
---
 libavutil/hwcontext_qsv.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

Comments

Chen, Wenbin Sept. 15, 2021, 2:37 a.m. UTC | #1
> Command below failed.
> ffmpeg -v verbose -init_hw_device vaapi=va:/dev/dri/renderD128
> -init_hw_device qsv=qs@va -hwaccel qsv -hwaccel_device qs
> -filter_hw_device va -c:v h264_qsv
> -i 1080P.264 -vf "hwmap,format=vaapi" -c:v h264_vaapi output.264
> 
> Cause: Assign pair->first directly to data[3] in vaapi frame.
> pair->first is *VASurfaceID while data[3] in vaapi frame is
> VASurfaceID. I fix this line of code. Now the command above works.
> 
> Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
> ---
>  libavutil/hwcontext_qsv.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c
> index d431e71eab..b01236889e 100644
> --- a/libavutil/hwcontext_qsv.c
> +++ b/libavutil/hwcontext_qsv.c
> @@ -781,7 +781,11 @@ static int qsv_map_from(AVHWFramesContext *ctx,
>      case AV_HWDEVICE_TYPE_VAAPI:
>      {
>          mfxHDLPair *pair = (mfxHDLPair*)surf->Data.MemId;
> -        child_data = pair->first;
> +        /* pair->first is *VASurfaceID while data[3] in vaapi frame is
> VASurfaceID, so
> +         * we need this casting for vaapi.
> +         * Add intptr_t to force cast from VASurfaceID(uint) type to
> pointer(long) type
> +         * to avoid compile warning */
> +        child_data = (uint8_t*)(intptr_t)*(VASurfaceID*)pair->first;
>          break;
>      }
>  #endif
> --
> 2.25.1

ping
Chen, Wenbin Sept. 24, 2021, 1:51 a.m. UTC | #2
> Command below failed.
> ffmpeg -v verbose -init_hw_device vaapi=va:/dev/dri/renderD128
> -init_hw_device qsv=qs@va -hwaccel qsv -hwaccel_device qs
> -filter_hw_device va -c:v h264_qsv
> -i 1080P.264 -vf "hwmap,format=vaapi" -c:v h264_vaapi output.264
> 
> Cause: Assign pair->first directly to data[3] in vaapi frame.
> pair->first is *VASurfaceID while data[3] in vaapi frame is
> VASurfaceID. I fix this line of code. Now the command above works.
> 
> Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
> ---
>  libavutil/hwcontext_qsv.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c
> index d431e71eab..b01236889e 100644
> --- a/libavutil/hwcontext_qsv.c
> +++ b/libavutil/hwcontext_qsv.c
> @@ -781,7 +781,11 @@ static int qsv_map_from(AVHWFramesContext *ctx,
>      case AV_HWDEVICE_TYPE_VAAPI:
>      {
>          mfxHDLPair *pair = (mfxHDLPair*)surf->Data.MemId;
> -        child_data = pair->first;
> +        /* pair->first is *VASurfaceID while data[3] in vaapi frame is
> VASurfaceID, so
> +         * we need this casting for vaapi.
> +         * Add intptr_t to force cast from VASurfaceID(uint) type to
> pointer(long) type
> +         * to avoid compile warning */
> +        child_data = (uint8_t*)(intptr_t)*(VASurfaceID*)pair->first;
>          break;
>      }
>  #endif
> --
> 2.25.1

Ping
James Almer Sept. 24, 2021, 1:59 a.m. UTC | #3
On 9/13/2021 3:22 AM, Wenbin Chen wrote:
> Command below failed.
> ffmpeg -v verbose -init_hw_device vaapi=va:/dev/dri/renderD128
> -init_hw_device qsv=qs@va -hwaccel qsv -hwaccel_device qs
> -filter_hw_device va -c:v h264_qsv
> -i 1080P.264 -vf "hwmap,format=vaapi" -c:v h264_vaapi output.264
> 
> Cause: Assign pair->first directly to data[3] in vaapi frame.
> pair->first is *VASurfaceID while data[3] in vaapi frame is
> VASurfaceID. I fix this line of code. Now the command above works.
> 
> Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
> ---
>   libavutil/hwcontext_qsv.c | 6 +++++-
>   1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c
> index d431e71eab..b01236889e 100644
> --- a/libavutil/hwcontext_qsv.c
> +++ b/libavutil/hwcontext_qsv.c
> @@ -781,7 +781,11 @@ static int qsv_map_from(AVHWFramesContext *ctx,
>       case AV_HWDEVICE_TYPE_VAAPI:
>       {
>           mfxHDLPair *pair = (mfxHDLPair*)surf->Data.MemId;
> -        child_data = pair->first;
> +        /* pair->first is *VASurfaceID while data[3] in vaapi frame is VASurfaceID, so
> +         * we need this casting for vaapi.
> +         * Add intptr_t to force cast from VASurfaceID(uint) type to pointer(long) type
> +         * to avoid compile warning */
> +        child_data = (uint8_t*)(intptr_t)*(VASurfaceID*)pair->first;
>           break;
>       }
>   #endif

Applied.
diff mbox series

Patch

diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c
index d431e71eab..b01236889e 100644
--- a/libavutil/hwcontext_qsv.c
+++ b/libavutil/hwcontext_qsv.c
@@ -781,7 +781,11 @@  static int qsv_map_from(AVHWFramesContext *ctx,
     case AV_HWDEVICE_TYPE_VAAPI:
     {
         mfxHDLPair *pair = (mfxHDLPair*)surf->Data.MemId;
-        child_data = pair->first;
+        /* pair->first is *VASurfaceID while data[3] in vaapi frame is VASurfaceID, so
+         * we need this casting for vaapi.
+         * Add intptr_t to force cast from VASurfaceID(uint) type to pointer(long) type
+         * to avoid compile warning */
+        child_data = (uint8_t*)(intptr_t)*(VASurfaceID*)pair->first;
         break;
     }
 #endif