| Message ID | 20220429104505.1747-3-tong1.wu@intel.com |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel,v2,1/5] avutil/hwcontext_d3d11va: fix the uninitialized texture bindflag | expand |
| Context | Check | Description |
|---|---|---|
| yinshiyou/make_loongarch64 | success | Make finished |
| yinshiyou/make_fate_loongarch64 | success | Make fate finished |
| andriy/make_x86 | success | Make finished |
| andriy/make_fate_x86 | success | Make fate finished |
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Tong > Wu > Sent: Friday, April 29, 2022 12:45 PM > To: ffmpeg-devel@ffmpeg.org > Cc: Tong Wu <tong1.wu@intel.com> > Subject: [FFmpeg-devel] [PATCH v2 3/5] avutil/hwcontext_d3d11va: add a > format check for staging texture > > The texDesc.Format needs to be filled in with a corresponding format. > I > add a format check to initialize the format in case sometimes the > ctx->internal->priv is not initialized, such as during the hwmap > process. ctx->internal->priv is D3D11VAFramesContext. When it wouldn't be initialized, then hwmap couldn't work. D3D11VAFramesContext.format is set during d3d11va_frames_init. You would need to find out whether init is not called or whether AVHWFramesContext.sw_format is not (yet) set during init. If that doesn't work out for some reason, I think the next best solution would be to add a 'format parameter' to d3d11va_create_staging_texture() and in d3d11va_transfer_data() (the only caller) do ID3D11Texture2D_GetDesc() on the frame texture ('texture' variable) and pass the returned format to d3d11va_create_staging_texture() Kind regards, softworkz > > $ ffmpeg.exe -y -hwaccel qsv -init_hw_device d3d11va=d3d11 \ > -init_hw_device qsv=qsv@d3d11 -c:v h264_qsv \ > -i input.h264 -vf > "hwmap=derive_device=d3d11va,format=d3d11,hwdownload,format=nv12" \ > -f null - > > Signed-off-by: Tong Wu <tong1.wu@intel.com> > --- > libavutil/hwcontext_d3d11va.c | 16 ++++++++++++++++ > 1 file changed, 16 insertions(+) > > diff --git a/libavutil/hwcontext_d3d11va.c > b/libavutil/hwcontext_d3d11va.c > index db529acbb4..0ec0e07d3a 100644 > --- a/libavutil/hwcontext_d3d11va.c > +++ b/libavutil/hwcontext_d3d11va.c > @@ -349,6 +349,8 @@ static int > d3d11va_create_staging_texture(AVHWFramesContext *ctx) > AVD3D11VADeviceContext *device_hwctx = ctx->device_ctx->hwctx; > D3D11VAFramesContext *s = ctx->internal->priv; > HRESULT hr; > + int i; > + > D3D11_TEXTURE2D_DESC texDesc = { > .Width = ctx->width, > .Height = ctx->height, > @@ -360,6 +362,20 @@ static int > d3d11va_create_staging_texture(AVHWFramesContext *ctx) > .CPUAccessFlags = D3D11_CPU_ACCESS_READ | > D3D11_CPU_ACCESS_WRITE, > }; > > + if (!texDesc.Format) { > + for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++) { > + if (ctx->sw_format == supported_formats[i].pix_fmt) { > + texDesc.Format = supported_formats[i].d3d_format; > + break; > + } > + } > + if (i == FF_ARRAY_ELEMS(supported_formats)) { > + av_log(ctx, AV_LOG_ERROR, "Unsupported pixel format: > %s\n", > + av_get_pix_fmt_name(ctx->sw_format)); > + return AVERROR(EINVAL); > + } > + } > + > hr = ID3D11Device_CreateTexture2D(device_hwctx->device, &texDesc, > NULL, &s->staging_texture); > if (FAILED(hr)) { > av_log(ctx, AV_LOG_ERROR, "Could not create the staging > texture (%lx)\n", (long)hr);
> > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Tong > > Wu > > Sent: Friday, April 29, 2022 12:45 PM > > To: ffmpeg-devel@ffmpeg.org > > Cc: Tong Wu <tong1.wu@intel.com> > > Subject: [FFmpeg-devel] [PATCH v2 3/5] avutil/hwcontext_d3d11va: add a > > format check for staging texture > > > > The texDesc.Format needs to be filled in with a corresponding format. > > I > > add a format check to initialize the format in case sometimes the > > ctx->internal->priv is not initialized, such as during the hwmap > > process. > > ctx->internal->priv is D3D11VAFramesContext. When it wouldn't be > initialized, then hwmap couldn't work. > > D3D11VAFramesContext.format is set during d3d11va_frames_init. > You would need to find out whether init is not called or > whether AVHWFramesContext.sw_format is not (yet) set during > init. > For the hwmap process, the new FramesContext is created in av_hwframe_ctx_create_derived and the init function is never called. > > If that doesn't work out for some reason, I think the next best > solution would be to add a 'format parameter' to > d3d11va_create_staging_texture() and in d3d11va_transfer_data() > (the only caller) do ID3D11Texture2D_GetDesc() on the frame > texture ('texture' variable) and pass the returned format to > d3d11va_create_staging_texture() > I think this solution makes sense. I will resubmit the patch. Thanks for the review. Regards, Tong > Kind regards, > softworkz > > > > > > $ ffmpeg.exe -y -hwaccel qsv -init_hw_device d3d11va=d3d11 \ > > -init_hw_device qsv=qsv@d3d11 -c:v h264_qsv \ > > -i input.h264 -vf > > > "hwmap=derive_device=d3d11va,format=d3d11,hwdownload,format=nv12" > \ > > -f null - > > > > Signed-off-by: Tong Wu <tong1.wu@intel.com> > > --- > > libavutil/hwcontext_d3d11va.c | 16 ++++++++++++++++ > > 1 file changed, 16 insertions(+) > > > > diff --git a/libavutil/hwcontext_d3d11va.c > > b/libavutil/hwcontext_d3d11va.c > > index db529acbb4..0ec0e07d3a 100644 > > --- a/libavutil/hwcontext_d3d11va.c > > +++ b/libavutil/hwcontext_d3d11va.c > > @@ -349,6 +349,8 @@ static int > > d3d11va_create_staging_texture(AVHWFramesContext *ctx) > > AVD3D11VADeviceContext *device_hwctx = ctx->device_ctx->hwctx; > > D3D11VAFramesContext *s = ctx->internal->priv; > > HRESULT hr; > > + int i; > > + > > D3D11_TEXTURE2D_DESC texDesc = { > > .Width = ctx->width, > > .Height = ctx->height, > > @@ -360,6 +362,20 @@ static int > > d3d11va_create_staging_texture(AVHWFramesContext *ctx) > > .CPUAccessFlags = D3D11_CPU_ACCESS_READ | > > D3D11_CPU_ACCESS_WRITE, > > }; > > > > + if (!texDesc.Format) { > > + for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++) { > > + if (ctx->sw_format == supported_formats[i].pix_fmt) { > > + texDesc.Format = supported_formats[i].d3d_format; > > + break; > > + } > > + } > > + if (i == FF_ARRAY_ELEMS(supported_formats)) { > > + av_log(ctx, AV_LOG_ERROR, "Unsupported pixel format: > > %s\n", > > + av_get_pix_fmt_name(ctx->sw_format)); > > + return AVERROR(EINVAL); > > + } > > + } > > + > > hr = ID3D11Device_CreateTexture2D(device_hwctx->device, &texDesc, > > NULL, &s->staging_texture); > > if (FAILED(hr)) { > > av_log(ctx, AV_LOG_ERROR, "Could not create the staging > > texture (%lx)\n", (long)hr); >
diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c index db529acbb4..0ec0e07d3a 100644 --- a/libavutil/hwcontext_d3d11va.c +++ b/libavutil/hwcontext_d3d11va.c @@ -349,6 +349,8 @@ static int d3d11va_create_staging_texture(AVHWFramesContext *ctx) AVD3D11VADeviceContext *device_hwctx = ctx->device_ctx->hwctx; D3D11VAFramesContext *s = ctx->internal->priv; HRESULT hr; + int i; + D3D11_TEXTURE2D_DESC texDesc = { .Width = ctx->width, .Height = ctx->height, @@ -360,6 +362,20 @@ static int d3d11va_create_staging_texture(AVHWFramesContext *ctx) .CPUAccessFlags = D3D11_CPU_ACCESS_READ | D3D11_CPU_ACCESS_WRITE, }; + if (!texDesc.Format) { + for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++) { + if (ctx->sw_format == supported_formats[i].pix_fmt) { + texDesc.Format = supported_formats[i].d3d_format; + break; + } + } + if (i == FF_ARRAY_ELEMS(supported_formats)) { + av_log(ctx, AV_LOG_ERROR, "Unsupported pixel format: %s\n", + av_get_pix_fmt_name(ctx->sw_format)); + return AVERROR(EINVAL); + } + } + hr = ID3D11Device_CreateTexture2D(device_hwctx->device, &texDesc, NULL, &s->staging_texture); if (FAILED(hr)) { av_log(ctx, AV_LOG_ERROR, "Could not create the staging texture (%lx)\n", (long)hr);
The texDesc.Format needs to be filled in with a corresponding format. I add a format check to initialize the format in case sometimes the ctx->internal->priv is not initialized, such as during the hwmap process. $ ffmpeg.exe -y -hwaccel qsv -init_hw_device d3d11va=d3d11 \ -init_hw_device qsv=qsv@d3d11 -c:v h264_qsv \ -i input.h264 -vf "hwmap=derive_device=d3d11va,format=d3d11,hwdownload,format=nv12" \ -f null - Signed-off-by: Tong Wu <tong1.wu@intel.com> --- libavutil/hwcontext_d3d11va.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)