| Message ID | 1515021735-11532-1-git-send-email-mont3z.claro5@gmail.com |
|---|---|
| State | New |
| Headers | show |
On Wed, 3 Jan 2018 15:22:15 -0800 Humberto Ribeiro <mont3z.claro5@gmail.com> wrote: > This prevents a possible crash in CreateDeviceEx when using faulty response from GetAdapterDisplayModeEx and allows ffmpeg to fallback to classic d3d9. > --- > libavutil/hwcontext_dxva2.c | 6 +++++- > 1 file changed, 5 insertions(+), 1 deletion(-) > > diff --git a/libavutil/hwcontext_dxva2.c b/libavutil/hwcontext_dxva2.c > index 2ddd4be..44ebdbc 100644 > --- a/libavutil/hwcontext_dxva2.c > +++ b/libavutil/hwcontext_dxva2.c > @@ -485,7 +485,11 @@ static int dxva2_device_create9ex(AVHWDeviceContext *ctx, UINT adapter) > if (FAILED(hr)) > return AVERROR_UNKNOWN; > > - IDirect3D9Ex_GetAdapterDisplayModeEx(d3d9ex, adapter, &modeex, NULL); > + hr = IDirect3D9Ex_GetAdapterDisplayModeEx(d3d9ex, adapter, &modeex, NULL); > + if (FAILED(hr)) { > + IDirect3D9Ex_Release(d3d9ex); > + return AVERROR_UNKNOWN; > + } > > d3dpp.BackBufferFormat = modeex.Format; > Tested, slightly modified your commit message (broke on 72 columns, though I left the overlong subject line alone), and pushed together with my patch.
On Thu, Jan 4, 2018 at 7:49 AM, wm4 <nfxjfg@googlemail.com> wrote: > On Wed, 3 Jan 2018 15:22:15 -0800 > Humberto Ribeiro <mont3z.claro5@gmail.com> wrote: > >> This prevents a possible crash in CreateDeviceEx when using faulty response from GetAdapterDisplayModeEx and allows ffmpeg to fallback to classic d3d9. >> --- ... >> > > Tested, slightly modified your commit message (broke on 72 columns, > though I left the overlong subject line alone), and pushed together > with my patch. > Thx a lot for this. Humberto
diff --git a/libavutil/hwcontext_dxva2.c b/libavutil/hwcontext_dxva2.c index 2ddd4be..44ebdbc 100644 --- a/libavutil/hwcontext_dxva2.c +++ b/libavutil/hwcontext_dxva2.c @@ -485,7 +485,11 @@ static int dxva2_device_create9ex(AVHWDeviceContext *ctx, UINT adapter) if (FAILED(hr)) return AVERROR_UNKNOWN; - IDirect3D9Ex_GetAdapterDisplayModeEx(d3d9ex, adapter, &modeex, NULL); + hr = IDirect3D9Ex_GetAdapterDisplayModeEx(d3d9ex, adapter, &modeex, NULL); + if (FAILED(hr)) { + IDirect3D9Ex_Release(d3d9ex); + return AVERROR_UNKNOWN; + } d3dpp.BackBufferFormat = modeex.Format;