| Message ID | d2ad0a3d-b866-a311-39e9-61da969cc696@aracnet.com |
|---|---|
| State | Accepted |
| Commit | 3d040513a1de4797a4f81dde4984395f51db76b7 |
| Headers | show |
Please disregard--I will submit a new patch with a better commit message. Aaron On 5/16/2017 4:09 AM, Aaron Levinson wrote: > Purpose: No longer improperly freeing IDirect3DSurface9 objects in > hwcontext_dxva2.c. Added dxva2_pool_release_dummy() and using it in > call to av_buffer_create() in dxva2_pool_alloc(). Prior to this > change, av_buffer_create() was called with NULL for the third > argument, which indicates that av_buffer_default_free() should be used > to free the buffer's data. Eventually, it gets to buffer_pool_free() > and calls buf->free() on a surface object (which is > av_buffer_default_free()). This can result in a crash when the debug > version of the C-runtime is used on Windows. While it doesn't appear > to result in a crash when the release version of the C-runtime is used > on Windows, it likely results in memory corruption, since av_free() is > being called on memory that was allocated using > IDirectXVideoAccelerationService::CreateSurface(). > > Signed-off-by: Aaron Levinson <alevinsn@aracnet.com> > --- > libavutil/hwcontext_dxva2.c | 9 ++++++++- > 1 file changed, 8 insertions(+), 1 deletion(-) > > diff --git a/libavutil/hwcontext_dxva2.c b/libavutil/hwcontext_dxva2.c > index 4ed0d56..6c41788 100644 > --- a/libavutil/hwcontext_dxva2.c > +++ b/libavutil/hwcontext_dxva2.c > @@ -121,6 +121,13 @@ static void dxva2_frames_uninit(AVHWFramesContext *ctx) > } > } > > +static void dxva2_pool_release_dummy(void *opaque, uint8_t *data) > +{ > + // important not to free anything here--data is a surface object > + // associated with the call to CreateSurface(), and these surfaces are > + // released in dxva2_frames_uninit() > +} > + > static AVBufferRef *dxva2_pool_alloc(void *opaque, int size) > { > AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; > @@ -130,7 +137,7 @@ static AVBufferRef *dxva2_pool_alloc(void *opaque, int size) > if (s->nb_surfaces_used < hwctx->nb_surfaces) { > s->nb_surfaces_used++; > return av_buffer_create((uint8_t*)s->surfaces_internal[s->nb_surfaces_used - 1], > - sizeof(*hwctx->surfaces), NULL, 0, 0); > + sizeof(*hwctx->surfaces), dxva2_pool_release_dummy, 0, 0); > } > > return NULL; >
diff --git a/libavutil/hwcontext_dxva2.c b/libavutil/hwcontext_dxva2.c index 4ed0d56..6c41788 100644 --- a/libavutil/hwcontext_dxva2.c +++ b/libavutil/hwcontext_dxva2.c @@ -121,6 +121,13 @@ static void dxva2_frames_uninit(AVHWFramesContext *ctx) } } +static void dxva2_pool_release_dummy(void *opaque, uint8_t *data) +{ + // important not to free anything here--data is a surface object + // associated with the call to CreateSurface(), and these surfaces are + // released in dxva2_frames_uninit() +} + static AVBufferRef *dxva2_pool_alloc(void *opaque, int size) { AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; @@ -130,7 +137,7 @@ static AVBufferRef *dxva2_pool_alloc(void *opaque, int size) if (s->nb_surfaces_used < hwctx->nb_surfaces) { s->nb_surfaces_used++; return av_buffer_create((uint8_t*)s->surfaces_internal[s->nb_surfaces_used - 1], - sizeof(*hwctx->surfaces), NULL, 0, 0); + sizeof(*hwctx->surfaces), dxva2_pool_release_dummy, 0, 0); } return NULL;
Purpose: No longer improperly freeing IDirect3DSurface9 objects in hwcontext_dxva2.c. Added dxva2_pool_release_dummy() and using it in call to av_buffer_create() in dxva2_pool_alloc(). Prior to this change, av_buffer_create() was called with NULL for the third argument, which indicates that av_buffer_default_free() should be used to free the buffer's data. Eventually, it gets to buffer_pool_free() and calls buf->free() on a surface object (which is av_buffer_default_free()). This can result in a crash when the debug version of the C-runtime is used on Windows. While it doesn't appear to result in a crash when the release version of the C-runtime is used on Windows, it likely results in memory corruption, since av_free() is being called on memory that was allocated using IDirectXVideoAccelerationService::CreateSurface(). Signed-off-by: Aaron Levinson <alevinsn@aracnet.com> --- libavutil/hwcontext_dxva2.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)