[FFmpeg-devel] avcodec/dxva2_mpeg2.c: don't try to get surface index for absent frame

Submitted by Anton Fedchin on Dec. 22, 2018, 1:12 p.m.

Details

Message ID 20181222131223.4192-1-afedchin@weezlabs.com
State New
Headers show

Commit Message

Anton Fedchin Dec. 22, 2018, 1:12 p.m.
From: Anton Fedchin <afedchin@ruswizards.com>

after 153b36f there is a possibility to crash when trying to get index of
a surface which points to nirvana. it may occurs when a mpeg2 stream starts
with non i-frame.
---
 libavcodec/dxva2_mpeg2.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Comments

Carl Eugen Hoyos Dec. 29, 2018, 10:52 a.m.
2018-12-22 14:12 GMT+01:00, Anton Fedchin <afedchin@weezlabs.com>:
> From: Anton Fedchin <afedchin@ruswizards.com>
>
> after 153b36f there is a possibility to crash when trying to get index of
> a surface which points to nirvana. it may occurs when a mpeg2 stream starts
> with non i-frame.
> ---
>  libavcodec/dxva2_mpeg2.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/libavcodec/dxva2_mpeg2.c b/libavcodec/dxva2_mpeg2.c
> index 8cc21bf199..c862e6e5e4 100644
> --- a/libavcodec/dxva2_mpeg2.c
> +++ b/libavcodec/dxva2_mpeg2.c
> @@ -48,11 +48,11 @@ static void fill_picture_parameters(AVCodecContext
> *avctx,
>      memset(pp, 0, sizeof(*pp));
>      pp->wDecodedPictureIndex         = ff_dxva2_get_surface_index(avctx,
> ctx, current_picture->f);
>      pp->wDeblockedPictureIndex       = 0;
> -    if (s->pict_type != AV_PICTURE_TYPE_I)
> +    if (s->pict_type != AV_PICTURE_TYPE_I && s->last_picture_ptr)
>          pp->wForwardRefPictureIndex  = ff_dxva2_get_surface_index(avctx,
> ctx, s->last_picture.f);
>      else
>          pp->wForwardRefPictureIndex  = 0xffff;
> -    if (s->pict_type == AV_PICTURE_TYPE_B)
> +    if (s->pict_type == AV_PICTURE_TYPE_B && s->next_picture_ptr)
>          pp->wBackwardRefPictureIndex = ff_dxva2_get_surface_index(avctx,
> ctx, s->next_picture.f);

Hendrik?

Carl Eugen
Hendrik Leppkes Dec. 29, 2018, 11:23 a.m.
On Wed, Dec 26, 2018 at 8:55 PM Anton Fedchin <afedchin@weezlabs.com> wrote:
>
> From: Anton Fedchin <afedchin@ruswizards.com>
>
> after 153b36f there is a possibility to crash when trying to get index of
> a surface which points to nirvana. it may occurs when a mpeg2 stream starts
> with non i-frame.
> ---
>  libavcodec/dxva2_mpeg2.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/libavcodec/dxva2_mpeg2.c b/libavcodec/dxva2_mpeg2.c
> index 8cc21bf199..c862e6e5e4 100644
> --- a/libavcodec/dxva2_mpeg2.c
> +++ b/libavcodec/dxva2_mpeg2.c
> @@ -48,11 +48,11 @@ static void fill_picture_parameters(AVCodecContext *avctx,
>      memset(pp, 0, sizeof(*pp));
>      pp->wDecodedPictureIndex         = ff_dxva2_get_surface_index(avctx, ctx, current_picture->f);
>      pp->wDeblockedPictureIndex       = 0;
> -    if (s->pict_type != AV_PICTURE_TYPE_I)
> +    if (s->pict_type != AV_PICTURE_TYPE_I && s->last_picture_ptr)
>          pp->wForwardRefPictureIndex  = ff_dxva2_get_surface_index(avctx, ctx, s->last_picture.f);
>      else
>          pp->wForwardRefPictureIndex  = 0xffff;
> -    if (s->pict_type == AV_PICTURE_TYPE_B)
> +    if (s->pict_type == AV_PICTURE_TYPE_B && s->next_picture_ptr)
>          pp->wBackwardRefPictureIndex = ff_dxva2_get_surface_index(avctx, ctx, s->next_picture.f);
>      else
>          pp->wBackwardRefPictureIndex = 0xffff;

I think it would be better to fix ff_dxva2_get_surface_index to not
crash when fed an empty frame. That ensures safety for all codecs and
future codecs as well.

- Hendrik

Patch hide | download patch | download mbox

diff --git a/libavcodec/dxva2_mpeg2.c b/libavcodec/dxva2_mpeg2.c
index 8cc21bf199..c862e6e5e4 100644
--- a/libavcodec/dxva2_mpeg2.c
+++ b/libavcodec/dxva2_mpeg2.c
@@ -48,11 +48,11 @@  static void fill_picture_parameters(AVCodecContext *avctx,
     memset(pp, 0, sizeof(*pp));
     pp->wDecodedPictureIndex         = ff_dxva2_get_surface_index(avctx, ctx, current_picture->f);
     pp->wDeblockedPictureIndex       = 0;
-    if (s->pict_type != AV_PICTURE_TYPE_I)
+    if (s->pict_type != AV_PICTURE_TYPE_I && s->last_picture_ptr)
         pp->wForwardRefPictureIndex  = ff_dxva2_get_surface_index(avctx, ctx, s->last_picture.f);
     else
         pp->wForwardRefPictureIndex  = 0xffff;
-    if (s->pict_type == AV_PICTURE_TYPE_B)
+    if (s->pict_type == AV_PICTURE_TYPE_B && s->next_picture_ptr)
         pp->wBackwardRefPictureIndex = ff_dxva2_get_surface_index(avctx, ctx, s->next_picture.f);
     else
         pp->wBackwardRefPictureIndex = 0xffff;