[FFmpeg-devel] lavc/qsv: improve the default GPU memory usage

Submitted by Linjie Fu on Nov. 1, 2019, 2:40 a.m.

Details

Message ID 1572576045-5252-1-git-send-email-linjie.fu@intel.com
State New
Headers show

Commit Message

Linjie Fu Nov. 1, 2019, 2:40 a.m.
A large initial_pool_size leads to redundant GPU memory allocations
compared with MSDK.

For some special cases which needs larger GPU memory like look_ahead,
add -extra_hw_frames to allocate more.

CMD:
ffmpeg -hwaccel qsv -extra_hw_frames 50 -c:v hevc_qsv -i hevc.h265 -c:v h264_qsv
-look_ahead 1 -look_ahead_depth 50 -y output.h264

Fix #7943.

Reported-by: Eero T
Signed-off-by: Linjie Fu <linjie.fu@intel.com>
---
 fftools/ffmpeg_qsv.c     | 2 +-
 libavcodec/qsvdec.c      | 5 ++++-
 libavcodec/qsvenc_h264.c | 4 ++--
 3 files changed, 7 insertions(+), 4 deletions(-)

Comments

李忠 Nov. 1, 2019, 10:14 a.m.
> A large initial_pool_size leads to redundant GPU memory allocations
> compared with MSDK.
>
> For some special cases which needs larger GPU memory like look_ahead,
> add -extra_hw_frames to allocate more.
>
> CMD:
> ffmpeg -hwaccel qsv -extra_hw_frames 50 -c:v hevc_qsv -i hevc.h265 -c:v h264_qsv
> -look_ahead 1 -look_ahead_depth 50 -y output.h264
>
> Fix #7943.
>
> Reported-by: Eero T
> Signed-off-by: Linjie Fu <linjie.fu@intel.com>
> ---
>  fftools/ffmpeg_qsv.c     | 2 +-
>  libavcodec/qsvdec.c      | 5 ++++-
>  libavcodec/qsvenc_h264.c | 4 ++--
>  3 files changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/fftools/ffmpeg_qsv.c b/fftools/ffmpeg_qsv.c
> index 9c4285b..5039a5a 100644
> --- a/fftools/ffmpeg_qsv.c
> +++ b/fftools/ffmpeg_qsv.c
> @@ -93,7 +93,7 @@ int qsv_init(AVCodecContext *s)
>      frames_ctx->height            = FFALIGN(s->coded_height, 32);
>      frames_ctx->format            = AV_PIX_FMT_QSV;
>      frames_ctx->sw_format         = s->sw_pix_fmt;
> -    frames_ctx->initial_pool_size = 64 + s->extra_hw_frames;
> +    frames_ctx->initial_pool_size = 24 + s->extra_hw_frames;

Should be better add pool size when look_ahead is enabled.

E:g
initial_pool_size = 24 + look_head_depth + s->extra_hw_frames;

you can set a default look_ahead_depth when look_ahead is enabled bu
no depth specified.

>      frames_hwctx->frame_type      = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET;
>
>      ret = av_hwframe_ctx_init(ist->hw_frames_ctx);
> diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c
> index 0d34021..407e6de 100644
> --- a/libavcodec/qsvdec.c
> +++ b/libavcodec/qsvdec.c
> @@ -300,8 +300,11 @@ static int alloc_frame(AVCodecContext *avctx, QSVContext *q, QSVFrame *frame)
>      else
>          ret = ff_get_buffer(avctx, frame->frame, AV_GET_BUFFER_FLAG_REF);
>
> -    if (ret < 0)
> +    if (ret < 0) {
> +        av_log(avctx, AV_LOG_ERROR, "More memory in need, "
> +            "try -extra_hw_frames to allocate more.\n");
>          return ret;
> +    }
>
>      if (frame->frame->format == AV_PIX_FMT_QSV) {
>          frame->surface = *(mfxFrameSurface1*)frame->frame->data[3];
> diff --git a/libavcodec/qsvenc_h264.c b/libavcodec/qsvenc_h264.c
> index 27f36b9..4e8035b 100644
> --- a/libavcodec/qsvenc_h264.c
> +++ b/libavcodec/qsvenc_h264.c
> @@ -112,8 +112,8 @@ static const AVOption options[] = {
>      { "max_dec_frame_buffering", "Maximum number of frames buffered in the DPB", OFFSET(qsv.max_dec_frame_buffering), AV_OPT_TYPE_INT, { .i64 = 0 },   0, UINT16_MAX, VE },
>
>  #if QSV_HAVE_LA
> -    { "look_ahead",       "Use VBR algorithm with look ahead",    OFFSET(qsv.look_ahead),       AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE },
> -    { "look_ahead_depth", "Depth of look ahead in number frames", OFFSET(qsv.look_ahead_depth), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 100, VE },
> +    { "look_ahead",       "Use VBR algorithm with look ahead, extra_hw_frames is needed",    OFFSET(qsv.look_ahead),       AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE },
> +    { "look_ahead_depth", "Depth of look ahead in number frames, extra_hw_frames is needed", OFFSET(qsv.look_ahead_depth), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 100, VE },

No need to update the message if follow comments above.
Maxym Dmytrychenko Nov. 1, 2019, 1:12 p.m.
On Fri, Nov 1, 2019 at 11:20 AM Zhong Li <lizhong1008@gmail.com> wrote:

> > A large initial_pool_size leads to redundant GPU memory allocations
> > compared with MSDK.
> >
> > For some special cases which needs larger GPU memory like look_ahead,
> > add -extra_hw_frames to allocate more.
> >
> > CMD:
> > ffmpeg -hwaccel qsv -extra_hw_frames 50 -c:v hevc_qsv -i hevc.h265 -c:v
> h264_qsv
> > -look_ahead 1 -look_ahead_depth 50 -y output.h264
> >
> > Fix #7943.
> >
> > Reported-by: Eero T
> > Signed-off-by: Linjie Fu <linjie.fu@intel.com>
> > ---
> >  fftools/ffmpeg_qsv.c     | 2 +-
> >  libavcodec/qsvdec.c      | 5 ++++-
> >  libavcodec/qsvenc_h264.c | 4 ++--
> >  3 files changed, 7 insertions(+), 4 deletions(-)
> >
> > diff --git a/fftools/ffmpeg_qsv.c b/fftools/ffmpeg_qsv.c
> > index 9c4285b..5039a5a 100644
> > --- a/fftools/ffmpeg_qsv.c
> > +++ b/fftools/ffmpeg_qsv.c
> > @@ -93,7 +93,7 @@ int qsv_init(AVCodecContext *s)
> >      frames_ctx->height            = FFALIGN(s->coded_height, 32);
> >      frames_ctx->format            = AV_PIX_FMT_QSV;
> >      frames_ctx->sw_format         = s->sw_pix_fmt;
> > -    frames_ctx->initial_pool_size = 64 + s->extra_hw_frames;
> > +    frames_ctx->initial_pool_size = 24 + s->extra_hw_frames;
>
> Should be better add pool size when look_ahead is enabled.
>
>

yes, please follow this.



> E:g
> initial_pool_size = 24 + look_head_depth + s->extra_hw_frames;
>
> you can set a default look_ahead_depth when look_ahead is enabled bu
> no depth specified.
>
> >      frames_hwctx->frame_type      =
> MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET;
> >
> >      ret = av_hwframe_ctx_init(ist->hw_frames_ctx);
> > diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c
> > index 0d34021..407e6de 100644
> > --- a/libavcodec/qsvdec.c
> > +++ b/libavcodec/qsvdec.c
> > @@ -300,8 +300,11 @@ static int alloc_frame(AVCodecContext *avctx,
> QSVContext *q, QSVFrame *frame)
> >      else
> >          ret = ff_get_buffer(avctx, frame->frame,
> AV_GET_BUFFER_FLAG_REF);
> >
> > -    if (ret < 0)
> > +    if (ret < 0) {
> > +        av_log(avctx, AV_LOG_ERROR, "More memory in need, "
> > +            "try -extra_hw_frames to allocate more.\n");
> >          return ret;
> > +    }
> >
> >      if (frame->frame->format == AV_PIX_FMT_QSV) {
> >          frame->surface = *(mfxFrameSurface1*)frame->frame->data[3];
> > diff --git a/libavcodec/qsvenc_h264.c b/libavcodec/qsvenc_h264.c
> > index 27f36b9..4e8035b 100644
> > --- a/libavcodec/qsvenc_h264.c
> > +++ b/libavcodec/qsvenc_h264.c
> > @@ -112,8 +112,8 @@ static const AVOption options[] = {
> >      { "max_dec_frame_buffering", "Maximum number of frames buffered in
> the DPB", OFFSET(qsv.max_dec_frame_buffering), AV_OPT_TYPE_INT, { .i64 = 0
> },   0, UINT16_MAX, VE },
> >
> >  #if QSV_HAVE_LA
> > -    { "look_ahead",       "Use VBR algorithm with look ahead",
> OFFSET(qsv.look_ahead),       AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE },
> > -    { "look_ahead_depth", "Depth of look ahead in number frames",
> OFFSET(qsv.look_ahead_depth), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 100, VE },
> > +    { "look_ahead",       "Use VBR algorithm with look ahead,
> extra_hw_frames is needed",    OFFSET(qsv.look_ahead),
>  AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE },
> > +    { "look_ahead_depth", "Depth of look ahead in number frames,
> extra_hw_frames is needed", OFFSET(qsv.look_ahead_depth), AV_OPT_TYPE_INT,
> { .i64 = 0 }, 0, 100, VE },
>
> No need to update the message if follow comments above.
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Patch hide | download patch | download mbox

diff --git a/fftools/ffmpeg_qsv.c b/fftools/ffmpeg_qsv.c
index 9c4285b..5039a5a 100644
--- a/fftools/ffmpeg_qsv.c
+++ b/fftools/ffmpeg_qsv.c
@@ -93,7 +93,7 @@  int qsv_init(AVCodecContext *s)
     frames_ctx->height            = FFALIGN(s->coded_height, 32);
     frames_ctx->format            = AV_PIX_FMT_QSV;
     frames_ctx->sw_format         = s->sw_pix_fmt;
-    frames_ctx->initial_pool_size = 64 + s->extra_hw_frames;
+    frames_ctx->initial_pool_size = 24 + s->extra_hw_frames;
     frames_hwctx->frame_type      = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET;
 
     ret = av_hwframe_ctx_init(ist->hw_frames_ctx);
diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c
index 0d34021..407e6de 100644
--- a/libavcodec/qsvdec.c
+++ b/libavcodec/qsvdec.c
@@ -300,8 +300,11 @@  static int alloc_frame(AVCodecContext *avctx, QSVContext *q, QSVFrame *frame)
     else
         ret = ff_get_buffer(avctx, frame->frame, AV_GET_BUFFER_FLAG_REF);
 
-    if (ret < 0)
+    if (ret < 0) {
+        av_log(avctx, AV_LOG_ERROR, "More memory in need, "
+            "try -extra_hw_frames to allocate more.\n");
         return ret;
+    }
 
     if (frame->frame->format == AV_PIX_FMT_QSV) {
         frame->surface = *(mfxFrameSurface1*)frame->frame->data[3];
diff --git a/libavcodec/qsvenc_h264.c b/libavcodec/qsvenc_h264.c
index 27f36b9..4e8035b 100644
--- a/libavcodec/qsvenc_h264.c
+++ b/libavcodec/qsvenc_h264.c
@@ -112,8 +112,8 @@  static const AVOption options[] = {
     { "max_dec_frame_buffering", "Maximum number of frames buffered in the DPB", OFFSET(qsv.max_dec_frame_buffering), AV_OPT_TYPE_INT, { .i64 = 0 },   0, UINT16_MAX, VE },
 
 #if QSV_HAVE_LA
-    { "look_ahead",       "Use VBR algorithm with look ahead",    OFFSET(qsv.look_ahead),       AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE },
-    { "look_ahead_depth", "Depth of look ahead in number frames", OFFSET(qsv.look_ahead_depth), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 100, VE },
+    { "look_ahead",       "Use VBR algorithm with look ahead, extra_hw_frames is needed",    OFFSET(qsv.look_ahead),       AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE },
+    { "look_ahead_depth", "Depth of look ahead in number frames, extra_hw_frames is needed", OFFSET(qsv.look_ahead_depth), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 100, VE },
 #endif
 #if QSV_HAVE_LA_DS
     { "look_ahead_downsampling", "Downscaling factor for the frames saved for the lookahead analysis", OFFSET(qsv.look_ahead_downsampling),