diff mbox series

[FFmpeg-devel] avcodec/nvenc: support dynamic resolution change

Message ID 1596788308-57068-1-git-send-email-leozhang@qiyi.com
State New
Headers show
Series [FFmpeg-devel] avcodec/nvenc: support dynamic resolution change
Related show

Checks

Context Check Description
andriy/default pending
andriy/make success Make finished
andriy/make_fate success Make fate finished

Commit Message

leozhang Aug. 7, 2020, 8:18 a.m. UTC
Allow dynamic resolution change, this is useful for real time video communication application.

Use below commands to test it,
ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v hevc_nvenc out.265 -loglevel verbose -y
ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v h264_nvenc out.264 -loglevel verbose -y

Signed-off-by: leozhang <leozhang@qiyi.com>
---
 libavcodec/nvenc.c | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

Comments

Linjie Fu Aug. 8, 2020, 12:48 a.m. UTC | #1
On Fri, Aug 7, 2020 at 4:19 PM leozhang <leozhang@qiyi.com> wrote:
>
> Allow dynamic resolution change, this is useful for real time video communication application.
>
> Use below commands to test it,
> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v hevc_nvenc out.265 -loglevel verbose -y
> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v h264_nvenc out.264 -loglevel verbose -y
>
> Signed-off-by: leozhang <leozhang@qiyi.com>
> ---
>  libavcodec/nvenc.c | 22 +++++++++++++++++++++-
>  1 file changed, 21 insertions(+), 1 deletion(-)

Hi leozhang,

Adding dynamic resolution encoding support is the tendency, and there
are some previous discussions[1] [2]
about adding support for dynamic resolution encoding.

And one conclusion is that we'd prefer to cope with this in a more
general way, like recreating the
encoder instance instead of modifying in specific codec.

- Linjie

[1] https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=1434
[2] https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=1470
Timo Rothenpieler Aug. 8, 2020, 11:21 a.m. UTC | #2
On 08.08.2020 02:48, Linjie Fu wrote:
> On Fri, Aug 7, 2020 at 4:19 PM leozhang <leozhang@qiyi.com> wrote:
>>
>> Allow dynamic resolution change, this is useful for real time video communication application.
>>
>> Use below commands to test it,
>> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v hevc_nvenc out.265 -loglevel verbose -y
>> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v h264_nvenc out.264 -loglevel verbose -y
>>
>> Signed-off-by: leozhang <leozhang@qiyi.com>
>> ---
>>   libavcodec/nvenc.c | 22 +++++++++++++++++++++-
>>   1 file changed, 21 insertions(+), 1 deletion(-)
> 
> Hi leozhang,
> 
> Adding dynamic resolution encoding support is the tendency, and there
> are some previous discussions[1] [2]
> about adding support for dynamic resolution encoding.
> 
> And one conclusion is that we'd prefer to cope with this in a more
> general way, like recreating the
> encoder instance instead of modifying in specific codec.
> 
> - Linjie

The problem is, that specially in the case of nvenc, re-creating the 
encoder can take several seconds.

Patch generally looks okay to me, and I'm fine with applying it.
Timo Rothenpieler Aug. 8, 2020, 11:22 a.m. UTC | #3
On 07.08.2020 10:18, leozhang wrote:
> Allow dynamic resolution change, this is useful for real time video communication application.
> 
> Use below commands to test it,
> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v hevc_nvenc out.265 -loglevel verbose -y
> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v h264_nvenc out.264 -loglevel verbose -y
> 

Are there any negative side-effects of this?
Like, what happens if the user is transcoding a live tv recording that 
switches resolutions? Will nvenc and the muxer after it handle it 
gracefully? How are they handling it right now?
Linjie Fu Aug. 8, 2020, 3:22 p.m. UTC | #4
On Sat, Aug 8, 2020 at 7:21 PM Timo Rothenpieler <timo@rothenpieler.org> wrote:
>
> On 08.08.2020 02:48, Linjie Fu wrote:
> > On Fri, Aug 7, 2020 at 4:19 PM leozhang <leozhang@qiyi.com> wrote:
> >>
> >> Allow dynamic resolution change, this is useful for real time video communication application.
> >>
> >> Use below commands to test it,
> >> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v hevc_nvenc out.265 -loglevel verbose -y
> >> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v h264_nvenc out.264 -loglevel verbose -y
> >>
> >> Signed-off-by: leozhang <leozhang@qiyi.com>
> >> ---
> >>   libavcodec/nvenc.c | 22 +++++++++++++++++++++-
> >>   1 file changed, 21 insertions(+), 1 deletion(-)
> >
> > Hi leozhang,
> >
> > Adding dynamic resolution encoding support is the tendency, and there
> > are some previous discussions[1] [2]
> > about adding support for dynamic resolution encoding.
> >
> > And one conclusion is that we'd prefer to cope with this in a more
> > general way, like recreating the
> > encoder instance instead of modifying in specific codec.
> >
> > - Linjie
>
> The problem is, that specially in the case of nvenc, re-creating the
> encoder can take several seconds.

There is once a concern about parameter changing in the AVCodecontext
in the last discussion:
[1] libavcodec encoders have always assumed that the stream parameters
are set once and never change afterwards.

> Patch generally looks okay to me, and I'm fine with applying it.
I'm ok with this if it didn't cause the side-effects you've mentioned.
(since before we got a decent support, better ever than never)

- Linjie

[1] https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2020-February/257377.html
Mark Thompson Aug. 8, 2020, 9:48 p.m. UTC | #5
On 08/08/2020 12:22, Timo Rothenpieler wrote:
> On 07.08.2020 10:18, leozhang wrote:
>> Allow dynamic resolution change, this is useful for real time video communication application.
>>
>> Use below commands to test it,
>> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v hevc_nvenc out.265 -loglevel verbose -y
>> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v h264_nvenc out.264 -loglevel verbose -y
>>
> 
> Are there any negative side-effects of this?
> Like, what happens if the user is transcoding a live tv recording that switches resolutions? Will nvenc and the muxer after it handle it gracefully? How are they handling it right now?

Many muxers won't.  Anything which requires global headers (such as mp4 or mkv) will write a nonconformant file which will work in some cases but fail in others - some players will not work at all, 
and seeking across the boundary will be broken if the headers are not included in front of every seek point.

I don't immediately see a bad case if you make sure it rejects the change immediately if the user tries to use it when avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER and have it always include inline 
headers on every seek point, but there might be something else I'm missing.  (There is a good reason why autoscale is the default.)

More generally, the API doesn't currently allow this and all other users seem to be happy with doing a close/reopen.  Is there something wrong with how libavcodec is using Nvidia to make it especially 
slow here?  They advertise being able to do more than 20 simultaneous streams on one card - does that really take 40 seconds to start?

- Mark
diff mbox series

Patch

diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c
index 8f5036b..68e2a35 100644
--- a/libavcodec/nvenc.c
+++ b/libavcodec/nvenc.c
@@ -2051,7 +2051,7 @@  static void reconfig_encoder(AVCodecContext *avctx, const AVFrame *frame)
     NV_ENC_RECONFIGURE_PARAMS params = { 0 };
     int needs_reconfig = 0;
     int needs_encode_config = 0;
-    int reconfig_bitrate = 0, reconfig_dar = 0;
+    int reconfig_bitrate = 0, reconfig_dar = 0, reconfig_res = 0;
     int dw, dh;
 
     params.version = NV_ENC_RECONFIGURE_PARAMS_VER;
@@ -2071,6 +2071,21 @@  static void reconfig_encoder(AVCodecContext *avctx, const AVFrame *frame)
         reconfig_dar = 1;
     }
 
+    if (frame && (frame->width != ctx->init_encode_params.encodeWidth ||
+                  frame->height != ctx->init_encode_params.encodeHeight)) {
+        av_log(avctx, AV_LOG_VERBOSE,
+               "input frame changed from %dx%d -> %dx%d\n",
+               ctx->init_encode_params.encodeWidth,
+               ctx->init_encode_params.encodeHeight,
+               frame->width, frame->height);
+
+        params.reInitEncodeParams.encodeWidth  = frame->width;
+        params.reInitEncodeParams.encodeHeight = frame->height;
+
+        needs_reconfig = 1;
+        reconfig_res = 1;
+    }
+
     if (ctx->rc != NV_ENC_PARAMS_RC_CONSTQP && ctx->support_dyn_bitrate) {
         if (avctx->bit_rate > 0 && params.reInitEncodeParams.encodeConfig->rcParams.averageBitRate != avctx->bit_rate) {
             av_log(avctx, AV_LOG_VERBOSE,
@@ -2124,6 +2139,11 @@  static void reconfig_encoder(AVCodecContext *avctx, const AVFrame *frame)
                 ctx->init_encode_params.darWidth = dw;
             }
 
+            if (reconfig_res) {
+                avctx->width  = ctx->init_encode_params.encodeWidth  = params.reInitEncodeParams.encodeWidth;
+                avctx->height = ctx->init_encode_params.encodeHeight = params.reInitEncodeParams.encodeHeight;
+            }
+
             if (reconfig_bitrate) {
                 ctx->encode_config.rcParams.averageBitRate = params.reInitEncodeParams.encodeConfig->rcParams.averageBitRate;
                 ctx->encode_config.rcParams.maxBitRate = params.reInitEncodeParams.encodeConfig->rcParams.maxBitRate;