| Message ID | 1596788308-57068-1-git-send-email-leozhang@qiyi.com |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel] avcodec/nvenc: support dynamic resolution change | expand |
| Context | Check | Description |
|---|---|---|
| andriy/default | pending | |
| andriy/make | success | Make finished |
| andriy/make_fate | success | Make fate finished |
On Fri, Aug 7, 2020 at 4:19 PM leozhang <leozhang@qiyi.com> wrote: > > Allow dynamic resolution change, this is useful for real time video communication application. > > Use below commands to test it, > ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v hevc_nvenc out.265 -loglevel verbose -y > ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v h264_nvenc out.264 -loglevel verbose -y > > Signed-off-by: leozhang <leozhang@qiyi.com> > --- > libavcodec/nvenc.c | 22 +++++++++++++++++++++- > 1 file changed, 21 insertions(+), 1 deletion(-) Hi leozhang, Adding dynamic resolution encoding support is the tendency, and there are some previous discussions[1] [2] about adding support for dynamic resolution encoding. And one conclusion is that we'd prefer to cope with this in a more general way, like recreating the encoder instance instead of modifying in specific codec. - Linjie [1] https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=1434 [2] https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=1470
On 08.08.2020 02:48, Linjie Fu wrote: > On Fri, Aug 7, 2020 at 4:19 PM leozhang <leozhang@qiyi.com> wrote: >> >> Allow dynamic resolution change, this is useful for real time video communication application. >> >> Use below commands to test it, >> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v hevc_nvenc out.265 -loglevel verbose -y >> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v h264_nvenc out.264 -loglevel verbose -y >> >> Signed-off-by: leozhang <leozhang@qiyi.com> >> --- >> libavcodec/nvenc.c | 22 +++++++++++++++++++++- >> 1 file changed, 21 insertions(+), 1 deletion(-) > > Hi leozhang, > > Adding dynamic resolution encoding support is the tendency, and there > are some previous discussions[1] [2] > about adding support for dynamic resolution encoding. > > And one conclusion is that we'd prefer to cope with this in a more > general way, like recreating the > encoder instance instead of modifying in specific codec. > > - Linjie The problem is, that specially in the case of nvenc, re-creating the encoder can take several seconds. Patch generally looks okay to me, and I'm fine with applying it.
On 07.08.2020 10:18, leozhang wrote: > Allow dynamic resolution change, this is useful for real time video communication application. > > Use below commands to test it, > ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v hevc_nvenc out.265 -loglevel verbose -y > ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v h264_nvenc out.264 -loglevel verbose -y > Are there any negative side-effects of this? Like, what happens if the user is transcoding a live tv recording that switches resolutions? Will nvenc and the muxer after it handle it gracefully? How are they handling it right now?
On Sat, Aug 8, 2020 at 7:21 PM Timo Rothenpieler <timo@rothenpieler.org> wrote: > > On 08.08.2020 02:48, Linjie Fu wrote: > > On Fri, Aug 7, 2020 at 4:19 PM leozhang <leozhang@qiyi.com> wrote: > >> > >> Allow dynamic resolution change, this is useful for real time video communication application. > >> > >> Use below commands to test it, > >> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v hevc_nvenc out.265 -loglevel verbose -y > >> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v h264_nvenc out.264 -loglevel verbose -y > >> > >> Signed-off-by: leozhang <leozhang@qiyi.com> > >> --- > >> libavcodec/nvenc.c | 22 +++++++++++++++++++++- > >> 1 file changed, 21 insertions(+), 1 deletion(-) > > > > Hi leozhang, > > > > Adding dynamic resolution encoding support is the tendency, and there > > are some previous discussions[1] [2] > > about adding support for dynamic resolution encoding. > > > > And one conclusion is that we'd prefer to cope with this in a more > > general way, like recreating the > > encoder instance instead of modifying in specific codec. > > > > - Linjie > > The problem is, that specially in the case of nvenc, re-creating the > encoder can take several seconds. There is once a concern about parameter changing in the AVCodecontext in the last discussion: [1] libavcodec encoders have always assumed that the stream parameters are set once and never change afterwards. > Patch generally looks okay to me, and I'm fine with applying it. I'm ok with this if it didn't cause the side-effects you've mentioned. (since before we got a decent support, better ever than never) - Linjie [1] https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2020-February/257377.html
On 08/08/2020 12:22, Timo Rothenpieler wrote: > On 07.08.2020 10:18, leozhang wrote: >> Allow dynamic resolution change, this is useful for real time video communication application. >> >> Use below commands to test it, >> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v hevc_nvenc out.265 -loglevel verbose -y >> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v h264_nvenc out.264 -loglevel verbose -y >> > > Are there any negative side-effects of this? > Like, what happens if the user is transcoding a live tv recording that switches resolutions? Will nvenc and the muxer after it handle it gracefully? How are they handling it right now? Many muxers won't. Anything which requires global headers (such as mp4 or mkv) will write a nonconformant file which will work in some cases but fail in others - some players will not work at all, and seeking across the boundary will be broken if the headers are not included in front of every seek point. I don't immediately see a bad case if you make sure it rejects the change immediately if the user tries to use it when avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER and have it always include inline headers on every seek point, but there might be something else I'm missing. (There is a good reason why autoscale is the default.) More generally, the API doesn't currently allow this and all other users seem to be happy with doing a close/reopen. Is there something wrong with how libavcodec is using Nvidia to make it especially slow here? They advertise being able to do more than 20 simultaneous streams on one card - does that really take 40 seconds to start? - Mark
On 08/08/2020 17:22, Linjie Fu wrote: > On Sat, Aug 8, 2020 at 7:21 PM Timo Rothenpieler <timo@rothenpieler.org> wrote: >> >> On 08.08.2020 02:48, Linjie Fu wrote: >>> On Fri, Aug 7, 2020 at 4:19 PM leozhang <leozhang@qiyi.com> wrote: >>>> >>>> Allow dynamic resolution change, this is useful for real time video communication application. >>>> >>>> Use below commands to test it, >>>> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v hevc_nvenc out.265 -loglevel verbose -y >>>> ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v h264_nvenc out.264 -loglevel verbose -y >>>> >>>> Signed-off-by: leozhang <leozhang@qiyi.com> >>>> --- >>>> libavcodec/nvenc.c | 22 +++++++++++++++++++++- >>>> 1 file changed, 21 insertions(+), 1 deletion(-) >>> >>> Hi leozhang, >>> >>> Adding dynamic resolution encoding support is the tendency, and there >>> are some previous discussions[1] [2] >>> about adding support for dynamic resolution encoding. >>> >>> And one conclusion is that we'd prefer to cope with this in a more >>> general way, like recreating the >>> encoder instance instead of modifying in specific codec. >>> >>> - Linjie >> >> The problem is, that specially in the case of nvenc, re-creating the >> encoder can take several seconds. > > There is once a concern about parameter changing in the AVCodecontext > in the last discussion: > [1] libavcodec encoders have always assumed that the stream parameters > are set once and never change afterwards. > >> Patch generally looks okay to me, and I'm fine with applying it. > I'm ok with this if it didn't cause the side-effects you've mentioned. > (since before we got a decent support, better ever than never) > > - Linjie > > [1] https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2020-February/257377.html > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > As far as I am aware, the NVENC encoder requires that both maxEncodeWidth and maxEncodeHeight are greater or equal to the new resolution you are targetting. If they aren't, the resolution change will fail. It is most likely safe to assume that the encoder is unsafe to use if that change fails. For example in live transcoding you would have to know the possible maximum resolution ahead of time - which unless documented or included in the container information is impossible. But it might be safe to just set them to the resolution that started the encode session, and only allow switching to smaller-than-original resolutions. Note that dynamic resolution changes should be impossible while maxEncodeWidth and maxEncodeHeight are at 0, according to the nvEncodeAPI.h file: > Client should allocate output buffers according to this dimension for dynamic resolution change. If set to 0, Encoder will not allow dynamic resolution change.
diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c index 8f5036b..68e2a35 100644 --- a/libavcodec/nvenc.c +++ b/libavcodec/nvenc.c @@ -2051,7 +2051,7 @@ static void reconfig_encoder(AVCodecContext *avctx, const AVFrame *frame) NV_ENC_RECONFIGURE_PARAMS params = { 0 }; int needs_reconfig = 0; int needs_encode_config = 0; - int reconfig_bitrate = 0, reconfig_dar = 0; + int reconfig_bitrate = 0, reconfig_dar = 0, reconfig_res = 0; int dw, dh; params.version = NV_ENC_RECONFIGURE_PARAMS_VER; @@ -2071,6 +2071,21 @@ static void reconfig_encoder(AVCodecContext *avctx, const AVFrame *frame) reconfig_dar = 1; } + if (frame && (frame->width != ctx->init_encode_params.encodeWidth || + frame->height != ctx->init_encode_params.encodeHeight)) { + av_log(avctx, AV_LOG_VERBOSE, + "input frame changed from %dx%d -> %dx%d\n", + ctx->init_encode_params.encodeWidth, + ctx->init_encode_params.encodeHeight, + frame->width, frame->height); + + params.reInitEncodeParams.encodeWidth = frame->width; + params.reInitEncodeParams.encodeHeight = frame->height; + + needs_reconfig = 1; + reconfig_res = 1; + } + if (ctx->rc != NV_ENC_PARAMS_RC_CONSTQP && ctx->support_dyn_bitrate) { if (avctx->bit_rate > 0 && params.reInitEncodeParams.encodeConfig->rcParams.averageBitRate != avctx->bit_rate) { av_log(avctx, AV_LOG_VERBOSE, @@ -2124,6 +2139,11 @@ static void reconfig_encoder(AVCodecContext *avctx, const AVFrame *frame) ctx->init_encode_params.darWidth = dw; } + if (reconfig_res) { + avctx->width = ctx->init_encode_params.encodeWidth = params.reInitEncodeParams.encodeWidth; + avctx->height = ctx->init_encode_params.encodeHeight = params.reInitEncodeParams.encodeHeight; + } + if (reconfig_bitrate) { ctx->encode_config.rcParams.averageBitRate = params.reInitEncodeParams.encodeConfig->rcParams.averageBitRate; ctx->encode_config.rcParams.maxBitRate = params.reInitEncodeParams.encodeConfig->rcParams.maxBitRate;
Allow dynamic resolution change, this is useful for real time video communication application. Use below commands to test it, ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v hevc_nvenc out.265 -loglevel verbose -y ffmpeg -i reinit-large_420_8-to-small_420_8.h264 -noautoscale -c:v h264_nvenc out.264 -loglevel verbose -y Signed-off-by: leozhang <leozhang@qiyi.com> --- libavcodec/nvenc.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-)