| Message ID | tencent_0126DCE7F26A8E13075C744BAEA61E508F06@qq.com |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel] avcodec/nvenc: fix NULL pointer dereference | expand |
| Context | Check | Description |
|---|---|---|
| yinshiyou/make_loongarch64 | success | Make finished |
| yinshiyou/make_fate_loongarch64 | success | Make fate finished |
| andriy/make_x86 | success | Make finished |
| andriy/make_fate_x86 | success | Make fate finished |
Zhao Zhili <quinkblack@foxmail.com> 于2023年2月28日周二 18:44写道： > > From: Zhao Zhili <zhilizhao@tencent.com> > > This can happen if encoder init failed before setup the queue, then > reorder_queue_flush() is called by close(). > > Fix ticket #10221 > > Signed-off-by: Zhao Zhili <zhilizhao@tencent.com> > --- > libavcodec/nvenc.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c > index 8a28454042..dda2dc3ba1 100644 > --- a/libavcodec/nvenc.c > +++ b/libavcodec/nvenc.c > @@ -178,6 +178,8 @@ static void reorder_queue_flush(AVFifo *queue) > { > FrameData fd; > > + if (!queue) > + return; > while (av_fifo_read(queue, &fd, 1) >= 0) > av_buffer_unref(&fd.frame_opaque_ref); > } > -- > 2.25.1 > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". lgtm Thanks Steven
On 28.02.2023 19:43, Zhao Zhili wrote: > From: Zhao Zhili <zhilizhao@tencent.com> > > This can happen if encoder init failed before setup the queue, then > reorder_queue_flush() is called by close(). > > Fix ticket #10221 > > Signed-off-by: Zhao Zhili <zhilizhao@tencent.com> > --- > libavcodec/nvenc.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c > index 8a28454042..dda2dc3ba1 100644 > --- a/libavcodec/nvenc.c > +++ b/libavcodec/nvenc.c > @@ -178,6 +178,8 @@ static void reorder_queue_flush(AVFifo *queue) > { > FrameData fd; > > + if (!queue) > + return; > while (av_fifo_read(queue, &fd, 1) >= 0) > av_buffer_unref(&fd.frame_opaque_ref); > } This should probably be an assert0, and the function should not be called to begin with if the queue is NULL.
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Timo Rothenpieler > Sent: 2023年2月28日 22:37 > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH] avcodec/nvenc: fix NULL pointer dereference > > On 28.02.2023 19:43, Zhao Zhili wrote: > > From: Zhao Zhili <zhilizhao@tencent.com> > > > > This can happen if encoder init failed before setup the queue, then > > reorder_queue_flush() is called by close(). > > > > Fix ticket #10221 > > > > Signed-off-by: Zhao Zhili <zhilizhao@tencent.com> > > --- > > libavcodec/nvenc.c | 2 ++ > > 1 file changed, 2 insertions(+) > > > > diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c > > index 8a28454042..dda2dc3ba1 100644 > > --- a/libavcodec/nvenc.c > > +++ b/libavcodec/nvenc.c > > @@ -178,6 +178,8 @@ static void reorder_queue_flush(AVFifo *queue) > > { > > FrameData fd; > > > > + if (!queue) > > + return; > > while (av_fifo_read(queue, &fd, 1) >= 0) > > av_buffer_unref(&fd.frame_opaque_ref); > > } > > This should probably be an assert0, and the function should not be > called to begin with if the queue is NULL. I can move the check from here to close(). assert is helpful if it can catch bugs during development. It doesn't help much here since this is a rare case which doesn't triggered during development. > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On 28.02.2023 16:03, Zhao Zhili wrote: > I can move the check from here to close(). assert is helpful if it can catch > bugs during development. It doesn't help much here since this is a rare case > which doesn't triggered during development. Crashing with an assertion is still much better than dereferencing a NULL pointer. I'll have a look at the calling function in a bit, to see if it can be refactored to never run into the chance of calling this function when not needed.
diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c index 8a28454042..dda2dc3ba1 100644 --- a/libavcodec/nvenc.c +++ b/libavcodec/nvenc.c @@ -178,6 +178,8 @@ static void reorder_queue_flush(AVFifo *queue) { FrameData fd; + if (!queue) + return; while (av_fifo_read(queue, &fd, 1) >= 0) av_buffer_unref(&fd.frame_opaque_ref); }