| Message ID | 20240621060820.3215-1-hguandl@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel] avcodec/videotoolboxenc: fix vtctx reset condition | expand |
| Context | Check | Description |
|---|---|---|
| yinshiyou/make_loongarch64 | success | Make finished |
| yinshiyou/make_fate_loongarch64 | success | Make fate finished |
| andriy/make_x86 | success | Make finished |
| andriy/make_fate_x86 | success | Make fate finished |
diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c index 15c34d59c3..0aa4885232 100644 --- a/libavcodec/videotoolboxenc.c +++ b/libavcodec/videotoolboxenc.c @@ -2766,7 +2766,14 @@ static int vtenc_populate_extradata(AVCodecContext *avctx, pe_cleanup: CVPixelBufferRelease(pix_buf); - vtenc_reset(vtctx); + + if (status) { + vtenc_reset(vtctx); + } else if (vtctx->session) { + CFRelease(vtctx->session); + vtctx->session = NULL; + } + vtctx->frame_ct_out = 0; av_assert0(status != 0 || (avctx->extradata && avctx->extradata_size > 0));
In vtenc_populate_extradata, the cleanup function vtenc_reset should not be used when no error occurs, otherwise some color information is lost (#11036). This patch checks the status code and conducts the correct cleanup. Signed-off-by: Hao Guan <hguandl@gmail.com> --- libavcodec/videotoolboxenc.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)