| Message ID | 20181207101433.28152-1-linjie.fu@intel.com |
|---|---|
| State | Accepted |
| Headers | show |
2018-12-07 11:14 GMT+01:00, Linjie Fu <linjie.fu@intel.com>: > bs->FrameType is not set in MSDK in some cases (mjpeg encode for example), > and assert on a value coming from an external library is not proper. > > Add default type check for bs->FrameType, and return invalid data error in > function > ff_qsv_encode to avoid using uninitialized value. > > Fix #7593. > > Signed-off-by: Linjie Fu <linjie.fu@intel.com> > --- > [v2]: Add default bs->FrameType check. > [v3]: Add log message. > > libavcodec/qsvenc.c | 10 ++++++++-- > 1 file changed, 8 insertions(+), 2 deletions(-) > > diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c > index 7f4592f878..96ffe8a639 100644 > --- a/libavcodec/qsvenc.c > +++ b/libavcodec/qsvenc.c > @@ -1337,8 +1337,14 @@ int ff_qsv_encode(AVCodecContext *avctx, > QSVEncContext *q, > pict_type = AV_PICTURE_TYPE_P; > else if (bs->FrameType & MFX_FRAMETYPE_B || bs->FrameType & > MFX_FRAMETYPE_xB) > pict_type = AV_PICTURE_TYPE_B; > - else > - av_assert0(!"Uninitialized pict_type!"); > + else if (bs->FrameType == MFX_FRAMETYPE_UNKNOWN) { > + pict_type = AV_PICTURE_TYPE_NONE; > + av_log(avctx, AV_LOG_WARNING, "Unkown FrameType, set pict_type > to AV_PICTURE_TYPE_NONE.\n"); > + } > + else { Please fix the indentation. > + av_log(avctx, AV_LOG_ERROR, "Invalid FrameType:%d.\n", > bs->FrameType); > + return AVERROR_INVALIDDATA; > + } Carl Eugen
diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 7f4592f878..96ffe8a639 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -1337,8 +1337,14 @@ int ff_qsv_encode(AVCodecContext *avctx, QSVEncContext *q, pict_type = AV_PICTURE_TYPE_P; else if (bs->FrameType & MFX_FRAMETYPE_B || bs->FrameType & MFX_FRAMETYPE_xB) pict_type = AV_PICTURE_TYPE_B; - else - av_assert0(!"Uninitialized pict_type!"); + else if (bs->FrameType == MFX_FRAMETYPE_UNKNOWN) { + pict_type = AV_PICTURE_TYPE_NONE; + av_log(avctx, AV_LOG_WARNING, "Unkown FrameType, set pict_type to AV_PICTURE_TYPE_NONE.\n"); + } + else { + av_log(avctx, AV_LOG_ERROR, "Invalid FrameType:%d.\n", bs->FrameType); + return AVERROR_INVALIDDATA; + } #if FF_API_CODED_FRAME FF_DISABLE_DEPRECATION_WARNINGS
bs->FrameType is not set in MSDK in some cases (mjpeg encode for example), and assert on a value coming from an external library is not proper. Add default type check for bs->FrameType, and return invalid data error in function ff_qsv_encode to avoid using uninitialized value. Fix #7593. Signed-off-by: Linjie Fu <linjie.fu@intel.com> --- [v2]: Add default bs->FrameType check. [v3]: Add log message. libavcodec/qsvenc.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)