From patchwork Thu Dec 6 11:19:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fu, Linjie" X-Patchwork-Id: 11303 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 76D3D44CF8A for ; Thu, 6 Dec 2018 13:19:42 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1E70E68A800; Thu, 6 Dec 2018 13:19:34 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 92E2B68A696 for ; Thu, 6 Dec 2018 13:19:27 +0200 (EET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Dec 2018 03:19:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,322,1539673200"; d="scan'208";a="96598455" Received: from kubernetes-master.sh.intel.com ([10.239.159.155]) by orsmga007.jf.intel.com with ESMTP; 06 Dec 2018 03:19:33 -0800 From: Linjie Fu To: ffmpeg-devel@ffmpeg.org Date: Thu, 6 Dec 2018 19:19:05 +0800 Message-Id: <20181206111905.28881-1-linjie.fu@intel.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH, v2] lavc/qsvenc: replace assert with error return X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Linjie Fu MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" bs->FrameType is not set in MSDK in some cases (mjpeg encode for example), and assert on a value coming from an external library is not proper. Add default type check for bs->FrameType, and return invalid data error in function ff_qsv_encode to avoid using uninitialized value. Fix #7593. Signed-off-by: Linjie Fu --- [v2]: Add default bs->FrameType check. libavcodec/qsvenc.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 7f4592f878..917344b60c 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -1337,8 +1337,10 @@ int ff_qsv_encode(AVCodecContext *avctx, QSVEncContext *q, pict_type = AV_PICTURE_TYPE_P; else if (bs->FrameType & MFX_FRAMETYPE_B || bs->FrameType & MFX_FRAMETYPE_xB) pict_type = AV_PICTURE_TYPE_B; + else if (bs->FrameType == MFX_FRAMETYPE_UNKNOWN) + pict_type = AV_PICTURE_TYPE_NONE; else - av_assert0(!"Uninitialized pict_type!"); + return AVERROR_INVALIDDATA; #if FF_API_CODED_FRAME FF_DISABLE_DEPRECATION_WARNINGS