From patchwork Tue Feb 25 02:02:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fu, Linjie" X-Patchwork-Id: 17919 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 60DCA44B450 for ; Tue, 25 Feb 2020 04:06:13 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4033168B641; Tue, 25 Feb 2020 04:06:13 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 61E7C68B041 for ; Tue, 25 Feb 2020 04:06:06 +0200 (EET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Feb 2020 18:06:04 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,482,1574150400"; d="scan'208";a="230863680" Received: from icl-dev.sh.intel.com ([10.239.158.73]) by orsmga008.jf.intel.com with ESMTP; 24 Feb 2020 18:06:03 -0800 From: Linjie Fu To: ffmpeg-devel@ffmpeg.org Date: Tue, 25 Feb 2020 10:02:33 +0800 Message-Id: <1582596153-1442-1-git-send-email-linjie.fu@intel.com> X-Mailer: git-send-email 2.7.4 Subject: [FFmpeg-devel] [PATCH] lavc/qsvenc: add return check for ff_qsv_map_pixfmt X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Linjie Fu MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Return an error directly if pixfmt is not supported for encoding, otherwise it may be hidden until query/check in MSDK. Signed-off-by: Linjie Fu --- libavcodec/qsvenc.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 571a711..40ff17c 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -436,7 +436,9 @@ static int init_video_param_jpeg(AVCodecContext *avctx, QSVEncContext *q) if (!desc) return AVERROR_BUG; - ff_qsv_map_pixfmt(sw_format, &q->param.mfx.FrameInfo.FourCC); + ret = ff_qsv_map_pixfmt(sw_format, &q->param.mfx.FrameInfo.FourCC); + if (ret < 0) + return AVERROR_BUG; q->param.mfx.FrameInfo.CropX = 0; q->param.mfx.FrameInfo.CropY = 0; @@ -537,7 +539,9 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q) if (!desc) return AVERROR_BUG; - ff_qsv_map_pixfmt(sw_format, &q->param.mfx.FrameInfo.FourCC); + ret = ff_qsv_map_pixfmt(sw_format, &q->param.mfx.FrameInfo.FourCC); + if (ret < 0) + return AVERROR_BUG; q->param.mfx.FrameInfo.CropX = 0; q->param.mfx.FrameInfo.CropY = 0;