From patchwork Mon Nov 27 01:52:50 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pan Bian X-Patchwork-Id: 6383 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.94 with SMTP id m30csp1100780jah; Sun, 26 Nov 2017 17:53:07 -0800 (PST) X-Google-Smtp-Source: AGs4zMbj3O2w5E1UM7yIibGvzMJWfCsfrLn1W8i8KxkvL4X145nNn4tx0HjTRxJB0yzzWTcxy2JN X-Received: by 10.223.169.100 with SMTP id u91mr28578678wrc.108.1511747587031; Sun, 26 Nov 2017 17:53:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1511747587; cv=none; d=google.com; s=arc-20160816; b=IKqWYl20ZmtKFFYXLOpQiWSE4mPDgIbVl4I+8nsDL01Ry+YRqcEERmQx9IRBXKHu/t A3ZJQZymyswvJQIKZg4dp/l5vWeAu3g5LYocSng2oNm7tH+0s+Qxssvl37xFK+3HDX+U 12PZQGbdgcK7RuMGpKSY9vy27KpLX3w7fB17M8RCJLIPoKlZdHwRqiIUEa2iWfpjumpL jOXhYrWFZ8epdbmQmKTxxebInk+RiQI28+DWKoP4TLOTqrnc5G8WM4uDvt5le8nXI1NG B+CmnYrbIDJOqhbMn5eDcsgbiFZKKMGlZmlrkTE9o+Wrde7tN1M8XB1ApE566vJnvXV6 codQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=R9T65ufXAAEC2iFg22IhJA+pO5GN2hPHAeGr6OSZWxU=; b=Xc0jDPjze6ve3GcnhreCbwEJz4oqeTN/X2n2DvapiEwdY/T52Z9vItDQ5lB3bjjt9J mJ0xkV816bDLZWHkprBNQ064NyRgAiyYm73++pMbIOn/Do1mAZBoYUgtxzmIM6nCk/Im H3LN6+XvO2Db6bXXwsBJ0htvfC7jBIXHiB3KyEJcW5KF/r20MspIvOMi+hdQGssesWkN Dqr99C+IPaZjk/vdYZBTMtTACs1Ziw1C2BG0ANib3rBg2ZNT9JLIiLs8miIePzlUUUV8 yX2jZWwCwdBrVjLMlSJiCNEtVEyoStCne0i4oiRPeLNY587yDeZ+jSac1RQV3Ye76j04 mNlw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@163.com header.s=s110527 header.b=ouVmu+ts; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f5si20132693wrf.288.2017.11.26.17.53.06; Sun, 26 Nov 2017 17:53:06 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@163.com header.s=s110527 header.b=ouVmu+ts; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 13E41689E1C; Mon, 27 Nov 2017 03:53:03 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from m12-15.163.com (m12-15.163.com [220.181.12.15]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EF060689C4F for ; Mon, 27 Nov 2017 03:52:55 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:Subject:Date:Message-Id; bh=3X8aGrjsRVs4JiAo/n M2OnZkrxVRqon9qKilChn+hK8=; b=ouVmu+tsXETAarPJ6tmht4hkthEjw2KQg4 uK0oTi7k/4SVF8qiz2Zyv7V0CDz0ztyqyvgxNq/pFam9UlKj8hOPQRM8ChIQ50Wp q5DuEK0uC2CuZqsjA5ONIev4Hu0Fsvy4KOFTtBPiP7R/DnAJKXHGA0LEzg7QjShL E/Rb3IHPU= Received: from localhost.localdomain (unknown [106.120.213.112]) by smtp11 (Coremail) with SMTP id D8CowACHGlTzbxta6Da5BQ--.53000S3; Mon, 27 Nov 2017 09:52:53 +0800 (CST) From: Pan Bian To: ffmpeg-devel@ffmpeg.org Date: Mon, 27 Nov 2017 09:52:50 +0800 Message-Id: <1511747570-23415-1-git-send-email-bianpan2016@163.com> X-Mailer: git-send-email 1.9.1 X-CM-TRANSID: D8CowACHGlTzbxta6Da5BQ--.53000S3 X-Coremail-Antispam: 1Uf129KBjvdXoWrKr4xur1UJFyxZr4UWFWxZwb_yoW3Krc_ua s7Zr1vqFsrX34Ikw4Fkw4Yyr4Iyr9Yyr4xA3Z7KrsIgr9rAF4DurnFqwnYvw43W39akry3 X345try3Jr12gjkaLaAFLSUrUUUUUb8apTn2vfkv8UJUUUU8Yxn0WfASr-VFAUDa7-sFnT 9fnUUvcSsGvfC2KfnxnUUI43ZEXa7IU8XeOPUUUUU== X-Originating-IP: [106.120.213.112] X-CM-SenderInfo: held01tdqsiiqw6rljoofrz/1tbiQBadclSIUEpMaAAAsb Subject: [FFmpeg-devel] avcodec/nvenc: set correct error code X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Pan Bian MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" In function process_output_surface(), the return value is 0 on the path that av_mallocz() returns a NULL pointer. 0 indicates success, which deviates from the fact. Return "AVERROR(ENOMEM)" instead of "0". Signed-off-by: Pan Bian --- libavcodec/nvenc.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c index 79f7dce..1506062 100644 --- a/libavcodec/nvenc.c +++ b/libavcodec/nvenc.c @@ -1763,8 +1763,10 @@ static int process_output_surface(AVCodecContext *avctx, AVPacket *pkt, NvencSur } slice_offsets = av_mallocz(slice_mode_data * sizeof(*slice_offsets)); - if (!slice_offsets) + if (!slice_offsets) { + res = AVERROR(ENOMEM); goto error; + } lock_params.version = NV_ENC_LOCK_BITSTREAM_VER;