From patchwork Wed Dec 4 00:04:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andriy Gelman X-Patchwork-Id: 16577 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 6D5E644AF0A for ; Wed, 4 Dec 2019 02:04:25 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 552E968B0E0; Wed, 4 Dec 2019 02:04:25 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ot1-f68.google.com (mail-ot1-f68.google.com [209.85.210.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A713568AD4F for ; Wed, 4 Dec 2019 02:04:18 +0200 (EET) Received: by mail-ot1-f68.google.com with SMTP id x3so4667814oto.11 for ; Tue, 03 Dec 2019 16:04:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=F4G+qAp7nfU0j4y6TELNKqcAiuZUCJuHqMB/V370ZH0=; b=cVGd9Hx2M5hLt4Eq1AhHosLCbrIIXYqzS/R9y0WiC7h/83ldxjgDnVzolOqSgQJ0+J QVUDxSvYTtiYEk+s9gEfz+pqhRu/BNMxav7jU7qs5+awh07MnCCg87WxPzevGaiq7LAy +rsecz8fm4fonVDt7zhfvZNb0S1/h6IQsY90P+UbSCma7JrBVtvmPbeO7twPV/0yZ0TR W56doGdL6SkdgFt3iF7BjpXMa/nJafPgkOy17trZAbRe4g3T3bLV46TvNtvXwRON6nFG TygnJ4s0JvldcSTLh7mnmPHgxnAoH+49e2HS5mskwYSz+NQqst+O8NDz2dI7cL2h6M4r XGtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=F4G+qAp7nfU0j4y6TELNKqcAiuZUCJuHqMB/V370ZH0=; b=Nr2/XOwV1V73w0MmHR1V9aH2hp6Uu1hrYe9UKQG/8PKrYdNM38GQD1HjksUCWE0WKm F+Ep0LlYELcjDDQdmdO5H0CjDslUJGV2f2pFz0UIzToCpl3YpoSz1PVWV17MiRXIRr2M kFTJS39VtO0WnCigVZCjHMqZ4JgbolpoywrlO9FQdXkRXLxxnA8WjJ4kXLMtiu9x1tU1 Rbe6G6ChNo0Al0pWORCbi/QCblMs/KzxGFFrOTFRLqjLOEM8CYdqpUPL8l6oO59av9ji vrMzwWaOYgD5FgIbUbh6ROwkcgCs2X+6SjnzewRQ0pQqRGTpgNo+Om2oJUgpDPSdqGM1 OM9w== X-Gm-Message-State: APjAAAVQN8eZgLDmmawRAl7F/73GOEFSY5GrZtqzkAwWR4jdLk0dzt7L Oqa7POgsrcGwYn/ItN2sCmBUZ5U2 X-Google-Smtp-Source: APXvYqz33Qk6YEbCFT+YbUovdngQw9Y0RkBQMyKinrYwTZUj0BQWR9DJzfKQea7lTzaxAxZQEmzRDg== X-Received: by 2002:a9d:4c08:: with SMTP id l8mr484395otf.360.1575417856941; Tue, 03 Dec 2019 16:04:16 -0800 (PST) Received: from manj.guest.slb.com ([192.23.22.48]) by smtp.gmail.com with ESMTPSA id t3sm1611013otq.32.2019.12.03.16.04.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Dec 2019 16:04:16 -0800 (PST) From: Andriy Gelman X-Google-Original-From: Andriy Gelman To: ffmpeg-devel@ffmpeg.org Date: Tue, 3 Dec 2019 19:04:09 -0500 Message-Id: <20191204000411.29765-1-andriy.gelman@gmail.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: References: MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/3] lavc/hevc_mp4toannexb: Fix integer overflow X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andriy Gelman Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Andriy Gelman Check packet grow size against INT_MAX instead of SIZE_MAX. Found with libFuzzer: 4294967044 cannot be represented as int. Signed-off-by: Andriy Gelman --- libavcodec/hevc_mp4toannexb_bsf.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/libavcodec/hevc_mp4toannexb_bsf.c b/libavcodec/hevc_mp4toannexb_bsf.c index 09bce5b34c2..bf4c25b3b7b 100644 --- a/libavcodec/hevc_mp4toannexb_bsf.c +++ b/libavcodec/hevc_mp4toannexb_bsf.c @@ -66,7 +66,7 @@ static int hevc_extradata_to_annexb(AVBSFContext *ctx) for (j = 0; j < cnt; j++) { int nalu_len = bytestream2_get_be16(&gb); - if (4 + AV_INPUT_BUFFER_PADDING_SIZE + nalu_len > SIZE_MAX - new_extradata_size) { + if (4 + AV_INPUT_BUFFER_PADDING_SIZE + nalu_len > INT_MAX - new_extradata_size) { ret = AVERROR_INVALIDDATA; goto fail; } @@ -152,8 +152,8 @@ static int hevc_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *out) extra_size = add_extradata * ctx->par_out->extradata_size; got_irap |= is_irap; - if (SIZE_MAX - nalu_size < 4 || - SIZE_MAX - 4 - nalu_size < extra_size) { + if (INT_MAX < 4 + (uint64_t)nalu_size || + INT_MAX - 4 < extra_size + (uint64_t)nalu_size) { ret = AVERROR_INVALIDDATA; goto fail; }