From patchwork Tue Dec 3 21:33:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andriy Gelman X-Patchwork-Id: 16576 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id E458444AB63 for ; Tue, 3 Dec 2019 23:41:27 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D318268B4A3; Tue, 3 Dec 2019 23:41:27 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oi1-f193.google.com (mail-oi1-f193.google.com [209.85.167.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6531E68A8FA for ; Tue, 3 Dec 2019 23:41:26 +0200 (EET) Received: by mail-oi1-f193.google.com with SMTP id a124so4787486oii.13 for ; Tue, 03 Dec 2019 13:41:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=pxkEgJdsgCWjuOyuLZjiwXOHnqKKIBHAOO9UaUPKhwM=; b=Y0diHmEGGBRTQ03br/WdZ2LxTHrpaZLqvToyfBfO4sObEE/9JAZGQ6SeZR0Tg1q5Jz HSAt1kjLqeeWdtIOCvkQ+PdRBV/9pWX8gmtRvPeaUfvR+z7JyCHQUhE6Eb+7AHs5Kh03 2phGXLBs6Mo7wLjKqCaOTsVfW3Qv1ktUsnNZkhrYzFQMes5VsWl5xd7u65qKF3Fex/Mk bAgS6HiCGsF9KgfwG0JryL72v/TTwdNWEbBKXgTUcELJEdzlaBUNbQDHtMYba9CyUZew sfiM45yi16SdamfU1pzcT1apCRoD1Vc6/qYjOU2YQphTo3LEqv2+jsZg6ajV39+sqnBB ieVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=pxkEgJdsgCWjuOyuLZjiwXOHnqKKIBHAOO9UaUPKhwM=; b=pooWAd6tzy3ylCY1RaRhasSfYdwEvFN19CjLYsVuD8FzNZOXW2JGHAbf35SzA7MflF 4fG9VhGca1L9NCjE8PCrH7Ep6MLuKwGneOk7RLxP9DymQNp4S3e/e3O0SoK8hNUES3wJ XQC1KK7P07F/5Nud9Jsz3rb2t+FiwgKCV9gTdndDqhKWZd0qW7h5EgokNhIRmdQWOm0N OtDjGkTVn9Pbcce8Av+ARWmYOcuFLenvEdaDGz6BFJWCsnitJfxFnjsUWrt4OrCVf15v 0MbuU/bYH5i3h7AhYyakjNDqSiuc5zFGkPAzhfUBd2+0jpXSdtT0SXIQLICMFMofyXtQ Fcew== X-Gm-Message-State: APjAAAW6wM6JsCrXxRWRJPjvInKBMV5xw2QGvMDTJIV/DxxTgS2eFCgj huJ2qF5RMpwp1qVKLqmSeeWCocsu X-Google-Smtp-Source: APXvYqwVN1MWC9kFfv7AA7iWic9BIIwvHZCF1z4fVNktZNLNZxAKLgEdy0P6RNEKUTLnYoI5cRWD+g== X-Received: by 2002:aca:cdd6:: with SMTP id d205mr83764oig.90.1575408844682; Tue, 03 Dec 2019 13:34:04 -0800 (PST) Received: from manj.guest.slb.com ([192.23.22.48]) by smtp.gmail.com with ESMTPSA id q5sm1580988oia.21.2019.12.03.13.34.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Dec 2019 13:34:04 -0800 (PST) From: Andriy Gelman X-Google-Original-From: Andriy Gelman To: ffmpeg-devel@ffmpeg.org Date: Tue, 3 Dec 2019 16:33:55 -0500 Message-Id: <20191203213356.17842-1-andriy.gelman@gmail.com> X-Mailer: git-send-email 2.24.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] lavc/hevc_mp4toannexb: Fix interger overflow X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andriy Gelman Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Andriy Gelman Check packet grow size against INT_MAX instead of SIZE_MAX. Found with libFuzzer: 4294967044 cannot be represented as int. Signed-off-by: Andriy Gelman --- libavcodec/hevc_mp4toannexb_bsf.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavcodec/hevc_mp4toannexb_bsf.c b/libavcodec/hevc_mp4toannexb_bsf.c index 09bce5b34c2..36fd6f0b15c 100644 --- a/libavcodec/hevc_mp4toannexb_bsf.c +++ b/libavcodec/hevc_mp4toannexb_bsf.c @@ -152,8 +152,8 @@ static int hevc_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *out) extra_size = add_extradata * ctx->par_out->extradata_size; got_irap |= is_irap; - if (SIZE_MAX - nalu_size < 4 || - SIZE_MAX - 4 - nalu_size < extra_size) { + if (INT_MAX < 4 + nalu_size || + INT_MAX - 4 < extra_size + nalu_size) { ret = AVERROR_INVALIDDATA; goto fail; }