From patchwork Tue Jun 18 13:48:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 49994 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:9196:0:b0:460:55fa:d5ed with SMTP id s22csp2569545vqg; Tue, 18 Jun 2024 06:49:09 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVlt9VNE2NyqCPtt8C42Uln7bKfEJYYM1Y3tntzGM8sY6Pr0+yInY5pJqIC/RdcDLqpzsWGBNTXes/xe1UPd3KD0QlpnRpmT+HdJg== X-Google-Smtp-Source: AGHT+IH7cYZPIxiL5zXW9pT2KT3iX7KRUr/BJ6/N4N0MOstL9rRKTUcHEPVBNAq7f8rUkH5Gp3vO X-Received: by 2002:a17:907:940c:b0:a6f:5ef5:2f63 with SMTP id a640c23a62f3a-a6f60d20f0fmr957829966b.18.1718718549461; Tue, 18 Jun 2024 06:49:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718718549; cv=none; d=google.com; s=arc-20160816; b=OQsaUKGZf+orxbrdccce3/J1g3eWRvv5PjmVyWH4nWFSnvvLkEBVDC0Fx2syUNQv5r bhhMBJOdszcr7LTXqHCrsLSy+4j4Kg4TmDmikCJpZbVYdnRrYlr+LS4D31TYZFKvEwnJ bgdB2AONyAazesAFD2apJExdRa6t9auvIsp2F57CrRTHRCZReVZc4OlbnJMBQRyudVNd 8jumplNsUo3n56H9rAaSkxocveL6ZbvY5komP9L2P37DBHDF1A+WDP8uMemxxnWv2Orq jBoMSzgY0Dmne2iCS23pWpiXEvzEa/lq1omueG6KTSZD7hBDqGntRIuX1O8OjjnrDkVD hoyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=fGVp5bqbRVB5rHmW/5Yv8kJxKECZKKkIW3sGrOC+tdM=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=tw3HMVmbfdmkA/bZlOT5tcaJpD9FgdWRsdcUWOZwCMs0O9J1hNHAZJa9FC3iyI4DYj 4oNrUghIpM9K4/yZvMHlo6AnPbn3xAsGFxMWiF/7cFGXBdyNLfZUGyj4NixjxJJisqSY ufSPAJykbZec6mlUGyifOjU+JGJRGUDXvEPa4PUJSZsNvRaIpGYgh+rIMx8PPrRHYSyj 13GhCXle4YbaUCWRytgJoFFmbZ0TCUk5w5jjWmbEZesHXU71HkAgdYQCPJY45HBfHfpw mSi3LxVlQxHUOwbQ28u8sjwZFv1seBIqCFVG3hQxU06YLpwr1Wz0yPaYXiKlfbdVQ6xv eCIA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=bGN1ukip; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a6f56e26910si540917866b.804.2024.06.18.06.49.08; Tue, 18 Jun 2024 06:49:09 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=bGN1ukip; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6C31368D7DA; Tue, 18 Jun 2024 16:48:40 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D22E168D28D for ; Tue, 18 Jun 2024 16:48:30 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id 06DA2240009 for ; Tue, 18 Jun 2024 13:48:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1718718510; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CBGAXaT9TMY+GjcOlTRLYak4UIJGt7Ze3OyM4h930E0=; b=bGN1ukipFbyeq7d7CGXIHaRQMrSm7ppcZQYu/MwrDrQeImm3rwYwe6IrS1RXB4y5LCmnxU WQigIOvsd3+bqYihYwtW99Sygu5Jn4EBQ3423nx5+zsBd2sN5a+IoAaU7hXqVV/GKR0RIL tqINEO7UP736ZUI6Qssx4vlJLD/Ul9DA+d9fb5lUJR41+c8yWF25FL9yjRVJ4Ss0Dh5a54 NncD4fuKKq6Jxq/lP0AGaphnxeD/nQNEyZxIjr0N/ByWpbwy7nC7fsVhai1ZCCy5SLqdW5 h/ftZdXw8nsk5nutm9TtbLFviSu1kEq40owGHJhOifI8pjO5remKkHIP8tjd6w== From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Tue, 18 Jun 2024 15:48:23 +0200 Message-ID: <20240618134826.2189719-4-michael@niedermayer.cc> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240618134826.2189719-1-michael@niedermayer.cc> References: <20240618134826.2189719-1-michael@niedermayer.cc> MIME-Version: 1.0 X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 4/7] avcodec/snowenc: MV limits due to mv_penalty table size X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: euaHF7OFqbL8 Fixes: out of array read Fixes: 69673/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SNOW_fuzzer-5476592894148608 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/snowenc.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/libavcodec/snowenc.c b/libavcodec/snowenc.c index 8d6dabae658..dd6ce36aa54 100644 --- a/libavcodec/snowenc.c +++ b/libavcodec/snowenc.c @@ -413,6 +413,7 @@ static int encode_q_branch(SnowEncContext *enc, int level, int x, int y) int my_context= av_log2(2*FFABS(left->my - top->my)); int s_context= 2*left->level + 2*top->level + tl->level + tr->level; int ref, best_ref, ref_score, ref_mx, ref_my; + int range = MAX_MV >> (1 + qpel); av_assert0(sizeof(s->block_state) >= 256); if(s->keyframe){ @@ -454,6 +455,11 @@ static int encode_q_branch(SnowEncContext *enc, int level, int x, int y) c->xmax = - (x+1)*block_w + (w<<(LOG2_MB_SIZE - s->block_max_depth)) + 16-3; c->ymax = - (y+1)*block_w + (h<<(LOG2_MB_SIZE - s->block_max_depth)) + 16-3; + c->xmin = FFMAX(c->xmin,-range); + c->xmax = FFMIN(c->xmax, range); + c->ymin = FFMAX(c->ymin,-range); + c->ymax = FFMIN(c->ymax, range); + if(P_LEFT[0] > (c->xmax<xmax< (c->ymax<ymax< (c->xmax<xmax<