From patchwork Sun Feb 25 18:44:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean McGovern X-Patchwork-Id: 35065 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c51b:b0:19e:cdac:8cce with SMTP id gm27csp996437pzb; Sun, 25 Feb 2024 10:44:48 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCW0/Y8aOGK9Mczfqruog4tyC3IQj1onMtQGB7xuwEA8IfydhVnjNZ8PzLl09wMdkYOj6q5pgMqZeC2KM/Q5uH3QgXZpIB+PbOLDhQ== X-Google-Smtp-Source: AGHT+IED6DAuSlm7Dc2sI7RmsQuVyz02/9njfc/fIdPeCFUjVzcFAduxhFuO6u0R3IMRxiGjzmOQ X-Received: by 2002:a17:906:fb07:b0:a43:4876:9840 with SMTP id lz7-20020a170906fb0700b00a4348769840mr465269ejb.60.1708886688660; Sun, 25 Feb 2024 10:44:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1708886688; cv=none; d=google.com; s=arc-20160816; b=Qe7Ephc7oGkmL60jttJwm1vgjvzY4XtQUcAN9ENOfEp5pamEFMjKv98cWOmEfPSK0g +5346wvKXLGQDTwqc8wye/Ym2R9DC6r/XDoHxDHcftzTjla+TLl/+QLPzh/redZE1yBX d7SHdDfaeUGxGOaAn+LrA3uNzAGzqnyA0dJIF0W+mGnpT5g0H+0LGNB3l50av36lbf06 dPHuGOHqGWH0FyZtqKMpTEqsQjzayoJ5f6sQ90dyqssz+QroqMJda+ycBbDfjGbd5uLZ 2Uembbg6PBwkNlCx8BU5FnjAZr+zHDadMjtAPbL8xqxcLJDUdqpzcekYS2ackDEORFsa 7IKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=1NUJFWnqrbkWXRocsuSfUm4AeqlzzqlSdqdyZCRzL5M=; fh=lg22JINJRPRV4WRaAhq1IuUubVLOATzDRndR8Kmxcbc=; b=EvqIYVoEX55s8xohH2AdVT8y6ASZpaE2WbTHlRTlBDj0eJfX7muzyRPR9Wy4obduLl R6f7BQ3s9ia4iSoPtTPr1G5D5rYnuj0q/knKJQxL5ebmqsp1JeG9MhgA7e//CkNFFdz5 bkaGMekZP6dAs5x/9WLF1CQQ3O/DZa4dX7cBp+t8seshAXuWlu9UTotfiR7yid7K21ob 3dzZTOKtqbT6QXI/lcC1pgo3aaxETL+52ci/bvgEU2Y//YJh6NKRDiAISzRJKo61N4/0 y7qt5HUYx63rhM9f2bTx1tNvtdhpoNd1HYMiYyl7pw6dAdR23ALQkyNte94vD8MB5ljl IX+Q==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b="MqoCW/FC"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hh12-20020a170906a94c00b00a433186d6ebsi503601ejb.897.2024.02.25.10.44.48; Sun, 25 Feb 2024 10:44:48 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b="MqoCW/FC"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3026068C733; Sun, 25 Feb 2024 20:44:46 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yb1-f172.google.com (mail-yb1-f172.google.com [209.85.219.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6026B68A0A2 for ; Sun, 25 Feb 2024 20:44:39 +0200 (EET) Received: by mail-yb1-f172.google.com with SMTP id 3f1490d57ef6-dcd9e34430cso2543945276.1 for ; Sun, 25 Feb 2024 10:44:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708886677; x=1709491477; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=xliuqyLa26Q8pdJM4L0lhoDUWp+0OaAIfZ1/n6pZuhg=; b=MqoCW/FCzG6jHvcIeGbgzX3Df58ej8xaZzOUHU8noTRSqxbfptRB4h+XpdJFTEUE+s HmbnbgqnQNGGEtQyX67Lbm8yQnfC/tW5W9v8QOEiScwGJCdJ4jwzdesKIzdqXua8Hvwf wkB7/AdVu54zwt8QLk8ckAdREJuctjxtYaWXz9qnF8rKMuJA2HuVR22rm/ODBxzuWSQ8 pHBnRyLTfHITQyWVIuxC8xdOPlFMCwYPlRa1i+SScXghAH4q39cEgog6ZFnWNp4qY+LA uIa/wGCBkrvImA0E0DfoLNeERHgcQh0glnnKcwgQ725JJWY9c9jUdWN7czsmxu8a4xqM 9f5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708886677; x=1709491477; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=xliuqyLa26Q8pdJM4L0lhoDUWp+0OaAIfZ1/n6pZuhg=; b=ducdFeaLg5ww4SJQHTBAiBvGHPfDnw+Y8iBefIqBqzEy3LimeUYHxNaiyI5rSBecnb QMmIsMiYzAteGbiFhQuVEadJWU8QuWumC6rcLNChBYFDeby02+kI9sYH6lWeT9h+4qrn M+feBTEBktFUJTGEFkBRj9MHW87KTklR7OV8EM1dOftrN6GOkw9dcA11IHPYdfK/n13r vqCLaQ4/8tuL673RH1cYPbnhSlyHDfP82/3syFtHI+q4INnS3EdtKKP7riijMAa865uw 4mHfCoOjUwcY/aze+jUiMQvmELLsuu95jhUwntQQ0eUkaQ8f/aiN2DVvDr8WCkuUzn61 sokA== X-Gm-Message-State: AOJu0YyHW0ldKgM3ToXVswBVlQhgRPuA0roJzvAeyvTWWhBqDLhLFlOd mA7Wl7L4wpqr082lhNzmjrGuC+cWF4zpPLSOslWmNo5iJXD31eeXpGGjUFSz X-Received: by 2002:a0d:d643:0:b0:608:c5eb:99b6 with SMTP id y64-20020a0dd643000000b00608c5eb99b6mr4586101ywd.2.1708886676704; Sun, 25 Feb 2024 10:44:36 -0800 (PST) Received: from elysia.seanmcgovern.ca ([2607:f2c0:9307:7100:48:9add:e8c7:742c]) by smtp.gmail.com with ESMTPSA id l37-20020a81ad25000000b00608d55efe59sm791941ywh.112.2024.02.25.10.44.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Feb 2024 10:44:36 -0800 (PST) From: Sean McGovern To: ffmpeg-devel@ffmpeg.org Date: Sun, 25 Feb 2024 13:44:28 -0500 Message-Id: <20240225184429.2962707-1-gseanmcg@gmail.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 0/1] fate-aac-encode-pred UBsan fix X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Sean McGovern Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: sjSCxEUefSUY From: Sean McGovern Hi FFmpeg-devel, I've started looking into the results posted by the UBsan FATE node -- http://fate.ffmpeg.org/history.cgi?slot=x86_64-archlinux-gcc-ubsan Here is the error reported by FATE (snipped for brevity) for 'fate-aac-encode-pred': [aist#0:0/pcm_s16le @ 0x77fee40] Guessed Channel Layout: stereo Input #0, wav, from '/srv/VIDEO/fate-suite/audio-reference/luckynight_2ch_44kHz_s16.wav': Duration: 00:00:09.50, bitrate: 1411 kb/s Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, stereo, s16, 1411 kb/s Stream mapping: Stream #0:0 -> #0:0 (pcm_s16le (native) -> aac (native)) Output #0, adts, to '/home/sean/build/ffmpeg-ubsan/tests/data/fate/aac-pred-encode.adts': Stream #0:0: Audio: aac (Main), 44100 Hz, stereo, fltp, 128 kb/s Metadata: encoder : Lavc aac src/libavcodec/aacenc_pred.c:169:48: runtime error: index 41 out of bounds for type 'uint8_t [41]' threads=8 Ignore the thread count, this fails the same when threads=1 Here is the loop it fails in -- for this test 'sfb == 49': libavcodec/aacenc_pred.c:165: for (w = 0; w < sce0->ics.num_windows; w += sce0->ics.group_len[w]) { libavcodec/aacenc_pred.c:166: start = 0; libavcodec/aacenc_pred.c:167: for (g = 0; g < sce0->ics.num_swb; g++) { libavcodec/aacenc_pred.c:168: int sfb = w*16+g; libavcodec/aacenc_pred.c:169: int sum = sce0->ics.prediction_used[sfb] + sce1->ics.prediction_used[sfb]; ... Here is the array definition for prediction_used: libavcodec/aac.h:188: uint8_t prediction_used[41]; Here is the line num_swb is set on: libavcodec/aacenc.c:899: ics->num_swb = tag == TYPE_LFE ? ics->num_swb : s->psy.num_bands[ics->num_windows == 8]; (side note: interesting, I've never seen a boolean condition used an an array index before...) My attached patch just uses FFMIN to ensure we don't navigate past the end of ics.prediction_used[] It corrects the undefined behaviour, but it feels naïve. Any thoughts? Sean McGovern (1): aacenc_pred: prevent UB in ff_aac_adjust_common_pred() libavcodec/aacenc_pred.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)