From patchwork Fri Sep 20 13:41:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 51681 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:d154:0:b0:48e:c0f8:d0de with SMTP id bt20csp979908vqb; Fri, 20 Sep 2024 06:42:33 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCU1JUtpoJz4u5/qmuvWl8Jr9SKqiPwRzuLEg05oZnDBMUxYfij6lr/5reI5bc2JFqMqkSf5C/hRg6cb0BmooSnJ@gmail.com X-Google-Smtp-Source: AGHT+IHb4bCRqAdBw1yWakQT+oeRae0Czh8mYwLoWqvfwub+8/We3ijfCABs9tdBOEC/tzpu0t6a X-Received: by 2002:a05:6402:4411:b0:5be:f3ae:b9ce with SMTP id 4fb4d7f45d1cf-5c464a5d37cmr2328056a12.27.1726839752638; Fri, 20 Sep 2024 06:42:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1726839752; cv=none; d=google.com; s=arc-20240605; b=ETHdBg049X9ScG5bzYe16UE/z89sGRxtwa8vRotSlw4ZLLSZXaoJcAPPagpFF8y1G7 /RCsOVYpIfdt1B1ic7yC1mf7gp9iC7w09nQbfM9fk8D4tr1kTpfsV3Z3Ac0HiVyrgueO ggoPRn8aEd5INEtkMSh+e2C5b70sAatWjWJqocJVo1Fb5ws5j4EphL4h1y1vZ7dBbT2V MUDTfiriudip4mZ0ZvSp16nM0TPKXSueAXEBAXUvcy3aTgTCnToYi/+78PizE8SmJI8i Um7KMd7wocHT83JAmTLLcHNA7d6y9zS8dP5PkLR74T2l94oSQCtqOUTw4YGWnH+fL7Nm MDkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:date:to:from:message-id :dkim-signature:delivered-to; bh=jmU9hfcaa5IDsYTlffVwCB59OJzThr2mGQqy37GeStg=; fh=HnHYuZ9XgUo86ZRXTLWWmQxhslYEI9B9taZ5X1DLFfc=; b=BJ4TeC03mtqNkCCG/AuilUmBD/t1+41O/ZCS0/EQMrWtxjijBeQs0H3YWAXCpvMuEN SAqgQ/STQs+bADfWzqlvtjWmaadzod50HNJnxw8EXnj6GYKHKJ33Vy4Kkjhbf7R1PSEy OXtd98iVA4e32u55GvXI8ZcQOyQnx8HjBjO6q0Gp2aiIMasFSDTohlRwgWXH9RCOBbj4 56dgIIUZwL/85bolekdffyZvLoioX83Yo+fiVSKX0h3veUtdEhT+86Mammyt1T7OAkeK yEG7yHN/MDNi+cxPnXKLvfzQskWN5JrxGEEAgWlqKmKxtoQMsmc8HR6DIH/T/n+GNsuE YDuw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b="VKnl/lom"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5c42bcb9bb4si9195989a12.686.2024.09.20.06.42.31; Fri, 20 Sep 2024 06:42:32 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b="VKnl/lom"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B1EC068DBA8; Fri, 20 Sep 2024 16:42:27 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-57-49.mail.qq.com (out162-62-57-49.mail.qq.com [162.62.57.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 24DE468DC78 for ; Fri, 20 Sep 2024 16:42:18 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1726839729; bh=JSd3LgkeoF0acos475clDzuFkSEQS1LhAgcwwngztJY=; h=From:To:Cc:Subject:Date; b=VKnl/lom6BPUfiecoSC54TBkC/j4dwYnjxtw/uRjF2sDsu4zu6v7IUbPIviM2AS60 SsgAhYmf6ea/0VU9L1MqcZ3eQjuzDO1hCIIq/odMlOOWuQPZWmjhDl5f6IhZAo5WBU zgVIDj+meIOJAyaES0SPuzxHGVRxjdbJ5QPE7RPI= Received: from ZHILIZHAO-MB1.tencent.com ([119.147.10.198]) by newxmesmtplogicsvrszc5-2.qq.com (NewEsmtp) with SMTP id A87B3AE0; Fri, 20 Sep 2024 21:42:07 +0800 X-QQ-mid: xmsmtpt1726839727t8kavnhka Message-ID: X-QQ-XMAILINFO: MOpJcPT3Yy24DHNByNm6wbhAo+FwWc1DEO3Omz2X10LasvIZvTbV9ekX6SsxaB ecd1ov7JNY2mb6w/ibIyvw8Pbm0RYRqEgZMCM+a52xcCV7Penc0NNvC5Llc7Gw2ATqVncetlknfi HxAfSkh45eutSyvmJ/dQBJuReseynsvv7XMOz/RgH8kMG3RKEEgCyPYkg8ObeKaV1dRzlsagYQBw syyLzxOEEdQg1QNBuTXUirBQrM9Bg2FThcfz2G9vJmuFUoyBxbOykDKAKhgr6k1POTbK/0LTNGql mDNPaxLlKtBsMWMwTcLn9viQvERYI41KNeqMlq3tCrcEYPFr2gn9dj/jkmFthPHwJnePd668qnfv au0V1RTslae9yggY9sHiBgdiKwjqwZZdirRB0Js+PsQ2+0Tcy/kSvIck/PbXcjFhqViTsFb1U1/a h6nBXxwWNpy+xAU5evhq749Qb4NJ4/xkb+DEbCF6gU+gf6mfHAskikvGoeQsNN9UsPF9trfJzwhI +GqytJeHYMx5tx1LnIm8PQ5xOcWVYKJIuUYr7iy6GvaJbA+XhNC99bOzM+HmXK/EQQcKzaDgfiiD 7M+jOlbGJXZsrQ7TFq2Wkw23Clrwnm0NMma0HlfYDMKAG2/LBq3EJo4BZ1S9d1IqDf7sfndW/ymz E8fGyC2TjWpsEBh6oHxLfG6bZLgYY0apyh7bCmUG0VXGMci1Y1g5HhBRP8oCHfWkfVbxQO06E42n NEXG4ubyW0QQ5Ft5/ZBd7vvL+H2oDQMoi7weTFRAykBL+BtBiah2P0i01HmcPJasObMQDu9cSGKb bwNRnQsIGeP9FLH3oa+HCJJyzeQt9uEI6OrDxnsTCw/fltaW2QP++zmbxP0Vl6XQ07bbiO/IB8sc YbUnRnKctBooGZ3Dmx7mjZo8qhhuXch94Egh06p8wC/DRYnytlKjVQkgH8w+OB6mKUtZyM3p89Ul W05AQKfib3TSDIUI36qigb0vyKTohvBqaDi4FydPnRfUB2DrA/PmjMLdZ/rvr5EJCO8KPexMk= X-QQ-XMRINFO: Mp0Kj//9VHAxr69bL5MkOOs= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Fri, 20 Sep 2024 21:41:59 +0800 X-OQ-MSGID: <20240920134159.83093-1-quinkblack@foxmail.com> X-Mailer: git-send-email 2.42.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3] avcodec/vvc: Don't use large array on stack X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: T98PDVdCeVfG From: Zhao Zhili tmp_array in dmvr_hv takes 33024 bytes on stack, which can be dangerous. --- libavcodec/vvc/inter_template.c | 28 +++++++++++++--------------- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/libavcodec/vvc/inter_template.c b/libavcodec/vvc/inter_template.c index c073a73e76..187d557853 100644 --- a/libavcodec/vvc/inter_template.c +++ b/libavcodec/vvc/inter_template.c @@ -541,11 +541,13 @@ static void FUNC(dmvr_v)(int16_t *dst, const uint8_t *_src, const ptrdiff_t _src static void FUNC(dmvr_hv)(int16_t *dst, const uint8_t *_src, const ptrdiff_t _src_stride, const int height, const intptr_t mx, const intptr_t my, const int width) { - int16_t tmp_array[(MAX_PB_SIZE + BILINEAR_EXTRA) * MAX_PB_SIZE]; - int16_t *tmp = tmp_array; + int16_t tmp_array[MAX_PB_SIZE * 2]; + int16_t *tmp0 = tmp_array; + int16_t *tmp1 = tmp_array + MAX_PB_SIZE; const pixel *src = (const pixel*)_src; const ptrdiff_t src_stride = _src_stride / sizeof(pixel); - const int8_t *filter = ff_vvc_inter_luma_dmvr_filters[mx]; + const int8_t *filter_x = ff_vvc_inter_luma_dmvr_filters[mx]; + const int8_t *filter_y = ff_vvc_inter_luma_dmvr_filters[my]; const int shift1 = BIT_DEPTH - 6; const int offset1 = 1 << (shift1 - 1); const int shift2 = 4; @@ -553,19 +555,15 @@ static void FUNC(dmvr_hv)(int16_t *dst, const uint8_t *_src, const ptrdiff_t _sr src -= BILINEAR_EXTRA_BEFORE * src_stride; for (int y = 0; y < height + BILINEAR_EXTRA; y++) { - for (int x = 0; x < width; x++) - tmp[x] = (DMVR_FILTER(src, 1) + offset1) >> shift1; + for (int x = 0; x < width; x++) { + tmp1[x] = ((filter_x[0] * src[x] + filter_x[1] * src[x + 1]) + offset1) >> shift1; + if (y > 0) + dst[x] = ((filter_y[0] * tmp0[x] + filter_y[1] * tmp1[x]) + offset2) >> shift2; + } src += src_stride; - tmp += MAX_PB_SIZE; - } - - tmp = tmp_array + BILINEAR_EXTRA_BEFORE * MAX_PB_SIZE; - filter = ff_vvc_inter_luma_dmvr_filters[my]; - for (int y = 0; y < height; y++) { - for (int x = 0; x < width; x++) - dst[x] = (DMVR_FILTER(tmp, MAX_PB_SIZE) + offset2) >> shift2; - tmp += MAX_PB_SIZE; - dst += MAX_PB_SIZE; + if (y > 0) + dst += MAX_PB_SIZE; + FFSWAP(int16_t *, tmp0, tmp1); } }