From patchwork Mon Sep 5 09:30:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Martin_Storsj=C3=B6?= X-Patchwork-Id: 37672 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:139a:b0:8f:1db5:eae2 with SMTP id w26csp2504249pzh; Mon, 5 Sep 2022 02:30:40 -0700 (PDT) X-Google-Smtp-Source: AA6agR40OU+Vuc9Dj6obWVK5U1cP/P2B43UdRxV5ux3SRi0qG6vjOeVgLWKG9BUE2uhg2UaB8xNB X-Received: by 2002:a17:906:5d04:b0:722:f46c:b891 with SMTP id g4-20020a1709065d0400b00722f46cb891mr34730382ejt.4.1662370239746; Mon, 05 Sep 2022 02:30:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1662370239; cv=none; d=google.com; s=arc-20160816; b=aP2YZttB/ceWcVXxBO0QwKkyUN+T5O6U73SwduG4438/Ky/JE/TikFO/WN9SvtDzl5 R07J7wEjUZusseI8ClcA/WMyKLJkPcQ1OkBpSUp7adjCqHiKuGe0m8NImuxOEvGg2ns0 2desIXGMQQxepWP1AHxteVHVuv+Zl40qhy/Y6u8MS9kFjN3JAlRSFaTOOhU11tFynzHO SfOzjzgQjg7nlRECuvevJ8lXNq2zzCYERy/AjUN44VE1subqpNzUy7Zi/rD2AZsZREMJ 8IRASz/YY7sfMRt4zgpgcasjX3AbSIiFN4T67llydxxWXCiMxqhtZP9+6y05yCkmR6sj HDMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=nqGtdHTfFhJgtV35c/ZhKUq6iPwFAw8eNOymR7mLZ34=; b=rw1V2IQ9XQRGM3vT8R+Rd3XWu2wCjwwmiRDxGZ7UiUq4r5CX+Go7xBa7FXF3Un2DP/ 8uyRbyK1YIHEpFHkmQwA4G50V6sRP6bXW2vChU0IoqeE2LzHgkyrWlrg2hCe3cZyqSl5 o1cBKFL9yEoWFhcc4W+4fcq0lqn6ngekzfSyuPek5ypdCRRNyiNsGVZIlUEoxizmp3Lf +ty2mNfTOZ9byHV5RWI3jwhd75h3AD5DPp4DN9TJQvqr7AMEZaSjW4NjMqqlTmXCKdLD X7Uj17CVr/SFBjniPmL0fE+8otxPrpH544UTKC0Ulk9I9TjOI8Og3zeUnTigS9HG6DZO nHrw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@martin-st.20210112.gappssmtp.com header.s=20210112 header.b=EBL3OkJW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r6-20020a50d686000000b00447b4fdc29fsi7621749edi.82.2022.09.05.02.30.39; Mon, 05 Sep 2022 02:30:39 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@martin-st.20210112.gappssmtp.com header.s=20210112 header.b=EBL3OkJW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E17B768A867; Mon, 5 Sep 2022 12:30:35 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 348F868B703 for ; Mon, 5 Sep 2022 12:30:29 +0300 (EEST) Received: by mail-lf1-f42.google.com with SMTP id v26so12253179lfd.10 for ; Mon, 05 Sep 2022 02:30:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=martin-st.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date; bh=2CjJGIYOIGX9QkT9zD1XWOHjJvhUh3oo3QKI9Vwf/4g=; b=EBL3OkJWsaobEBrITGHbjgsBvkTibU5Myhba8fht8/T9mRGY1N6VPBRDHf0UGd2G3n QdjXfMevsrwO1qOXe49aM7exKwiHH48jzkn+iiUPvul1sRrqyNqdB2mXV0B3FoS+fx6p 5CeMFk6maBVKr2U0+PnQILowEMWAwDrqZ8bQdIFDXCVvlpHh+o39OFoydhyQLfQyn2VZ WP7BdaaNfWvG4ci9xjcXoneTfWvzE+GHwm4M/aDe15Nx5rsz+6TimjDkr9zHnYlFv85+ qKY/S6G6Ek4ayUhSY/PE/m0YcWTxA/jCPllNemSSZsYMYiLmHcQdh3mtYmkfL3S+euDZ 5E+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date; bh=2CjJGIYOIGX9QkT9zD1XWOHjJvhUh3oo3QKI9Vwf/4g=; b=feYOb3Bif1gFEF6Lz17nsTfA/sAnMgSA1XyIXnDc0XY2OZA1bn9hP+gdqABwMvYvxp z68GmjyHgnJg1YjfBR3aTWwP1J9SKTMJ/ZhjqK7UUiuLGCBm7Bw5HNSZPaUkHW0MSgQx SOPNT+yDGqgvuCMrGhydmzqLxAMtJhrZBCODNqdmeUiiCl3+wp/oO7qKaA7ctjtm9ljb JeSLELrzCriW+9dYxsCZHLZ1EYT1SC22cVu/WV5pqGRZcu1o84k6nhBr4y8HGkjzSFR5 jT72UHWgPgY5kGIapEHb1GDudanzyFWUmCxPaBiMP4SfXGY8T9jxk3Lu1bYJEUarTzU0 hnGw== X-Gm-Message-State: ACgBeo3RyebAB1LRFtycMs/7jHDz2mb3qMrfxfbXiDzCMtrcAhzHXwe3 SEhDlwYSn4OSna7o9zcJCzwpqJ1C/TelmFZQ X-Received: by 2002:a05:6512:3f0a:b0:492:da2e:ce69 with SMTP id y10-20020a0565123f0a00b00492da2ece69mr14756005lfa.269.1662370228326; Mon, 05 Sep 2022 02:30:28 -0700 (PDT) Received: from localhost.localdomain (dsl-tkubng21-58c01c-243.dhcp.inet.fi. [88.192.28.243]) by smtp.gmail.com with ESMTPSA id c7-20020a056512074700b004946b3d4517sm1134944lfs.55.2022.09.05.02.30.27 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Sep 2022 02:30:27 -0700 (PDT) From: =?utf-8?q?Martin_Storsj=C3=B6?= To: ffmpeg-devel@ffmpeg.org Date: Mon, 5 Sep 2022 12:30:27 +0300 Message-Id: <20220905093027.4090687-1-martin@martin.st> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] cpu: Limit the number of auto threads in 32 bit builds X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: xZRr8yCnxiLO Limit the returned value from av_cpu_count to sensible amounts in 32 bit builds. This chosen limit, 64, is somewhat arbitrary - a 32 bit process is capable of creating much more than 64 threads. But in many cases, multiple parts of the encoding pipeline (decoder, filters, encoders) all create a pool of threads, auto sized according to the number of cores. In one failing test, the process had managed to create 506 threads before a pthread_create call failed. In the current set of fate tests, the filter-lavd-scalenorm test seems to be the limiting factor; in a 32 bit build (arm linux, running on an aarch64 kernel), it starts failing with an auto thread count somewhere around 85. Therefore, pick the maximum with some margin below this. This fixes running fate without any manually set number of threads in 32 bit builds on machines with huge numbers of cores. --- libavutil/cpu.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/libavutil/cpu.c b/libavutil/cpu.c index 0035e927a5..094bd71d3d 100644 --- a/libavutil/cpu.c +++ b/libavutil/cpu.c @@ -233,6 +233,12 @@ int av_cpu_count(void) nb_cpus = sysinfo.dwNumberOfProcessors; #endif +#if SIZE_MAX <= UINT32_MAX + // Avoid running out of memory/address space in 32 bit builds, by + // limiting the number of auto threads. + nb_cpus = FFMIN(nb_cpus, 64); +#endif + if (!atomic_exchange_explicit(&printed, 1, memory_order_relaxed)) av_log(NULL, AV_LOG_DEBUG, "detected %d logical cores\n", nb_cpus);