From patchwork Tue Sep 20 17:50:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: rcombs X-Patchwork-Id: 38111 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1c:b0:96:9ee8:5cfd with SMTP id c28csp2106276pzh; Tue, 20 Sep 2022 10:51:22 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7cVjRNTu4x4E6tw+sRwzbNbVd8TMRIUKJGa3DFQSbAdSXiemN3MdqdQEaa+hrZi7c6xWCr X-Received: by 2002:a05:6402:f07:b0:450:bf95:59e9 with SMTP id i7-20020a0564020f0700b00450bf9559e9mr21361759eda.46.1663696282560; Tue, 20 Sep 2022 10:51:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1663696282; cv=none; d=google.com; s=arc-20160816; b=cT/8XzkJIFchWVqcY3NNfR25s9WydtZ7MLVZ0OG6DWUa3bsKfm99WjTrbN9gVn1kzq DMtc8V3nip2jx2uAZ0GLBkx0AawjOLdFDgL+Q06U4WW1TbpFS0NBNHwTq1xY11t178AT J0o3DAthILGP/Kbs1y9t5pic05JFJN5hRn/yDQhK36OYcVeZroFDxZWiD4d/G4lfl7AD doGoA9mWz7PV+WnMOXM9NwhKRKppMnKtuSBuslwaWRZyjsxspeR9o4goksTmt3BghVIU 3AlgMFsTA3vZjPYk6qOea5g1LBgitKWT2d78mEjOZQFA4e22EmQ2qJrraiLBbZnIH6Jl Rvqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=CRSVPbuCTh2US4eFGy2qWD8ROkl1gK/l3bCt+KiLOds=; b=ArXoGhfRRbnoqfyFUOV2WXzOv5Jwzua416+OL81LBsIkr3qFVifPA0wE3yhZCgY4/d JBA5YMBNz2XAe8dROQwHMxkCPiflBa5Ehy4DtmU2jYEg7n+9RmpsVCVagxY8DbvMvf67 IUtcgAwGcY2SUm8/+UDa4h8khe7MH5qi2kwgLvRsB2FYF+jH9S8fGA9jKbr2UMl+VURq 8cnBrV4/XY4bfh/ycqF9EbNQmSZS7jV/bcum0p1N6jrX/97jmomN9yQYUyNbHcZ9dkQ6 Bj70sfh7f2mOdp3JX48icsDeuG9tB37KfCykLXhGSAsr0HsblFB3lIbMnuKTLfbPbWRN g1TA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rcombs.me header.s=google header.b=LRf2vU6o; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=rcombs.me Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2-20020a508e42000000b0043a6e796231si292830edx.544.2022.09.20.10.51.20; Tue, 20 Sep 2022 10:51:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rcombs.me header.s=google header.b=LRf2vU6o; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=rcombs.me Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8879568BB52; Tue, 20 Sep 2022 20:50:48 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f171.google.com (mail-qk1-f171.google.com [209.85.222.171]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1514068BB26 for ; Tue, 20 Sep 2022 20:50:40 +0300 (EEST) Received: by mail-qk1-f171.google.com with SMTP id s9so2192756qkg.4 for ; Tue, 20 Sep 2022 10:50:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rcombs.me; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date; bh=ja+fWqQ1ECRGkZ4GnddTTCs/4/eijkN4pfhsx+57U90=; b=LRf2vU6oijCXqyPKdsgpsWt51TYQH+V+BQNkkpXqeW5vy8VqpO7d3Sb4phTe6+pimn 2j/uhdSXJ/Jq6jjuqKSkfS4OEGmVENNRkPkSu3Z/SJyKOy+hJvTQiH7mjHpNIGG1wYLE UjHUKk2GYWvIJ8ldWUbsvQF+JYuu7rjw5n2kIe9uPxMI5AJdUcEWvAUk2MeNKmhzIciz s8qGP517kyvOOmSpm1tQm/LY3PBghFp4cQE8GMt37j2WWvL7ZO/JD2iJclJ2Hfjy2KXI mqJYfmxFlfRiSO3N0x8iH+J1h2J+JvCX2ID7tAjyjwOAmtOVjieFsM/jSzGI4x3PzhHi RB6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date; bh=ja+fWqQ1ECRGkZ4GnddTTCs/4/eijkN4pfhsx+57U90=; b=NQx1usl4fCfJW82WKKeFNu3DTIV4HiY8e1kEUGZefOFT9phrAKzbAFaVN9DRoYnIxJ gvanpTPoRQEt1zLSW7sT+xqTRCZqKU05tIV3a7qbKXwFRssEaZAbISATrKkYKlU6MZbZ vAU5NBnSWiAdu7KgtDRhlRCuM7EgphhcQLJhVPqgluVSk/B4lJdmJAMzo+azKNSMTXCB /q5M73UKd1+bubRblZvgw8/FbE1+g2I+QqljQwo/VnW1v6ikrx9EuthZ6twOz9/CEVSF 4Ddd6BGQmFvSqWpYJAwg5U626GN4xYuM899Lb7c0GK03JPPkPC7Hi1yMQk2b5hivDOaG DXxQ== X-Gm-Message-State: ACrzQf36QtmyU+Sm0ePN1KqTd9USejpCCgahFw0a91zH8zgzqK1lt/s3 V1hfA2fdGDdOoEV/FpdUiOnBZ/l3l/Cb X-Received: by 2002:a05:620a:4589:b0:6ce:4b66:b05e with SMTP id bp9-20020a05620a458900b006ce4b66b05emr17674591qkb.235.1663696238295; Tue, 20 Sep 2022 10:50:38 -0700 (PDT) Received: from rcombs-mbp.localdomain ([2601:243:2000:5ac:50a3:2f65:4798:4cad]) by smtp.gmail.com with ESMTPSA id o1-20020a05622a008100b003431446588fsm189017qtw.5.2022.09.20.10.50.37 for (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 20 Sep 2022 10:50:37 -0700 (PDT) From: rcombs To: ffmpeg-devel@ffmpeg.org Date: Tue, 20 Sep 2022 12:50:18 -0500 Message-Id: <20220920175021.60790-5-rcombs@rcombs.me> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20220920175021.60790-1-rcombs@rcombs.me> References: <20220920175021.60790-1-rcombs@rcombs.me> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/7] ffmpeg: add filter_jobs and filter_complex_jobs args X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 1UkudbRwWrdZ These are similar to filter_threads and filter_complex_threads, but control job count. --- doc/ffmpeg.texi | 12 ++++++++++++ fftools/ffmpeg.c | 1 + fftools/ffmpeg.h | 2 ++ fftools/ffmpeg_filter.c | 7 +++++++ fftools/ffmpeg_opt.c | 13 +++++++++++++ 5 files changed, 35 insertions(+) diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi index 42440d93b4..6b8248df3b 100644 --- a/doc/ffmpeg.texi +++ b/doc/ffmpeg.texi @@ -789,6 +789,13 @@ Defines how many threads are used to process a filter pipeline. Each pipeline will produce a thread pool with this many threads available for parallel processing. The default is the number of available CPUs. +@item -filter_jobs @var{nb_jobs} (@emph{global}) +Defines how many jobs are used to process a filter pipeline. Each filter's work +may be split into up to this many tasks for parallel processing, which are then +dispatched to the thread pool of size @var{nb_threads}. +The default is the number of available CPUs, times a multiplier determined based on +the detected CPU topology. For homogenous systems, this multiplier is 1. + @item -pre[:@var{stream_specifier}] @var{preset_name} (@emph{output,per-stream}) Specify the preset for matching stream(s). @@ -1916,6 +1923,11 @@ Defines how many threads are used to process a filter_complex graph. Similar to filter_threads but used for @code{-filter_complex} graphs only. The default is the number of available CPUs. +@item -filter_complex_jobs @var{nb_jobs} (@emph{global}) +Defines how many jobs are used to process a filter_complex graph. +Similar to filter_jobs but used for @code{-filter_complex} graphs only. +The default is the number of available CPUs times a topology multiplier. + @item -lavfi @var{filtergraph} (@emph{global}) Define a complex filtergraph, i.e. one with arbitrary number of inputs and/or outputs. Equivalent to @option{-filter_complex}. diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 0e1477299d..9ff48141cd 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -621,6 +621,7 @@ static void ffmpeg_cleanup(int ret) } av_freep(&vstats_filename); av_freep(&filter_nbthreads); + av_freep(&filter_nbjobs); av_freep(&input_streams); av_freep(&input_files); diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index ede0b2bd96..89ee9caca7 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -679,7 +679,9 @@ extern AVIOContext *progress_avio; extern float max_error_rate; extern char *filter_nbthreads; +extern char *filter_nbjobs; extern int filter_complex_nbthreads; +extern int filter_complex_nbjobs; extern int vstats_version; extern int auto_conversion_filters; diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 7a5308425d..34e0fce7ab 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -998,6 +998,12 @@ int configure_filtergraph(FilterGraph *fg) av_opt_set(fg->graph, "threads", e->value, 0); } + if (filter_nbjobs) { + ret = av_opt_set(fg->graph, "jobs", filter_nbjobs, 0); + if (ret < 0) + goto fail; + } + args[0] = 0; e = NULL; while ((e = av_dict_get(ost->sws_dict, "", e, @@ -1020,6 +1026,7 @@ int configure_filtergraph(FilterGraph *fg) av_opt_set(fg->graph, "aresample_swr_opts", args, 0); } else { fg->graph->nb_threads = filter_complex_nbthreads; + fg->graph->nb_jobs = filter_complex_nbjobs; } if ((ret = avfilter_graph_parse2(fg->graph, graph_desc, &inputs, &outputs)) < 0) diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 5febe319e4..196064d1b9 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -175,7 +175,9 @@ int qp_hist = 0; int stdin_interaction = 1; float max_error_rate = 2.0/3; char *filter_nbthreads; +char *filter_nbjobs; int filter_complex_nbthreads = 0; +int filter_complex_nbjobs = 0; int vstats_version = 2; int auto_conversion_filters = 1; int64_t stats_period = 500000; @@ -352,6 +354,13 @@ static int opt_filter_threads(void *optctx, const char *opt, const char *arg) return 0; } +static int opt_filter_jobs(void *optctx, const char *opt, const char *arg) +{ + av_free(filter_nbjobs); + filter_nbjobs = av_strdup(arg); + return 0; +} + static int opt_abort_on(void *optctx, const char *opt, const char *arg) { static const AVOption opts[] = { @@ -3961,6 +3970,8 @@ const OptionDef options[] = { "set stream filtergraph", "filter_graph" }, { "filter_threads", HAS_ARG, { .func_arg = opt_filter_threads }, "number of non-complex filter threads" }, + { "filter_jobs", HAS_ARG, { .func_arg = opt_filter_jobs }, + "number of non-complex filter jobs" }, { "filter_script", HAS_ARG | OPT_STRING | OPT_SPEC | OPT_OUTPUT, { .off = OFFSET(filter_scripts) }, "read stream filtergraph description from a file", "filename" }, { "reinit_filter", HAS_ARG | OPT_INT | OPT_SPEC | OPT_INPUT, { .off = OFFSET(reinit_filters) }, @@ -3969,6 +3980,8 @@ const OptionDef options[] = { "create a complex filtergraph", "graph_description" }, { "filter_complex_threads", HAS_ARG | OPT_INT, { &filter_complex_nbthreads }, "number of threads for -filter_complex" }, + { "filter_complex_jobs", HAS_ARG | OPT_INT, { &filter_complex_nbjobs }, + "number of jobs for -filter_complex" }, { "lavfi", HAS_ARG | OPT_EXPERT, { .func_arg = opt_filter_complex }, "create a complex filtergraph", "graph_description" }, { "filter_complex_script", HAS_ARG | OPT_EXPERT, { .func_arg = opt_filter_complex_script },