From 17b42195afb8e3cdd1fd03c5b14634c64c78a1bc Mon Sep 17 00:00:00 2001
From: "Avi Halachmi (:avih)" <avihpit@yahoo.com>
Date: Mon, 30 Jul 2018 22:39:15 +0300
Subject: [PATCH] configure: flatten_extralibs_wrapper(): speedup x50 - x200
Currently configure spends 50-70% of its runtime inside a single
function: flatten_extralibs[_wrapper] - which does string processing.
During its run, nearly 20K command substitutions (subshells) are used,
including its callees unique() and resolve(), which is the reason
for its lengthy run.
This commit avoids all subshells during its execution, speeding it up
by about two orders of magnitude, and reducing the overall configure
runtime by 50-70% .
resolve() is rewritten to avoid subshells, and in unique() and
flatten_extralibs() we "inline" the filter[_out] functionality.
Note that logically, "unique" functionality has more than one possible
output (depending on which of the recurring items is kept). As it
turns out, other parts expect the last recurring item to be kept
(which was the original behavior of uniqie()). This patch preservs
its output order.
---
configure | 46 +++++++++++++++++++++++++++++++++-------------
1 file changed, 33 insertions(+), 13 deletions(-)
@@ -844,22 +844,37 @@ prepend(){
eval "$var=\"$* \$$var\""
}
+reverse () {
+ eval '
+ reverse_out=
+ for v in $'$1'; do
+ reverse_out="$v $reverse_out"
+ done
+ '$1'=$reverse_out
+ '
+}
+
+# keeps the last occurence of each non-unique item
unique(){
- var=$1
- uniq_list=""
- for tok in $(eval echo \$$var); do
- uniq_list="$(filter_out $tok $uniq_list) $tok"
+ unique_out=
+ eval unique_in=\$$1
+ reverse unique_in
+ for v in $unique_in; do
+ # " $unique_out" +space such that every item is surrounded with spaces
+ case " $unique_out" in *" $v "*) continue; esac # already in list
+ unique_out="$unique_out$v "
done
- eval "$var=\"${uniq_list}\""
+ reverse unique_out
+ eval $1=\$unique_out
}
resolve(){
- var=$1
- tmpvar=
- for entry in $(eval echo \$$var); do
- tmpvar="$tmpvar $(eval echo \$${entry})"
+ resolve_out=
+ eval resolve_in=\$$1
+ for v in $resolve_in; do
+ eval 'resolve_out="$resolve_out$'$v' "'
done
- eval "$var=\"${tmpvar}\""
+ eval $1=\$resolve_out
}
add_cppflags(){
@@ -6734,14 +6749,19 @@ if test $target_os = "haiku"; then
fi
flatten_extralibs(){
- unset nested_entries
+ nested_entries=
list_name=$1
eval list=\$${1}
for entry in $list; do
entry_copy=$entry
resolve entry_copy
- append nested_entries $(filter '*_extralibs' $entry_copy)
- flat_entries=$(filter_out '*_extralibs' $entry_copy)
+ flat_entries=
+ for e in $entry_copy; do
+ case $e in
+ *_extralibs) nested_entries="$nested_entries$e ";;
+ *) flat_entries="$flat_entries$e ";;
+ esac
+ done
eval $entry="\$flat_entries"
done
append $list_name "$nested_entries"
--
2.17.1
It turns out the the output order of "unique()" was important, andfailing to keep the original order (where it keeps the last instanceof recurring items) caused some linkage failures. This Patch 1/3 v2 squashes opt1-reorder-unique.patch onto the originalpatch 1/3, and updates the commit message accordingly. It fixes the linkage failures mentioned at:http://ffmpeg.org/pipermail/ffmpeg-devel/2018-August/233674.html On Saturday, August 25, 2018 9:09 PM, avih <avihpit-at-yahoo.com@ffmpeg.org> wrote: For overview see: http://ffmpeg.org/pipermail/ffmpeg-devel/2018-August/233665.html Attached is patch 1/3 which was at "main.patch" of that message. About 50-70% of configure runtime was being spent inside one function: flatten_extralibs() and callees resolve() and unique(). It manipulates strings and invoked nearly 20K (20000) subshells. It was rewritten to avoid subshells, and ended up x50-x250 faster. unique() now outputs a different order: it was keeping the last instance of recurring items, now it keeps the first. It affects libs order at ffbuild/config.{mak,sh} - but I don't think it matters. If it does, "opt1-reorder-unique.patch" restores the original order. Let me know if/why it matters and I'll squash it and update the commit message accordingly if required. On Saturday, August 25, 2018 7:55 PM, Timo Rothenpieler <timo@rothenpieler.org> wrote: Please use git send-email to send your patches, or at least send each patch, created by git format-patch, as individual attachment. Your files seem to contain multiple patches one after another, which makes them very hard to follow. But nice work! Let's hope this does not cause any regressions.