tech-pkg archive
[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index][
Old Index]
Improving 'bmake wrapper' speed
I recently got fed up with how long the 'wrapper' phase was taking,
especially on packages with a large number of dependencies, so decided
to take a look.
Breaking out DTrace on an otherwise idle system running just the
'wrapper' phase ('patch' had already been done) showed that we are
running a huge number of exec'd processes:
# dtrace -n 'syscall::exece:return { @num[execname] = count(); }'
[ full output snipped for brevity ]
grep 94
sort 164
nbsed 241
mkdir 399
bash 912
cat 3893
ln 7631
rm 7766
dirname 7769
Tracking these down, the 'cat's are primarily from
mk/wrapper/gen-transform.sh, and the ln/rm/dirname are from each
symlink done by mk/buildlink3/bsd.buildlink3.mk.
I have a proposed patch which significantly reduces the number of
execs we need to perform:
http://us-east.manta.joyent.com/jperkin/public/patches/wrapper-perf.diff
The patch reduces the runtime for 'bmake wrapper' in net/kppp on a
SmartOS zone from this:
real 2:26.094442113
user 32.463077360
sys 1:48.647178135
to this:
real 49.648642097
user 14.952946135
sys 33.989975053
with DTrace confirming the change:
# dtrace -n 'syscall::exece:return { @num[execname] = count(); }'
[ full output snipped for brevity ]
grep 94
cat 106
sort 164
nbsed 241
mkdir 399
bash 912
ln 7631
On a Linux system it reduces the runtime from this:
real 1m3.939s
user 0m17.437s
sys 0m43.280s
to this:
real 0m25.436s
user 0m8.816s
sys 0m15.406s
So between a 2.5x and 3x speedup.
I have verified that the WRKDIR is identical after this change, but
will run it through a full bulk build anyway.
Comments? There are a lot of other 'useless use of cat' cleanups that
can be done, but they do not have such a significant performance
impact.
Regards,
--
Jonathan Perkin - Joyent, Inc. - www.joyent.com
Home |
Main Index |
Thread Index |
Old Index