tech-pkg archive
[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index][
Old Index]
Wider testing for wrapper performance patch
Hey all,
This is earlier than I'd normally ask for review of something, but it's
a huge win and I'm interested in testing on a wide variety of systems to
see what the performance wins are, and if there are any regressions.
One of the slowest parts of the pkgsrc infrastructure is the wrapper
phase, where symlinks for all includes, libraries, and ancillary support
files are created under "${WRKDIR}/.buildlink/". Each file results in a
fork of dirname(1) and ln(1).
When you have packages such as devel/boost-headers which is pulled in by
many packages and alone has over 15,000 files to symlink, the terrible
performance becomes clear. For example:
$ cd x11/kde-workspace4
$ bmake patch; ptime bmake wrapper
real 3:20.696394375
user 50.553556463
sys 2:23.883823896
I'm currently testing an alternative which uses a small program to batch
up the symlink creation, and is significantly faster:
real 19.873100598
user 8.141441461
sys 11.740602820
You can grab it and the patch to the infrastructure from here:
https://github.com/TritonDataCenter/pkgsrc/commit/d119b14357b26ac282c3d5088200cc9986295b6a
It currently lives in the cwrappers package for ease of testing, but I'm
not sure this is a good long term home, and maybe we want to create a
new "pkg-mk-tools" package or something that can house a few other
useful utilities to speed up other areas of the infrastructure.
Cheers,
--
Jonathan Perkin - Joyent, Inc. - www.joyent.com
Home |
Main Index |
Thread Index |
Old Index