tech-pkg archive
[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index][
Old Index]
Re: grafana hits pkgsrc limits again
* On 2024年07月09日 at 09:04 BST, Thomas Klausner wrote:
For the last grafana update, I had to update the distinfo code to
handle a huge list of go module files that grafana needs.
I just tried updating grafana to 11.1.0 (see wip/grafana), and now I
run into a similar limit in the checksum step, from
mk/checksum/checksum.mk:
${_CHECKSUM_CMD} ${DISTINFO_FILE:tA} ${_CKSUMFILES}
The ${_CKSUMFILES} variable gets too big for the sh(1) command line
length limit.
Try these two patches:
https://github.com/TritonDataCenter/pkgsrc/compare/release/trunk...TritonDataCenter:pkgsrc:feature/parallel-digest/trunk
This implements threaded mode in digest, as well as adding an -F flag
which changes the behaviour of data coming in to be a list of filenames
to checksum.
An example of the threaded mode speedup on my macOS laptop (passing the
data through shasum to verify it's identical):
$ cd wip/grafana
$ time bmake checksum | shasum; echo $?
4345957499803ad801a6e4e9aa1bffaeb968c9ef -
real 0m6.696s
user 0m6.350s
sys 0m0.468s
0
$ time bmake TOOLS_PATH.digest='/path/to/patched/digest -j 16' checksum | shasum; echo $?
4345957499803ad801a6e4e9aa1bffaeb968c9ef -
real 0m2.090s
user 0m7.526s
sys 0m0.924s
0
I don't have any patches for pkgsrc to use -F yet, but it'll look
something like this:
$ time bmake -v DISTFILES | tr ' ' '\n' | (cd ../../distfiles; /path/to/digest -j 16 -F md5)
MD5 (grafana-11.1.0.tar.gz) = 73645b7d5fd376b43957994a4e4722da
MD5 (grafana-11.1.0.linux-amd64.tar.gz) = 09f56f35fcfb42a4d04407d654561fb3
[...]
MD5 (buf.build_gen_go_parca-dev_parca_protocolbuffers_go_@v_v1.33.0-20240414232344-9ca06271cb73.1.zip) = d37f47809ffd0e14424c4ef6f065267a
real 0m1.350s
user 0m2.982s
sys 0m5.855s
That should avoid having to use temporary files (which could be tricky
as we might not have WRKDIR created), as well as providing a decent
performance improvement on most machines.
--
Jonathan Perkin - mnx.io - pkgsrc.smartos.org
Open Source Complete Cloud www.tritondatacenter.com
Home |
Main Index |
Thread Index |
Old Index