-
Notifications
You must be signed in to change notification settings - Fork 15
Conversation
It looks like hashFile()/hashFileBytes() does the same thing as FileSha256sum()
I also tried an implementation with XXH3
import "github.com/zeebo/xxh3" func FileXXH3Sum(path string) (string, error) { defer timeTaken("FileXXH3Sum")() mfile, err := os.Open(path) if err != nil { return "", err } defer mfile.Close() h := xxh3.New() _, err = io.Copy(h, mfile) if err != nil { return "", err } return fmt.Sprintf("%x", h.Sum128().Bytes()), nil } func timeTaken(name string) func() { start := time.Now() return func() { fmt.Printf("%s execution time: %v\n", name, time.Since(start)) } }
A similar thing could be done with blake3 which would allow 256bits again.
Dirty benchmarks (m64py)
FileXXH3Sum execution time: 5.10317828s (uncached)
FileXXH3Sum execution time: 1.441753051s (cached)
FileSha256sum execution time: 9.536810397s (uncached)
FileSha256sum execution time: 5.823365451s (cached)
edit: it looks like your hashFileBytes() implementation using io.Copy is slightly faster than the MapFile approach.
Friendly reminder that this solbuild version appears to rundep on xxhash, which contains the xxh128sum binary used to hash the layers.
Should probably be added in the cover letter or in the readme + added as a proper solbuild rundep once this hits the repo...
GZGavinZhao
commented
Jun 14, 2024
Feature-wise complete, I've been using this since April as well. Need some code cleanup and add this feature behind a flag, then this should be ready for review.
joebonrichie
commented
Jun 16, 2024
Chrooting after a interrupted build seems to fail, even after deleting the layers
^C 🗲 CTRL+C interrupted, cleaning up
✗ Exiting due to interruption
task: Failed to run task "build": exit status 1
[ble: exit 201]
⮞ ninya git:main ~/solus/.../l/libvpx 1 sudo rm -rf /var/cache/solbuild/layers/
⮞ ninya git:main ~/solus/.../l/libvpx sudo solbuild chroot
✓ Using default profile name=unstable-x86_64
mount: /var/cache/solbuild/unstable-x86_64/libvpx/union: fsconfig system call failed: Stale file handle.
dmesg(1) may have more information after failed mount system call.
✗ Failed to mount overlayfs point=/var/cache/solbuild/unstable-x86_64/libvpx/union err="exit status 32"
[ble: exit 1]
GZGavinZhao
commented
Jun 26, 2024
@joebonrichie Arghh I completely overlooked the chroot command because I don't use it... I fixed the layer cleanup issue which caused the mount failure, but currently the layer is not being brought into the chroot so I'm going to fix that next.
Signed-off-by: Gavin Zhao <git@gzgz.dev>
Signed-off-by: Gavin Zhao <git@gzgz.dev>
Signed-off-by: Gavin Zhao <git@gzgz.dev>
Signed-off-by: Gavin Zhao <git@gzgz.dev>
Signed-off-by: Gavin Zhao <git@gzgz.dev>
Signed-off-by: Gavin Zhao <git@gzgz.dev>
Signed-off-by: Gavin Zhao <git@gzgz.dev>
Signed-off-by: Gavin Zhao <git@gzgz.dev>
Signed-off-by: Gavin Zhao <git@gzgz.dev>
Signed-off-by: Gavin Zhao <git@gzgz.dev>
Signed-off-by: Gavin Zhao <git@gzgz.dev>
Uh oh!
There was an error while loading. Please reload this page.
WIP.
Current issues:
solbuild buildandsudo rm -rf /var/cache/solbuild/layers/<hash>. Or justsudo rm -rf /var/cache/solbuild/layersto delete all layers.This PR shouldn't touch anything outside of
/var/cache/solbuild/layers, so if anything catastrophic happens, just runsudo solbuild dcand start over.Test instructions:
xxhashpackage installed because you need thexxxh128sumbinary in your PATH.gh pr checkout 95or checkout this branch however you prefer to.makesolbuildbinary in./bin/solbuild. Use thissolbuildas how you would normally build packages. You should notice that after you build a package once, when you build it the second time, it should jump straight to the setup phase, skipping the step of installing dependencies.