continuous integration #37

BillyDM commented 2026年06月04日 21:24:13 +02:00

Having CI for testing PRs would be nice.

The main challenge is that Codeberg's CI only runs on Linux. It might be possible to set up some custom docker thing that can cross-compile to Windows/MacOS platforms. But even just being able to test the Linux target would be something.

A list of checks that would be nice:

• cargo test --workspace --all-features
• cargo doc --workspace --all-features
• cargo clippy --workspace --all-features
• cargo bundle --package <example_plugin>, and then running that plugin through pluginval.

We should also add a custom "CI" profile to the workspace root to reduce compile times.

Having CI for testing PRs would be nice. The main challenge is that Codeberg's CI only runs on Linux. It *might* be possible to set up some custom docker thing that can cross-compile to Windows/MacOS platforms. But even just being able to test the Linux target would be something. A list of checks that would be nice: * `cargo test --workspace --all-features` * `cargo doc --workspace --all-features` * `cargo clippy --workspace --all-features` * `cargo bundle --package <example_plugin>`, and then running that plugin through [pluginval](https://github.com/Tracktion/pluginval/blob/develop/docs/Testing%20plugins%20with%20pluginval.md). We should also add a custom "CI" profile to the workspace root to reduce compile times.

BillyDM commented 2026年06月05日 22:51:50 +02:00

Actually, using --workspace is probably not ideal since the gain_egui, gain_iced, and byo_gui_wgpu bring in a lot of dependencies which will make compile time take much longer. Those examples should be excluded in the CI.

Actually, using `--workspace` is probably not ideal since the `gain_egui`, `gain_iced`, and `byo_gui_wgpu` bring in a *lot* of dependencies which will make compile time take much longer. Those examples should be excluded in the CI.

jonnie commented 2026年06月08日 11:17:51 +02:00

I'd be happy to set this up. You're thinking the woodpecker CI instances right? Or forgejo actions? As far as I can see we don't get any kind of caching on codeberg's public CI, I guess we can do without that.

There is of course circleCI, which does macos and windows along with caching etc.

I'd be happy to set this up. You're thinking the woodpecker CI instances right? Or forgejo actions? As far as I can see we don't get any kind of caching on codeberg's public CI, I guess we can do without that. There is of course circleCI, which does macos and windows along with caching etc.

BillyDM commented 2026年06月08日 21:03:51 +02:00

Alright, looking into this, I think the woodpecker CI would probably be the best one. (circleCI doesn't offer MacOS for free, and plus I'm not a fan of the heavy bombardment of AI buzzwords.)

Codeberg of course appreciates if you optimize the CI to the absolute bare minimum. So after some playing around I've come up with this:

# --- necessary ---------------------------------------------------------------
# Run tests (the nice-plug and nice-plug-core crates are the only ones with tests)
cargo test --profile ci -p nice-plug -p nice-plug-core
# Test that plugin bundling works. This by proxy will also test that all of the core
# crates compile correctly (byo_gui_gl is a good candidate because is supports
# bundling both plugins and a standalone target, and it tests the common opengl support)
cargo xtask bundle -p byo_gui_gl --profile ci
# --- nice to have ------------------------------------------------------------
# Run clippy (exclude packes with heavy GUI dependencies to reduce strain on
# Codeberg's CI)
cargo clippy --profile ci --workspace --exclude byo_gui_softbuffer --exclude byo_gui_wgpu --exclude gain_egui --exclude gain_iced --exclude nice-plug-egui --exclude nice-plug-iced
# Make sure there are no errors or warnings in docs (exclude packes with heavy
# GUI dependencies to reduce strain on Codeberg's CI)
cargo doc --profile ci --workspace --exclude byo_gui_softbuffer --exclude byo_gui_wgpu --exclude gain_egui --exclude gain_iced --exclude nice-plug-egui --exclude nice-plug-iced

Alright, looking into this, I think the woodpecker CI would probably be the best one. (circleCI doesn't offer MacOS for free, and plus I'm not a fan of the heavy bombardment of AI buzzwords.) Codeberg of course appreciates if you optimize the CI to the absolute bare minimum. So after some playing around I've come up with this: ``` # --- necessary --------------------------------------------------------------- # Run tests (the nice-plug and nice-plug-core crates are the only ones with tests) cargo test --profile ci -p nice-plug -p nice-plug-core # Test that plugin bundling works. This by proxy will also test that all of the core # crates compile correctly (byo_gui_gl is a good candidate because is supports # bundling both plugins and a standalone target, and it tests the common opengl support) cargo xtask bundle -p byo_gui_gl --profile ci # --- nice to have ------------------------------------------------------------ # Run clippy (exclude packes with heavy GUI dependencies to reduce strain on # Codeberg's CI) cargo clippy --profile ci --workspace --exclude byo_gui_softbuffer --exclude byo_gui_wgpu --exclude gain_egui --exclude gain_iced --exclude nice-plug-egui --exclude nice-plug-iced # Make sure there are no errors or warnings in docs (exclude packes with heavy # GUI dependencies to reduce strain on Codeberg's CI) cargo doc --profile ci --workspace --exclude byo_gui_softbuffer --exclude byo_gui_wgpu --exclude gain_egui --exclude gain_iced --exclude nice-plug-egui --exclude nice-plug-iced ```

jonnie commented 2026年06月08日 21:39:42 +02:00

Cool, the advantage of woodpecker is I can test it locally.

Wrt the dependencies, if we can get some sort of caching then they shouldn't need to be rebuilt at all. I'll look into it.

Cool, the advantage of woodpecker is I can test it locally. Wrt the dependencies, if we can get some sort of caching then they shouldn't need to be rebuilt at all. I'll look into it.

jonnie commented 2026年06月11日 15:46:50 +02:00

TL;DR woodpecker doesn't do any caching, builds on it will be totally cold unless we provide our own external cache storage. This will be a big load on codeberg infra that could easily be avoided, even if we're using a reduced test set. Forgejo actions do provide cache storage.

=======

I've done some more research, and if we want to reduce the workload for codeberg then forgejo actions is seeming like a better bet. It provides automatic caching like github actions, so we can reuse the same cargo build directory for every run. On woodpecker, to do that you have to provide your own object-store and hook it up to a plugin.

Running the minimal test set from warm takes like 1 second, whereas downloading all the cargo deps and compiling them from cold takes over a minute of CPU time on my dev machine, so if we want to keep it light then the caching is really essential.

Beyond the cargo build cache, we have the build environment: system dependencies and the rust toolchain. We can reuse this by running the action in a custom container image built on the rust-nightly image, with the project deps installed. This is a bit more involved, we'd have to publish the image to the codeberg container registry and manage it to some extent.

The alternative would be to just use the base Actions image provided by codeberg, and install the deps and the rust toolchain on every run. This seems inefficient, but it's just pulling binaries and might actually be less work than pulling a 2gb+ prebuilt OS image. This is what I'd recommend.

The other thing to note is that while the main docs do refer to codeberg's Actions deployment as "limited" and recommend woodpecker, these seemingly more up to date notes describe it as "production ready," and without any approval requirement.

**TL;DR** woodpecker doesn't do any caching, builds on it will be totally cold unless we provide our own external cache storage. This will be a big load on codeberg infra that could easily be avoided, even if we're using a reduced test set. Forgejo actions do provide cache storage. ======= I've done some more research, and if we want to reduce the workload for codeberg then forgejo actions is seeming like a better bet. It provides automatic caching like github actions, so we can reuse the same cargo build directory for every run. On woodpecker, to do that you have to provide your own object-store and hook it up to a plugin. Running the minimal test set from warm takes like 1 second, whereas downloading all the cargo deps and compiling them from cold takes over a minute of CPU time on my dev machine, so if we want to keep it light then the caching is really essential. Beyond the cargo build cache, we have the build environment: system dependencies and the rust toolchain. We can reuse this by running the action in a custom container image built on the rust-nightly image, with the project deps installed. This is a bit more involved, we'd have to publish the image to the codeberg container registry and manage it to some extent. The alternative would be to just use the base Actions image provided by codeberg, and install the deps and the rust toolchain on every run. This seems inefficient, but it's just pulling binaries and might actually be less work than pulling a 2gb+ prebuilt OS image. This is what I'd recommend. The other thing to note is that while [the main docs](https://docs.codeberg.org/ci/actions/) do refer to codeberg's Actions deployment as "limited" and recommend woodpecker, these [seemingly more up to date notes](https://codeberg.org/actions/meta) describe it as "production ready," and without any approval requirement.

BillyDM commented 2026年06月12日 20:54:21 +02:00

Hmm, ok

Hmm, ok

jonnie commented 2026年06月25日 13:49:06 +02:00

I implemented it with woodpecker after all: #54

After actually trying it, it turns out forgejo actions default cache is local to the runner node, so it will miss most of the time anyway. Apologies for making you read all of that!

I implemented it with woodpecker after all: #54 After actually trying it, it turns out forgejo actions default cache is local to the runner node, so it will miss most of the time anyway. Apologies for making you read all of that!

BillyDM commented 2026年06月26日 19:53:36 +02:00

PR merged, thanks!

PR merged, thanks!