Consider using a single-threaded async runtime #457

LordMZTE commented 2026年05月09日 20:02:51 +02:00

Motivation

Currently, we decorate our main function with #[tokio::main]. This causes tokio to spawn a number of workers equal to the number of CPU cores on the host (this can be easily observed using strace, for example). Needless to say, there is little benefit to this given that almost everything that forgejo-cli does is purely sequential in nature, so we're paying for thread spawning overhead without getting much of any performance gain from parallelism in return. In fact, I'd even be in favor of using blocking IO everywhere instead, as asynchronous IO offers little advantage in scenarios where all IO actions are performed sequentially anyways, but this is likely unrealistic due to Rust's fundamental distinction between async and synchronous IO functions and forgejo-api also being based on tokio.

Notable functions

• Here, we use spawn_blocking. While this is idiomatic, it would mean forcing tokio to spawn a worker if there isn't one already. If we don't want to do anything else on the tokio runtime anyways, there's also no reason not to block it.
• Here, we spawn an HTTP server for an authentication callback. This is the only place I could find where we actually benefit from using an async runtime, as this allows us to handle multiple connections with one OS thread.

Tiny Benchmark

For what it's worth, we can also see a minor performance improvement for fj whoami simply by changing the attribute on main to #[tokio::main(flavor = "current_thread")]:

$ poop -- "target/release/fj-old whoami" "target/release/fj whoami"
Benchmark 1 (263 runs): target/release/fj-old whoami
 measurement mean ± σ min ... max outliers delta
 wall_time 19.0ms ± 558us 18.2ms ... 22.6ms 12 ( 5%) 0%
 peak_rss 18.4MB ± 45.0KB 18.3MB ... 18.5MB 30 (11%) 0%
 cpu_cycles 77.0M ± 2.19M 74.7M ... 92.5M 18 ( 7%) 0%
 instructions 153M ± 11.3K 153M ... 153M 53 (20%) 0%
 cache_references 1.97M ± 53.2K 1.81M ... 2.20M 3 ( 1%) 0%
 cache_misses 150K ± 12.4K 133K ... 203K 10 ( 4%) 0%
 branch_misses 315K ± 37.6K 280K ... 582K 14 ( 5%) 0%
Benchmark 2 (271 runs): target/release/fj whoami
 measurement mean ± σ min ... max outliers delta
 wall_time 18.5ms ± 394us 17.9ms ... 21.3ms 13 ( 5%) ⚡- 2.7% ± 0.4%
 peak_rss 18.4MB ± 11.1KB 18.3MB ... 18.4MB 2 ( 1%) - 0.0% ± 0.0%
 cpu_cycles 75.4M ± 1.81M 73.6M ... 89.2M 19 ( 7%) ⚡- 2.1% ± 0.4%
 instructions 153M ± 58.3 153M ... 153M 0 ( 0%) - 0.1% ± 0.0%
 cache_references 1.86M ± 65.6K 1.64M ... 2.10M 5 ( 2%) ⚡- 5.8% ± 0.5%
 cache_misses 143K ± 15.2K 121K ... 223K 15 ( 6%) ⚡- 4.6% ± 1.6%
 branch_misses 294K ± 28.2K 273K ... 450K 23 ( 8%) ⚡- 6.8% ± 1.8%

It would be interesting how this performs with blocking IO, but I cannot think of a good way to get a benchmark for that, sadly.

The binary size is currently identical for both builds, but we may also be able to reduce the feature flags we build tokio with, potentially improving binary size.

Code of Conduct

• I agree to act in accordance with the CoC & AI Agreement.
• This issue was not generated by an LLM, even in part.

## Motivation Currently, we decorate our main function with `#[tokio::main]`. This causes tokio to spawn a number of workers equal to the number of CPU cores on the host (this can be easily observed using `strace`, for example). Needless to say, there is little benefit to this given that almost everything that forgejo-cli does is purely sequential in nature, so we're paying for thread spawning overhead without getting much of any performance gain from parallelism in return. In fact, I'd even be in favor of using blocking IO everywhere instead, as asynchronous IO offers little advantage in scenarios where all IO actions are performed sequentially anyways, but this is likely unrealistic due to Rust's fundamental distinction between async and synchronous IO functions and forgejo-api also being based on tokio. ## Notable functions - [Here](https://codeberg.org/forgejo-contrib/forgejo-cli/src/commit/be366606a41c4add1b229fba100f16aef51a2817/src/main.rs#L107), we use `spawn_blocking`. While this is idiomatic, it would mean forcing tokio to spawn a worker if there isn't one already. If we don't want to do anything else on the tokio runtime anyways, there's also no reason not to block it. - [Here](https://codeberg.org/forgejo-contrib/forgejo-cli/src/commit/be366606a41c4add1b229fba100f16aef51a2817/src/auth.rs#L300), we spawn an HTTP server for an authentication callback. This is the only place I could find where we actually benefit from using an async runtime, as this allows us to handle multiple connections with one OS thread. ## Tiny Benchmark For what it's worth, we can also see a minor performance improvement for `fj whoami` simply by changing the attribute on `main` to `#[tokio::main(flavor = "current_thread")]`: ``` $ poop -- "target/release/fj-old whoami" "target/release/fj whoami" Benchmark 1 (263 runs): target/release/fj-old whoami measurement mean ± σ min ... max outliers delta wall_time 19.0ms ± 558us 18.2ms ... 22.6ms 12 ( 5%) 0% peak_rss 18.4MB ± 45.0KB 18.3MB ... 18.5MB 30 (11%) 0% cpu_cycles 77.0M ± 2.19M 74.7M ... 92.5M 18 ( 7%) 0% instructions 153M ± 11.3K 153M ... 153M 53 (20%) 0% cache_references 1.97M ± 53.2K 1.81M ... 2.20M 3 ( 1%) 0% cache_misses 150K ± 12.4K 133K ... 203K 10 ( 4%) 0% branch_misses 315K ± 37.6K 280K ... 582K 14 ( 5%) 0% Benchmark 2 (271 runs): target/release/fj whoami measurement mean ± σ min ... max outliers delta wall_time 18.5ms ± 394us 17.9ms ... 21.3ms 13 ( 5%) ⚡- 2.7% ± 0.4% peak_rss 18.4MB ± 11.1KB 18.3MB ... 18.4MB 2 ( 1%) - 0.0% ± 0.0% cpu_cycles 75.4M ± 1.81M 73.6M ... 89.2M 19 ( 7%) ⚡- 2.1% ± 0.4% instructions 153M ± 58.3 153M ... 153M 0 ( 0%) - 0.1% ± 0.0% cache_references 1.86M ± 65.6K 1.64M ... 2.10M 5 ( 2%) ⚡- 5.8% ± 0.5% cache_misses 143K ± 15.2K 121K ... 223K 15 ( 6%) ⚡- 4.6% ± 1.6% branch_misses 294K ± 28.2K 273K ... 450K 23 ( 8%) ⚡- 6.8% ± 1.8% ``` It would be interesting how this performs with blocking IO, but I cannot think of a good way to get a benchmark for that, sadly. The binary size is currently identical for both builds, but we may also be able to reduce the feature flags we build tokio with, potentially improving binary size. ### Code of Conduct - [x] I agree to act in accordance with the CoC & AI Agreement. - [x] This issue was not generated by an LLM, even in part.

Cyborus commented 2026年05月13日 22:59:34 +02:00

I definitely agree with using a single-threaded executor, and very open to even using blocking IO. I can't think of a single place in fj that actually uses concurrency, and only one that could in theory benefit from it (here)

No reason we can't do both! Blocking IO is a much larger endeavor, so we could set flavor = "current_thread" right away while deciding on it.

[...] forgejo-api also being based on tokio.

forgejo-api does have a sync feature flag that gives a blocking version of the interface.

• [...] This is the only place I could find where we actually benefit from using an async runtime, as this allows us to handle multiple connections with one OS thread.

Ideally this server only ever needs to handle one connection (from the browser after being authenticated). If any other app sends a request, it'll cause it to fail anyway. Not sure if hyper supports blocking though? If not, we could create a smol executor contained to just this function.

[...] we can also see a minor performance improvement for fj whoami simply by changing the attribute on main to #[tokio::main(flavor = "current_thread")]

I'd be extra interested in seeing what it does for something that makes a network call, since I imagine that's where most of the time is spent, and currently fj whoami only access the file system.

The binary size is currently identical for both builds, but we may also be able to reduce the feature flags we build tokio with, potentially improving binary size.

IIRC Rust trims unused things, so this isn't a concern.

I definitely agree with using a single-threaded executor, and very open to even using blocking IO. I can't think of a single place in `fj` that actually uses concurrency, and only one that could in theory benefit from it ([here](https://codeberg.org/forgejo-contrib/forgejo-cli/src/commit/ade375d23c07a5c9dc1b01568ef0c9703f18a3aa/src/prs.rs#L740-L753)) No reason we can't do both! Blocking IO is a much larger endeavor, so we could set `flavor = "current_thread"` right away while deciding on it. > [...] forgejo-api also being based on tokio. forgejo-api does have a `sync` feature flag that gives a blocking version of the interface. > * [...] This is the only place I could find where we actually benefit from using an async runtime, as this allows us to handle multiple connections with one OS thread. Ideally this server only ever needs to handle one connection (from the browser after being authenticated). If any other app sends a request, it'll cause it to fail anyway. Not sure if hyper supports blocking though? If not, we could create a `smol` executor contained to just this function. > [...] we can also see a minor performance improvement for `fj whoami` simply by changing the attribute on `main` to `#[tokio::main(flavor = "current_thread")]` I'd be extra interested in seeing what it does for something that makes a network call, since I imagine that's where most of the time is spent, and currently `fj whoami` only access the file system. > The binary size is currently identical for both builds, but we may also be able to reduce the feature flags we build tokio with, potentially improving binary size. IIRC Rust trims unused things, so this isn't a concern.

LordMZTE commented 2026年05月13日 23:41:04 +02:00

If [hyper does not support blocking], we could create a smol executor contained to just this function.

Good point! I doubt we'll find a blocking HTTP server, so that might be our only option here.

I'd be extra interested in seeing what it does for something that makes a network call, since I imagine that's where most of the time is spent

I'm not sure how we'd measure this. Since, when doing IO, the time we spend on the CPU is essentially always going to be negligible, hence why I chose to test using a command that doesn't do any networking - we'd just get noise measurements otherwise. This is another important point to bring up. fj is a very IO-bound application, which makes CPU-bound overhead like this rather irrelevant in the large picture. I, however, just don't really like things being unecessarily complicated like this whole runtime situation is.

> If [`hyper` does not support blocking], we could create a smol executor contained to just this function. Good point! I doubt we'll find a blocking HTTP server, so that might be our only option here. > I'd be extra interested in seeing what it does for something that makes a network call, since I imagine that's where most of the time is spent I'm not sure how we'd measure this. Since, when doing IO, the time we spend on the CPU is essentially always going to be negligible, hence why I chose to test using a command that doesn't do any networking - we'd just get noise measurements otherwise. This is another important point to bring up. `fj` is a very IO-bound application, which makes CPU-bound overhead like this rather irrelevant in the large picture. I, however, just don't really like things being unecessarily complicated like this whole runtime situation is.