Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

perf(fuzz): parallel stateless fuzzing #11842

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
yash-atreya wants to merge 31 commits into yash/shared-corpus
base: yash/shared-corpus
Choose a base branch
Loading
from yash/parallel-fuzz

Conversation

Copy link
Contributor

@yash-atreya yash-atreya commented Sep 26, 2025
edited
Loading

Motivation

Stacked on #11769

Solution

  • Introduce SharedFuzzState which consists of global values of runs, timer, fail_fast and rejects to determine whether the fuzz test should_continue using the SharedFuzzState::should_continue
  • Fuzz workers atomically update the values in SharedFuzzState
  • For timed campaigns, timeout specified in foundry.toml is applied to each worker as-is to fit as many runs as possible i.e, timeout value is NOT divided by number of workers.
  • fuzz.runs config value is divided by the number of workers i.e, worker_runs = fuzz.runs / num_worker — reducing the time required for the campaign
  • fuzz.seed set in config is altered per worker so that the workers are covering different inputs. For worker0 / master worker, the provided seed is used as is. For workers with WorkerID > 0, the seed is set to keccak256(fuzz.seed || WorkerID)
  • Currently, the corpus SYNC_INTERVAL is naively set to 1000, we can adjust this after some more empirical feedback

TODO

  • Add max_rejects in SharedFuzzState to track total rejects across workers and fail accordingly 40263fd
  • Set failures in run_worker via SharedState::try_claim_failure - This will intercept other workers as well and stop fuzzing entirely e4c6060
  • Introduce sync_interval for the WorkerCorpus and call WorkerCorpus::sync ed234d8
  • Introduce GlobalCorpusMetrics and sync worker corpus metrics d973b79
  • Determine number of workers - all available cores / --jobs e991826
  • Integrate into fn fuzz and run workers using rayon::IntoParallelIterator
  • Aggregate worker results 47bd12d
  • Address breaking tests

Benchmarks

  1. Uniswap/v4-core - 1000 fuzz runs
hyperfine "forge test --match-test 'test[^(]*\([^)]+\)'" "forge-pf test --match-test 'test[^(]*\([^)]+\)'" --warmup 1 --setup "forge b"
Benchmark 1: forge test --match-test 'test[^(]*\([^)]+\)'
 Time (mean ± σ): 6.147 s ± 0.355 s [User: 24.952 s, System: 0.484 s]
 Range (min ... max): 5.335 s ... 6.545 s 10 runs
 
Benchmark 2: forge-pf test --match-test 'test[^(]*\([^)]+\)'
 Time (mean ± σ): 4.494 s ± 0.140 s [User: 30.911 s, System: 0.517 s]
 Range (min ... max): 4.305 s ... 4.714 s 10 runs
 
Summary
 forge-pf test --match-test 'test[^(]*\([^)]+\)' ran
 1.37 ± 0.09 times faster than forge test --match-test 'test[^(]*\([^)]+\)'

With coverage-guided fuzzing enabled:

hyperfine "forge test --match-test 'test[^(]*\([^)]+\)'" "forge-pf test --match-test 'test[^(]*\([^)]+\)'" --warmup 1 --setup "forge b" --runs 5 --conclude 'rm -rf corpus/'
Benchmark 1: forge test --match-test 'test[^(]*\([^)]+\)'
 Time (mean ± σ): 14.171 s ± 2.630 s [User: 56.604 s, System: 1.333 s]
 Range (min ... max): 11.709 s ... 18.144 s 5 runs
 
Benchmark 2: forge-pf test --match-test 'test[^(]*\([^)]+\)'
 Time (mean ± σ): 12.531 s ± 0.644 s [User: 61.457 s, System: 2.709 s]
 Range (min ... max): 11.943 s ... 13.344 s 5 runs
 
Summary
 forge-pf test --match-test 'test[^(]*\([^)]+\)' ran
 1.13 ± 0.22 times faster than forge test --match-test 'test[^(]*\([^)]+\)'
  1. Ithacaxyz/account - 1000 fuzz runs
hyperfine "forge test --match-test 'test[^(]*\([^)]+\)'" "forge-pf test --match-test 'test[^(]*\([^)]+\)'" --warmup 1 --setup "forge b" --runs 10
Benchmark 1: forge t --mt 'test[^(]*\([^)]+\)'
 Time (mean ± σ): 15.754 s ± 2.097 s [User: 67.377 s, System: 0.297 s]
 Range (min ... max): 13.318 s ... 18.829 s 10 runs
 
Benchmark 2: forge-pf test --match-test 'test[^(]*\([^)]+\)'
 Time (mean ± σ): 11.183 s ± 0.783 s [User: 80.358 s, System: 0.339 s]
 Range (min ... max): 10.260 s ... 12.490 s 10 runs
 
Summary
 forge-pf test --match-test 'test[^(]*\([^)]+\)' ran
 1.41 ± 0.21 times faster than forge t --mt 'test[^(]*\([^)]+\)'

PR Checklist

  • Added Tests
  • Added Documentation
  • Breaking changes

0xClandestine reacted with eyes emoji
@yash-atreya yash-atreya changed the base branch from master to yash/shared-corpus September 26, 2025 12:48
@yash-atreya yash-atreya moved this to In Progress in Foundry Sep 29, 2025
@yash-atreya yash-atreya changed the title (削除) wip: parallel stateless fuzzing (削除ここまで) (追記) wip perf(fuzz): parallel stateless fuzzing (追記ここまで) Sep 29, 2025
@yash-atreya yash-atreya added this to the v1.5.0 milestone Sep 29, 2025
@yash-atreya yash-atreya changed the title (削除) wip perf(fuzz): parallel stateless fuzzing (削除ここまで) (追記) perf(fuzz): parallel stateless fuzzing (追記ここまで) Oct 2, 2025
@yash-atreya yash-atreya marked this pull request as ready for review October 3, 2025 14:36
@yash-atreya yash-atreya moved this from In Progress to Ready For Review in Foundry Oct 6, 2025
@grandizzy grandizzy self-assigned this Oct 7, 2025
@jenpaff jenpaff assigned DaniPopes and unassigned grandizzy Oct 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@DaniPopes DaniPopes Awaiting requested review from DaniPopes DaniPopes is a code owner

@mattsse mattsse Awaiting requested review from mattsse mattsse is a code owner

@grandizzy grandizzy Awaiting requested review from grandizzy grandizzy is a code owner

@zerosnacks zerosnacks Awaiting requested review from zerosnacks zerosnacks is a code owner

@onbjerg onbjerg Awaiting requested review from onbjerg onbjerg is a code owner

@0xrusowsky 0xrusowsky Awaiting requested review from 0xrusowsky 0xrusowsky is a code owner

Labels

None yet

Projects

Status: Ready For Review

Milestone

v1.5.0

Development

Successfully merging this pull request may close these issues.

AltStyle によって変換されたページ (->オリジナル) /