-
Notifications
You must be signed in to change notification settings - Fork 1
Releases: Venkat2811/myelon
v0.1.0-alpha.2 — Post-launch dep + CI hygiene
[0.1.0-alpha.2] — 2026年05月21日
Post-launch dependency and CI hygiene. No public-API changes; safe drop-in upgrade from 0.1.0-alpha.1.
Changed
- GitHub Actions dependencies bumped:
- Cargo patch-updates group (#8):
macroquad0.4.14 → 0.4.15 (used bymyelon-pulse-vanitybrand demo)serde_jsonpatch bump
CI infrastructure fixes (already in v0.1.0-alpha.1 tag, recorded here for completeness)
The first public push to main exposed several CI gaps that were patched before v0.1.0-alpha.1 was tagged. Listing them so the alpha.2 release notes are a complete snapshot of what's in CI today:
- Removed the broken duplicate
ci.ymlworkflow (hadenv.RUST_MSRVin a job-levelname:which Actions validation rejected; the focused per-gate workflows already cover its surface). actions/checkoutnow fetches submodules (submodules: recursive) in every workflow that runs cargo against the workspace. Required becausecompetitive-benchhas a path dep on thecrossbarsubmodule.- Excluded
competitive-benchfrom main CI workflows (build_and_test.yml,lint.yml,dst.yml,test_coverage.yml). It pulls heavy C / C++ system deps (libbsd, libzmq, libboost, libopenmpi) that aren't worth installing on every PR for a crate that isn't published. - Dropped the Miri job from
build_and_test.yml; Miri rejectsshm_openandmmapcalls (unsupported operation: can't call foreign function). A properly-scoped Miri lane against pointer-math helpers is on the roadmap (Safety & quality gates Tier 2). - Loosened the
perf-bench::infra::output::log::test_overhead_is_lowthreshold from 1 μs to 5 μs to tolerate CI debug-build hardware variance; the steady-state release-mode cost is still ~2ns/op.
Acknowledgements
- Codex (OpenAI) and Claude Code (Anthropic) shared the agentic-engineering load on this release. See the README's "Built with" section.
Assets 2
v0.1.0-alpha.1 — Initial public launch
[0.1.0-alpha.1] — 2026年05月21日
Initial public launch of disruptor-mp and myelon on crates.io.
disruptor-mp is the raw multiprocess substrate that extends the single-process LMAX Disruptor design to cross-process IPC on POSIX shared memory and memory-mapped files. myelon is the layered typed transport built on top of it (framing, codecs, typed zero-copy, topology helpers, layout helpers).
The two crates ship together at the same version because their public surfaces are tightly coupled. They move in lockstep until the surfaces stabilize.
Features included in this release
disruptor-mp: raw multiprocess substrate
- Cross-process Single Producer Single Consumer (SPSC).
- Cross-process Single Producer Multi Consumer (SPMC).
- Cross-process Multi Producer Single Consumer (MPSC).
- Cross-process Multi Producer Multi Consumer (MPMC).
- Communication patterns:
- Ping-pong: request/response RTT (two SPSC rings).
- Broadcast: strict fan-out; every consumer sees every event, slowest gates the producer.
- Signal: pipelined fan-out, no ack; maximum throughput.
- Broadcast + per-rank ping-pong: one SPMC dispatch ring + N SPSC return rings (driver ↔ N worker ranks); the inference-fabric shape.
- Two type-identical backends:
- POSIX shared memory (
shm_open). - Memory-mapped file (
mmap).
- POSIX shared memory (
- Memory-level zero-copy reads (
&Einto the ring slot). - Wait strategies (
AutoWaitStrategy):-
BusySpin,BusySpinWithSpinLoopHint,SpinThenYield { spins },Sleep(Duration),Block.
-
- Liveness for gating consumers: producer-side stall detection with cold-path alert, optional hook, recoverable rejoin.
- Portable shared-memory naming (macOS 31-byte budget enforced).
- Hot-path observability counters:
-
metrics-rs facade (default). - Prometheus exporter (
metrics-prometheusfeature). - OpenTelemetry / OTLP exporter (
metrics-otelfeature).
-
- Deterministic-simulation hooks behind
RUSTFLAGS="--cfg dst"(FoundationDB BUGGIFY / TigerBeetle VOPR-style assertions, scenario profiles, runtime). - HFT-grade deployment tuning (deferred):
- Hugepages-backed SHM segments.
- Core pinning /
isolcpusintegration in the builder API. - NUMA-aware SHM placement.
myelon: layered transport on top of disruptor-mp
- Re-exports the raw substrate at type-identical types (one dep gets you everything).
- Framed transport:
&[u8]payloads in fixed-size frames; multi-frame fragmentation for payloads larger than one frame (start/end flags + message id let the consumer reassemble). - Compile-time-fixed frame size:
FixedFrame<N>andAlignedFixedFrame<N>(aligned variant for zero-copy reads). - Typed transport (codec encodes
T→ bytes; consumer decodes back into an ownedT):- bincode.
- rkyv.
- flatbuffers.
- Typed zero-copy (consumer reads fields in place via
ZeroCopyCodec::access; no decode step, no allocation):- rkyv (
Archived<T>). - flatbuffers root tables.
- rkyv (
- Topology helpers for inference fabrics: rank-scoped request/response, producer-owned startup, attach-time wait-strategy metadata.
myelon-dst: internal deterministic-simulation harness
- Runner with fault injection and invariant oracle.
- Verification, report emission, DST-coverage sweep.
perf-bench: internal broad transport sweep harness
- Pingpong, broadcast, signal, repeatability binaries.
- Layer matrix: raw, framed, codec, typed-zero-copy (all ×ばつ shm / mmap).
- Throughput and CO-aware fixed-rate measurement modes.
- Tier ladder:
super-tiny,simple-smoke,smoke,quick,extensive.
competitive-bench: internal external-comparison harness
- Adapters: Crossbeam, Iceoryx2, Rusteron (Aeron), shmipc-rs, ZeroMQ (IPC / IPC-abs / TCP), Boost.MQ, OpenMPI.
- Tier ladder matching
perf-bench. - Aggregate report and Pareto-frontier summary per run.
Bindings
- Python.
- C / C++.
- Zig.
Validation
Two reference machines exercise different parts of the surface.
Reference machine for the README's headline numbers — venkat-pc:
- AMD Ryzen 7 5800X (8 cores / 16 threads, 4.85 GHz boost, 32 MiB L3 cache).
- 64 GiB DDR4.
- Ubuntu 22.04, kernel 6.8.
The Linux Ryzen box is the bench-grade reference: pinned cores, stable frequency, deterministic NUMA topology. All five headline tables (signal scaling, raw ping-pong, framed payload scaling, broadcast scaling, typed zero-copy ping-pong) come from this machine.
Dev machine for pre-release validation gates:
- Apple M3 Max (14 cores).
- 96 GiB unified memory.
- macOS 26.1,
rustc 1.90.0.
Gates that passed clean on the dev machine before tagging this release:
cargo check --workspace --all-features✓cargo clippy --workspace --all-targets -- -D warnings✓cargo fmt --check✓cargo test --workspace --all-features✓cargo doc --workspace --no-deps✓ (workspacebroken_intra_doc_links = deny)RUSTFLAGS="--cfg dst" cargo test -p myelon-dst✓ (DST harness assertions + verifier)make -C crates/perf-bench super-tiny✓: 94 measurements parsed (40 single-shot rows + 54 inline-sweep rows + 40 codec rows) across raw-ring, raw-myelon, framed, broadcast, signal, and codec layers ×ばつ shm / mmap ×ばつ max-throughput + CO@50K modes.make -C crates/competitive-bench super-tiny✓: 220 / 220 scenario JSON files non-empty and non-zero across the 12 adapters listed above + internal baselines (Disruptor-shm, Disruptor-mmap, Myelon-Raw-shm, Myelon-Raw-mmap). Two third-party adapters (shmipc-rs,rusteron-aeron-ipc) were excluded from the run because of pre-existing build / runtime issues unrelated to this release; they are tracked separately.- MSRV check at declared
rust-version = "1.86"for the publishable crates ✓. cargo publish --dry-run -p disruptor-mp✓ (68 files, 146 KiB compressed).
Smoke-grade absolute numbers from the M3 Max super-tiny runs are intentionally not promoted alongside the README's headline tables. The bench-grade reference is the pinned-frequency Linux Ryzen box; Apple-silicon peak numbers will land in a separate cross-platform comparison artifact in a follow-up release rather than being mixed into the headline surface.
Safety & quality gates
Both crates do non-trivial unsafe work internally (raw ring slots, SHM mmap, atomic ordering, manual cache-line layout). The public API is safe Rust, with a small set of documented escape hatches. The full tier-by-tier checklist also lives in the workspace README and the two are kept in sync.
Tier 1: public-surface contract (required for crates.io)
-
myelonpublic surface: zeropub unsafe fnacross the crate. -
disruptor-mppublic surface: four documentedpub unsafe fnescape hatches inobservability:CountersFile::initCountersFile::attachCountersFile::from_ptrAggregatorHandle::spawn
Each carries a
# Safetyrustdoc section stating the caller's obligation. These are intentional power-user escape hatches for callers who already hold a verified pointer or own a shared-memory segment. -
Workspace lint
unsafe_op_in_unsafe_fn = warn(everyunsafeaction inside anunsafe fnmust be re-wrapped in an explicitunsafe { ... }block). -
Workspace lint
clippy::missing_safety_doc = warn(every publicunsafe fnmust carry a# Safetyrustdoc section). -
cargo clippy --workspace --all-targets -- -D warningsclean.
Tier 2: internal correctness
- DST harness:
assert_always/assert_sometimes+ BUGGIFY fault injection, FoundationDB + TigerBeetle-style. Run withRUSTFLAGS="--cfg dst". Plays the role Loom plays for in-process atomics, but scaled to cross-process scenarios. - Per-block
// SAFETY:comment on every internalunsafe { ... }block. Currently 12 / 81 blocks ≈ 15% coverage. The remaining blocks rely on cursor monotonicity, cache-line alignment, and slot-lifecycle invariants established at the type-system / builder layer, but lack explicit per-block justification comments. Backfill scheduled forv0.1.0-alpha.2. -
cargo miri testlane (pointer-math helpers; not the syscall paths). - AddressSanitizer lane.
- ThreadSanitizer lane.
Tier 3: enforcement and audit
-
clippy::undocumented_unsafe_blocks = warnenabled workspace-wide (after Tier 2 backfill). - Safe wrappers around the four
pub unsafe fn(boxed(),with_shm_segment()); the four escape hatches stay as power-user surface. -
cargo geigeraudit reported.
Why 0.1.0-alpha.1 (and not 3.x.y)
disruptor-mp carried version = "3.7.1" internally because this fork started from upstream disruptor v3.7.1. That number was never on crates.io under the disruptor-mp name; using it on first publish would have falsely implied a mature 3.x history. The reset to 0.1.0-alpha.1 explicitly signals "first public release, API may evolve during the early-iteration window."
myelon is similarly first-time-public at 0.1.0-alpha.1.
The pre-release tag (Cargo equivalent of PyPI's 0.1.0a1) means cargo add / cargo update won't pick these versions by default. Consumers opt in explicitly:
disruptor-mp = "=0.1.0-alpha.1" myelon = "=0.1.0-alpha.1"
Deferred
Tracked for follow-up releases. Safety-related follow-ups are listed inline in the Safety & quality gates tier checklist above; this section covers the rest.
- Hugepages-backed SHM, core pinning, NUMA-aware placement (HFT-grade deployment tuning).
- DST-coverage report (assertion-log review pipeline).
- Multi-process liveness integration test.
- Workspace-wide
cargo-llvm-covworkflow. - Exporter feature smoke tests (
metrics-prometheus,metrics-otel). - `flat...