std.Io.Evented (io_uring): consider registered/fixed buffers for Reader/Writer fill paths #35734

The current read/write path of std.Io for the io_uring backend issues unregistered READ / READV / WRITE/ WRITEV SQEs with a caller-supplied buffer pointer and buf_index = 0. There's no use of registered/fixed buffers (IORING_REGISTER_BUFFERS + READ_FIXED/WRITE_FIXED).

Unregistered SQEs suffer from a performance penalty. For repeated I/O against a long-lived buffer, fixed buffers let the kernel skip per-op page pinning, which measurably reduces CPU for small/frequent I/O. Using registered SQEs could improve performance and is considered the expected model to use io_uring. A Reader's internal buffer looks like a near-ideal candidate: allocated once, reused across many fills, contiguous. My thought is that an API could be added to register an Io to the io_uring upfront.

I wanted to raise this as a possible optimization but also flag a design wrinkle I don't have a clean answer to: The backend uses per-thread rings plus a work-stealing scheduler (findReadyFiber/steal_ready_search). Registered-buffer tables are per-ring, and a buf_index is only valid on the ring it was registered against. If a fiber registers its buffer on thread A's ring and is later stolen by thread B, a READ_FIXED issued on B's ring would reference the wrong/unregistered slot. The ring init already does ATTACH_WQ, but that shares the worker pool, not the buffer table. A solution is to pin Io to its registered ring, but it works against the work-stealing design.

A solution is to pin Io to its registered ring, but it works against the work-stealing design.

Side-tracking a bit here, but in general I think work-stealing tradeoffs are tricky when you're building a general-case abstraction and have no idea what the workload patterns really look like at runtime. I also think it goes against the general grain of ZIg's usual explicitness.

I tend to think there will be many cases where software wants precise control (or at least, predictability) in the mapping of tasks/work -> threads/rings, and even to control/pin the mapping of threads/rings to CPU cores and/or NUMA domains for higher-performance software, and we shouldn't have to exit the realm of std.Io to do so.

For the simpler case of Threaded, if work-stealing is an option that can be turned off (last I checked it doesn't exist yet, but I think it's planned!), then io.concurrent() explicitly and predictably makes new threads, which is good enough (cpu or numa pinning can then be handled even today with early OS-specific calls within the threads, or later abstracted if possible?).

For a backend like Uring, support for something similar is trickier to design. InitOptions could carry some parameters about mapping threads -> cpus and/or numas, but we'd also need a way to specify a target thread(/ring) when calling some equivalent of io.concurrent(). Another option is just to have two different Uring backends: one that's general-purpose/automagic/work-stealing, and one that's built for explicit control: pinning threads, assigning concurrent tasks to specific threads/rings, registered buffers, etc.

I'm considering this and cannot think of a reason why Uring has to use a multithread and work-stealing design. Maybe I lack of some previous knowledge and information but I believe one uring can already handle submissions from concurrent sources. Submissions are in batch and non-blocking. And anyway the kernel is managing a thread pool for IO works if needed. I don't think there's a bottleneck or architectural requirement here forcing the backend to use multiple urings. As the SQE queue is just a memory space, multiple threads can write SQEs to it in a atomic lock-free way and let a dedicated thread to submit and reap them.

For sharing the buffer registration across multiple urings, Linux 6.12 has introduced io_uring_clone_buffers. This is more handy than manually registering buffers in different urings.

@ruihe774 wrote in #35734 (comment):

I'm considering this and cannot think of a reason why Uring has to use a multithread and work-stealing design.

Well, a couple of points here re threading:

• Not every kind of "I/O" will always be covered by io_uring. So the Uring abstraction would probably need some level of threading just to be able to abstract blocking non-ring calls regardless.
• In the kinds of "high perf network server" sorts of environments I'm talking about re: cpu pinning, you'd want multiple rings anyways (one per thread). The idea is that setup code (both in the application, and outside at the system level) may want to arrange to use NIC steering efficiently. You can program NIC hardware, for instance, to split (by e.g. hashing on source IPs) all the incoming traffic into, say, 16 separate queues, and you can set up mapping so that you have 16 identical listening socket copies, each of which is bound to a specific queue, which is bound to a specific IRQ, which is all bound to a specific CPU core where you have a dedicated Thread for that socket (and if using io_uring, then a dedicated ring for this Thread). This keeps all the traffic local to a specific CPU core and scales ~linearly as strategy for handling large-scale traffic efficiently.

In the kinds of "high perf network server" sorts of environments I'm talking about re: cpu pinning, you'd want multiple rings anyways (one per thread).

I think buffer registration can be easily implemented if cpu pinning is implemented. We only need to register the buffers to the pinned ring then

In zio, I have a flag that disallows tasks to migrate across threads, so it is possible to have both models in a single codebase.

However, registered buffers are a problem for the std.Io API, which is really designed to work over user-provided buffers. I don't see how this could be integrated cleanly, unless you abandon the concept of std.Io.Reader/Writer which I think would be a bad decision. In my opinion, if you need optimizations like zero-copy io_uring, you should use io_uring directly, not via the std.Io interface.

However, registered buffers are a problem for the std.Io API, which is really designed to work over user-provided buffers. I don't see how this could be integrated cleanly, unless you abandon the concept of std.Io.Reader/Writer which I think would be a bad decision. In my opinion, if you need optimizations like zero-copy io_uring, you should use io_uring directly, not via the std.Io interface.

The buffers are managed by std.Io interfaces, not supplied by the user. So I think registration can be done transparently or semi-transparently without major API changes.

@ruihe774 wrote in #35734 (comment):

The buffers are managed by std.Io interfaces, not supplied by the user. So I think registration can be done transparently or semi-transparently without major API changes.

Are you familiar with the API? Users provide buffers when doing stream.reader(io, buf). How you make it work with managed buffers? Copy from the managed ones to the user ones? Ignore the user ones?

@lukasl wrote in #35734 (comment):

@ruihe774 wrote in #35734 (comment):

The buffers are managed by std.Io interfaces, not supplied by the user. So I think registration can be done transparently or semi-transparently without major API changes.

Are you familiar with the API? Users provide buffers when doing stream.reader(io, buf). How you make it work with managed buffers? Copy from the managed ones to the user ones? Ignore the user ones?

Let me clarify it. What I meant is that the buffers are managed by Reader & Writer after they were used to initialize them. So it's possible to make the buffers registered within the lifetime of Reader & Writer. And, alternative initialization methods can be added to have Reader & Writer created with buffers owned and registered, so further IO performed on the interfaces can be done using indexed SQEs.

@ruihe774 wrote in #35734 (comment):

Let me clarify it. What I meant is that the buffers are managed by Reader & Writer after they were used to initialize them. So it's possible to make the buffers registered within the lifetime of Reader & Writer. And, alternative initialization methods can be added to have Reader & Writer created with buffers owned and registered, so further IO performed on the interfaces can be done using indexed SQEs.

But this would essentially mean "WriterGate v2.0". I don't know if that's justifiable, especially now that the std.Io.Uring backend doesn't even work properly. Maybe finishing it up should come first, and then decide if it makes sense to break the reader/writer interface in order to support managed buffers.

@lukasl wrote in #35734 (comment):

@ruihe774 wrote in #35734 (comment):

Let me clarify it. What I meant is that the buffers are managed by Reader & Writer after they were used to initialize them. So it's possible to make the buffers registered within the lifetime of Reader & Writer. And, alternative initialization methods can be added to have Reader & Writer created with buffers owned and registered, so further IO performed on the interfaces can be done using indexed SQEs.

But this would essentially mean "WriterGate v2.0". I don't know if that's justifiable, especially now that the std.Io.Uring backend doesn't even work properly. Maybe finishing it up should come first, and then decide if it makes sense to break the reader/writer interface in order to support managed buffers.

Yes, new methods are required in the vtable. But the downstream APIs of Reader & Writer can remain unchanged.