Providing a non-blocking IO API in a C library

Question

I am working on a C library (SlipRock) for interprocess communication.

The library currently exposes a simple, blocking API. This is easy to use, makes misuse (relatively) difficult (this is C after all), and will be (relatively) easy to implement cross-platform, once I get around to implementing the Windows port (which shares essentially no common code).

However, I am worried that the blocking API might cause problems in certain use cases. Nevertheless, I am very worried about providing a non-blocking API that is easy to use and doesn’t require the use of heavyweight libraries like libuv or libevent.

Bindings to languages that naturally do pseudo-blocking (blocks the green thread) IO (such as Go, Erlang, or GHC Haskell) can always just implement the higher-level API (or even the entire protocol – it isn’t that complex) in the higher level language. Similarly, it is possible to implement the parts of the protocol that are not by-nature blocking (listening on a named pipe/AF_UNIX socket and connecting to an AF_UNIX socket, as well as the following exchange of passwords) in the higher-level language.

Should I provide a non-blocking C API, or should I just provide the 2 or 3 functions thatbindings would need to use as the basis for the API?

The way I was thinking of implementing the non-blocking API was by having a state machine that the user is required to manually invoke. But that requires the user to have access to the underlying IO handle – which is the intended return value! So I either need to break encapsulation or tie myself to an event loop implementation if I want to expose an async API for this not-very-performance-critical code.

• Robert Harvey: If you provide a non-blocking API, how will you implement it? Will the user register a callback function? Will you support Futures or Promises? Will you allow continuations? If you don't provide a non-blocking API, can you rely on the user properly implementing asynchrony himself? Would you provide guidance for such? Do C calling conventions (or the C language itself) provide adequate support for asynchrony? Would you recommend that your users employ some external library to implement the asynchrony over your synchronous methods?
• Demi: @RobertHarvey C most certainly does not provide adequate support. At least not without tying myself to an event loop implementation, which I really don’t want.
• Blrfl: If you're dependent on file descriptors, what's wrong with using select(2) or poll(2) (both portable to any POSIX.1 system, BTW) with a zero or infinite timeout depending on whether or not the user wants to block? If you're providing a non-blocking interface, then by definition you're leaving responsibility for how it's (mis)used to the callers.
• Blrfl: Microsoft pretty clearly didn't get the memo about that.
• ChrisW: The C-compatible Win32 API supports asynchronous I/O, see e.g. here and here.

Answer 1

You might also provide low-level operating system specific hooks to enable (or facilitate) non-blocking IO

For example, for Linux and POSIX system, you could provide a way to get the active file descriptor sets (and leave your user to call poll(2) on them, probably providing the user some hooks to do the actual I/O when poll detects available IO), and perhaps to be notified when that set is changing. The multi interface of libcurl could be inspirational (and perhaps also 0mq). And you might even try to target aio(7)

Perhaps you might design your library above some existing event loop library (like libevent or libev).

Don't forget to document wisely the asynchronous aspects of your library, since they are always tricky. You could also document how to use it in a multithreaded application.

Answer 2

Whether you should provide a non-blocking API depends on the needs of your users. I agree that forcing users to add a dependency to a heavyweight library is a bad idea. You might put the non-blocking API in a separate library.

On the other hand, I don't agree that you must break encapsulation. I don't know the details of your API, but I am sure that you can hide the file handle in some way.

Did you ask actual users of your library to see what they need? If not, why add a non-blocking API?

• Demi: My library has no users – yet. It is very incomplete and nowhere near production-ready. That will need to wait at least until it works on Windows.
• Frank Hileman: @Demi Perhaps you could speak to potential users? There must be a reason you are creating this.
• Demi: Because so many systems get it wrong. Zinc (Scala compiler). LLDB. SLIME. ZeroBrane Studio. Py4J. All of the trivially vulnerable to local (and possibly remote!) code execution exploits. All of them. That needs to change.

Answer 3

If you do want to provide a non-blocking interface, there are only a few ways of doing it:

1. Embed the client code in your event loop.

   You provide your own event loop/dispatcher. Add an interface for the client to add their own sockets, timeouts and event callbacks.

   You run your event loop, manage your internal async state, call the client callbacks when their sliprock write completes, or when a select/poll/whatever event happens on a socket they registered with you

2. Expose your fds to the client event loop.

   Require the client to call your handler when something happens to one of your internal fds.

The hybrid option is run your own event loop in a dedicated thread which handles your sockets and your async state, and communicate with the client code via a single exposed socketpair or eventfd which they can poll in their existing event loop.

In both the first option and the hybrid one, you avoid the requirement for your client to manually invoke your state machine.