API design: stream objects vs. functions vs. messages

Question

I'm designing API for a python library that accepts asynchronous input and produces the asynchronous output: various signals come in, and various signals are generated in response (there's no one-to-one relationship between input and output signals).

The input could come from a network socket/file/interactive (terminal-based) user input; or from a web framework view function. Similarly, the output may have to be sent to a socket/file/screen, or returned to a caller (in this last case, it should be buffered, and all output accumulated since the last call should be returned). The choice between different I/O channels is made once at the beginning and isn't changed while the program is running.

I assume the I/O layer uses multiple threads or processes or is asynchronous. And it calls my library functions.

What are the pros/cons of accepting as arguments two stream objects (for input and output); exposing functions that need to be called when an input is available or output is desired; using messages to communicate? Or perhaps some other approach is better?

• Walfrat: that feel like some kind of IO's god no ? Are you sure you're not aiming too wide ? Whatever, there is some missing information : what if your application crash : does the pending works is lost or do you need to be able to execute it after the application restart ?
• max: @Walfrat Yeah, now I'm guessing it's just too much to ask, since it's kinda combining synchronous and asynchronous API in one. Re: pending work - that would be too hard, it's fine if it's lost; although some limited recovery can be achieved by some middle layer buffering and resubmitting some inputs when it figures out that they definitely weren't processed.
• Walfrat: Well a very generic API would be that you act like everything is aynschronous, and you give an input stream, an output stream, and a callback which get called when the work is done.
• k3b: For my feeling the question is to broad. It is hard to answer without knowing details of the usecase.
• yegodm: Are you going to validate input? If yes, how and where?

Answer 1

Briefly:

• messages: a straightforward paradigm, and often the easiest to interface with, especially if you want to target a wide range of i/o (tcp, http, terminal...). In most cases, you have an independent in-between layer to cache and handle i/o streams.

• functions: well, that's perfect for libraries used in the same language. But if you want to interface with it at a higher level, you will need a wrapper around it anyway.

• streams: accepting streams directly in the lib is similar to messages but has a drawback. Mainly, everything regarding what can go wrong with streams (disconnected, timeouts, errors...) tends to end up as exception handling in your "business logic" instead of a separate layer.

IMHO, I wouldn't see it as an "this or that" question. Rather it might be interesting to look at it from this perspective:

streams are used to transmit messages which are the result of function calls