[ANN] cqueues 20140729
[
Date Prev][
Date Next][
Thread Prev][
Thread Next]
[
Date Index]
[
Thread Index]
- Subject: [ANN] cqueues 20140729
- From: William Ahern <william@...>
- Date: Fri, 1 Aug 2014 09:18:51 -0700
I recently tagged a new release of cqueues, a non-blocking event framework
for modern Unix systems. I wanted to wait until I finished adding O(1)
timeouts (worst case, any operation), making it the only open source event
framework in any language to have O(1) timeouts. But I haven't yet had the
time to finish the integration.
Notable recent additions include:
* Promise/Future module patterned after C++11. Not strictly necessary
when you have coroutines, but eases writing things like caches. Built
around the Condition variable module added in the last big release.
* DNS resolver pools. The underling non-blocking resolver library uses one
socket per query for security and to simplify descriptor management. For
DNS-heavy applications it was suggested to use a resolver pool in order
not to not waste descriptors. Now the pool interface comes built-in. The
resolver pool module has been running in a high-traffic (millions of
queries per day per instance) DNS-heavy cloud service for several weeks.
* All buffered I/O routines now support timeouts. Note that this slightly
changed the behavior of :starttls.
* HTTP multipart MIME body parsing is now much easier and more performant
thanks to a specialized socket read format which takes a MIME boundary and
returns chunks until the boundary is found.
* Text-mode block reads now should behave just like C's stdio. That is, if
1024 bytes are requested, then 1024 bytes are returned, which may have
necessitated reading more than 1024 bytes from the socket given any EOL
translations or to ensure a trailing carriage return is not followed by a
linefeed. Previously all block reads were performed as-if in binary mode,
without any EOL translation.
* Fast metatable checking in the sockets module using upvalues.
* Functions can now be passed through thread.start. dladdr+dlopen is used to
pin C functions in memory to make things thread-safe. Otherwise a function
pointer could become invalid if the Lua VM it was sent from unloaded the
module.
* Socket module now preserves errors across invocations to prevent buggy
applications from silently dropping data. :clearerr should be used when a
recoverable error is encountered and dealt with.
* Socket module now throws an error when it detects too many consecutive
uncleared errors. This is a back-stop to help catch bugs exposed by the
previous feature of preserving errors.
Homepage: http://25thandclement.com/~william/projects/cqueues.html
Documentation: http://25thandclement.com/~william/projects/cqueues.pdf
About cqueues:
cqueues is a high-performance, coroutine based event loop which wraps epoll,
kqueue, and Solaris ports event descriptors, as well as their cousin
interfaces such as signalfd and inotify.
* Composable: Each event loop is itself pollable, and can be easily tied
into existing event loop systems in the larger application, whether in
Lua, C, or some other language. loop:pollfd() returns the underlying
epoll, kqueue, or ports descriptor, and will signal ready when a child
coroutine is ready to resume.
Additionally, the modules maintain no global state in the user process,
nor will they ever block the process. No callbacks are used, so C code
within or outside the event loop does not need to worry about reentrancy.
You can synthesize pollable objects by providing the methods :pollfd,
:events, and :timeout.
* High-performance: Because the loop controller pins every polled object
into memory for at least one full scheduling cycle, descriptor management
is optimal, without recourse to one-shot polling or complex reference
counting.
* No Dependencies: The only dependencies needed are already installed on
your server--pthreads, OpenSSL, and of course Lua!
* Comprehensive: The package provides modules for sockets, signals, native
file change notification, POSIX thread management, DNS resolution, and
light-weight Lua thread synchronization.
+ cqueues.socket: Full-, line-, and non-buffering sockets with CRLF/LF
translation, automagic DNS resolution, SSL/TLS capabilities, and Unix
file descriptor passing.
+ cqueues.signal: Employs Linux signalfd and BSD EVFILT_SIGNAL.
On Solaris synthesizes non-blocking signal polling with sigtimedwait.
+ cqueues.notify: Employs Linux inotify, BSD EVFILT_VNODE, and Solaris
PORT_SOURCE_FILE for efficient file change notifications.
+ cqueues.thread: Simple POSIX thread management, with pollable
thread handles and automatic socket channel creation for easy
inter-thread communication. As the sockets module supports descriptor
passing, it's trivial to transfer open files or sockets between POSIX
threads.
+ cqueues.dns: Full DNS resolver (stub and recursive) with ability
to configure itself from common /etc configuration files on BSD and SysV
systems.
+ cqueues.condition: Light-weight, user-defined condition variables
for efficient synchronization of Lua threads.
+ cqueues.promise: Promise/Future pattern module.
* Maintained: cqueues is used to power multiple commercial services and
products with significant traffic. Bugs are fixed quickly, and new
features added on a steady basis. This and the non-open source predecessor
project have been in continuous use and development for nearly 4 years.