Interception library

The libioil library can be used to improve the performance of read and write operations to DFuse from applications which use libc. The library bypasses the kernel by intercepting POSIX read and write calls from the application and servicing them directly in user-space.

Using libioil

To use the interception library, set LD_PRELOAD to point to the shared library in the DAOS install directory:

LD_PRELOAD=/usr/lib64/libioil.so

For example:

$ddif=/dev/zeroof=./foobs=1Gcount=20
20+0recordsin
20+0recordsout
21474836480bytes(21GB,20GiB)copied,45.6925s,470MB/s
$LD_PRELOAD=/usr/lib64/libioil.soddif=/dev/zeroof=./barbs=1Gcount=20
20+0recordsin
20+0recordsout
21474836480bytes(21GB,20GiB)copied,15.735s,1.4GB/s

It is also possible to link applications with the interception library at compile time with the -lioil flag.

For evaluating performance using the interception library, see Performance considerations and Test performance.

Monitor activity

To enable activity reports for the interception library, set the D_IL_REPORT environment variable. This returns a short summary of activity in the shared library destructor, typically as a program exits.

The value assigned to D_IL_REPORT defines the number of read and write calls to log to stderr. To log all calls, set the value to -1. To print the summary at program exit only, set the value to 0.

For example:

$D_IL_REPORT=1LD_PRELOAD=/usr/lib64/libioil.soddif=/dev/zeroof=./barbs=1Gcount=5
[libioil]Interceptingwriteofsize1073741824
5+0recordsin
5+0recordsout
5368709120bytes(5.4GB,5.0GiB)copied,4.13708s,1.3GB/s
$D_IL_REPORT=3LD_PRELOAD=/usr/lib64/libioil.soddif=/dev/zeroof=./barbs=1Gcount=5
[libioil]Interceptingwriteofsize1073741824
[libioil]Interceptingwriteofsize1073741824
[libioil]Interceptingwriteofsize1073741824
5+0recordsin
5+0recordsout
5368709120bytes(5.4GB,5.0GiB)copied,4.18626s,1.3GB/s
$D_IL_REPORT=-1LD_PRELOAD=/usr/lib64/libioil.soddif=/dev/zeroof=./barbs=1Gcount=5
[libioil]Interceptingwriteofsize1073741824
[libioil]Interceptingwriteofsize1073741824
[libioil]Interceptingwriteofsize1073741824
[libioil]Interceptingwriteofsize1073741824
[libioil]Interceptingwriteofsize1073741824
5+0recordsin
5+0recordsout
5368709120bytes(5.4GB,5.0GiB)copied,4.12114s,1.3GB/s

Some programs, including most GNU utilities from the coreutils package, have a destructor function to close stderr on exit. This means that, while the interception library works for many basic commands such as cp and cat, it's not possible to see the summary generated by the interception library.