loglimit

loglimit is a very simple command line tool that limits the amount of data a program outputs. This is intended to limit the amount of log data written to disk, preventing an appliaction-layer denial-of-service attack where an attacker fills up all free disk space with logs.

Usage

loglimit is very simple. It reads data from standard input and writes that data to standard output, unless the limit for the number of bytes written in the given time period has been reached. If this limit is bypassed, all further standard input during this time period will be discarded, and a message about this is written to standard error.

Example

Consider a publicly facing server called server that outputs its logs to standard output. One very easy way of saving these logs is to redirect stdout to a file, but that might allow an attacker to exhaust the disk space, depending if it's possible to make the server produce a large amount of logs quick enough to exhaust the disk space before a sysadmin notices.

Suppose you want to write output logs to a file, but you want to limit the log rate to 1 GB per day. This is done with the following command:

server | loglimit --limit 1000000000 --time 86400 > out.log 2> loglimit.log

Some notes:

• All logs from server are written to out.log.
• Any logs produced by loglimit are written to loglimit.log. These are error messages that are produced if the daily limit of data written is exceeded. Once log data can be output again, a log message is printed including the exact byte position resuming from and the number of bytes discarded.
• 86400 is the number of seconds in a day (60 * 60 * 24).
• A limit of 1 GB in 24 hours is not equivalent to, say, 500 MB in 12 hours. To see why that's the case, suppose that no data has been input for the past day and then all of a sudden 800 MB is input over an hour.
  • In the former example, all 800 MB would be written, with 200 MB allowed for the remaining day.
  • In the latter example, only 500 MB is written and 300 MB is discarded. 12 hours after the data is input, the amount of data that can be written is reset back to 500 MB.