Parsing logs in sed efficiently

Question

I have an application which log file has most lines start with a time/date, followed by a debugging level and the actual log message. Often I want to watch the output live and colors make it a lot better readable. This is what I use at the moment for piping the logs:

while read line; do
echo "$line" |
sed -e "s,^.* DEBUG,$(tput setaf 2)&$(tput sgr0)," \
 -e "s,^.* INFO,$(tput setaf 2)&$(tput sgr0)," \
 -e "s,^.* WARN,$(tput setaf 3)&$(tput sgr0)," \
 -e "s,^.* ERROR,$(tput setaf 1)&$(tput sgr0)," \
 -e "s,^.* SEVERE,$(tput setaf 1)$(tput bold)&$(tput sgr0),"
done

The call on the command line looks like this:

tail -fn 200 /path/server.log | ./logparser

This all works as expected but when many lines are printed at once the parser lags behind. I think this is because I run sed once for every single line. Is there a better solution?

• itsbruce: You're wasting time having the shell read input and repeat it to sed. Why are you doing that? sed would just read from standard input anyway? The logparser script could just be one line or you could put the commands in an input file and throw away the logparser script.
• itsbruce: tail -fn 200 /path/server.log | sed -f /path/to/sedcommands
• Hugo G: How would I use tput in a sed script? I really prefer tput over the arcane obscure escape key notation for terminal colors.
• Hugo G: @itsbruce feel free to post a real answer and you'll get some points :-)
• itsbruce: Oh, well, I would but I wasn't so much reviewing your shell script as talking unix admin basics. When you have it working the way you want, post your own answer and say what worked for you. Perfectly acceptable StackExchange behaviour.

Answer 1

Thanks to itsbruce this runs much faster:

sed -e "s,^.* DEBUG,$(tput setaf 2)&$(tput sgr0)," \
 -e "s,^.* INFO,$(tput setaf 2)&$(tput sgr0)," \
 -e "s,^.* WARN,$(tput setaf 3)&$(tput sgr0)," \
 -e "s,^.* ERROR,$(tput setaf 1)&$(tput sgr0)," \
 -e "s,^.* SEVERE,$(tput setaf 1)$(tput bold)&$(tput sgr0),"

Note: first two and the last line are removed.

Since sed takes the STDIN, this basically starts one sed instance for whole stream in contrast to one per line.