Last Updated: February 25, 2016

Generate Redis protocol from csv using Awk

An example approach for reformatting csv data into Redis protocol, to be bulk loaded with the console client. Data is binned into 5 minute samples for time series analysis.

awk -f redis-proto.awk data.csv | redis-cli --pipe

redis-proto.awk:

BEGIN { FS="," }
 {
 bin=sprintf("%s%04d", strftime("%Y%m%d",2ドル), 1ドル - (1ドル % 5))
 fld=sprintf("%s:%s", 3,ドル 4ドル)
 rate=int((5ドル) / 1000)

 print "*4\r\n7ドル\r\nHINCRBY\r\n12ドル\r\n" bin "\r\n$" length(fld) "\r\n" fld "\r\n$" length(rate) "\r\n" rate "\r"
 }

#redis

#awk

Written by Matt Keranen

Say Thanks

Respond

Related protips

Get the total number of jobs on all Resque queues

1.867K

Quickly move and delete Redis keys by pattern

92.67K

List all databases in a Redis instance

44.65K

2 Responses

Add your response

inkel

I think you're missing a \n after rate "\r", but other than that, this is pretty cool!

over 1 year ago ·

datasaur

IIRC that was because the final \n is added to the line automatically by the print statement. Redis protocol does not like extra carriage returns. I could probably use a printf and be more explicit.

over 1 year ago ·