how to Redis-cli command to mass insert CSV data into Redis persistence system?

Question

My CSV data looks something like this ->

    Header1   Header2
    Key100    Value100
    Key200    Value200

It has only two columns with Header as first row. The columns are keys and respective values. Now, my requirement is -

1. I want to store key & value as String data type
2. Need to exclude the Header(the first row) while storing
3. Need to exclude null or " " keys
4. it is a ',' separated data
5. The word "Data:" should be appended to each key while inserting.

For example - for the above provided csv data, redis should store as -

Data:Key100
Data:Key200

Result:

When I do get Data:Key100 my output should be : "Value100"

Have tried with awk command but it is keep on giving error - invalid stream id specified

awk -F ',' 'FNR > 1 {print "XADD myStream \""($1=="" ? "NA" : $1)" column_1 "($2=="" ? "NA" : $2)" column_2 ... \n"}' data.csv | redis-cli --pipe

Please help me to get the command. Thank you

Answer 1

I think you mean this, which sets an individual key to a value for each line of your file, but I am not sure why you seem to be trying to use a stream:

awk -F ',' 'FNR>1 { if(!$1){$1="NA"}; if(!$2){$2="NA"}; printf("SET %s %s\n",$1,$2)}'  data.csv | redis-cli --pipe

For this file (which has the key missing on line 3 and the value missing on line 5):

Header1,Header2
Key100,Value100
,fred
Key200,Value200
bill,

That produces this code:

SET Key100 Value100
SET NA fred
SET Key200 Value200
SET bill NA

Which then enables you to do this:

redis-cli get Key100
"Value100"

Also, you say you want to exclude null keys, but your code replaces them with "NA" and sets them anyway, so you may prefer to ignore them altogether with:

awk -F ',' 'FNR > 1 && $1 && $2 {printf("SET %s %s\n",$1,$2)}'  data.csv 

which produces:

SET Key100 Value100
SET Key200 Value200