I have a csv log file with two columns, each having time stamp for request(1 st column) and response(2nd column).
Here is some sample data:
2013-07-11 08:39:08.748,2013-07-11 08:39:08.748
2013-07-11 08:39:08.826,2013-07-11 08:39:08.826
2013-07-11 08:39:08.860,2013-07-11 08:39:08.860
2013-07-11 08:39:08.919,2013-07-11 08:39:08.919
2013-07-11 08:39:08.941,2013-07-11 08:39:08.941
2013-07-11 08:39:09.390,2013-07-11 08:39:09.390
2013-07-11 08:39:09.594,2013-07-11 08:39:09.594
2013-07-11 08:39:09.619,2013-07-11 08:39:09.619
2013-07-11 08:39:09.787,2013-07-11 08:39:09.787
2013-07-11 08:39:10.006,2013-07-11 08:39:10.006
2013-07-11 08:39:10.017,2013-07-11 08:39:10.017
2013-07-11 08:39:10.088,2013-07-11 08:39:10.088
2013-07-11 08:39:10.214,2013-07-11 08:39:10.214
I want to calculate average of difference of two columns (response - request) for complete file. This file can contain million entries in a day.
I am looking for a way to do it with shell script. Please help.
Thanks Fedorqui, I tried your script:
while read line;
do
d1=$(echo $line | cut -d, -f1);
d2=$(echo $line | cut -d, -f2);
ds1=$(date -d"$d1" "+%s");
ds2=$(date -d"$d2" "+%s");
echo "$ds2 - $ds1 = $(( $ds1 - $ds2))";
done < requestResponse.csv
It is giving me following results:
1373543260 - 1373543260 = 0
1373543260 - 1373543260 = 0
1373543260 - 1373543260 = 0
1373543260 - 1373543260 = 0
1373543260 - 1373543260 = 0
1373543260 - 1373543260 = 0
1373543260 - 1373543260 = 0
1373543261 - 1373543261 = 0
1373543262 - 1373543262 = 0
1373543262 - 1373543262 = 0
I need difference in milliseconds.
I did it in basic shell script with nanoseconds:
while read line
do
d1=$(echo $line | cut -d, -f1)
d2=$(echo $line | cut -d, -f2)
ds1=$(date -d"$d1" "+%s.%N")
ds2=$(date -d"$d2" "+%s.%N")
diff=$(echo "$ds2 - $ds1" | bc)
printf "$ds2 - $ds1 = %.3f\n" $diff
done < file
$ while read line; do d1=$(echo $line | cut -d, -f1); d2=$(echo $line | cut -d, -f2); ds1=$(date -d"$d1" "+%s.%N"); ds2=$(date -d"$d2" "+%s.%N"); diff=$(echo "$ds2 - $ds1" | bc); printf "$ds2 - $ds1 = %.3f\n" $diff; done < file
1373524748.748000000 - 1373524748.748000000 = 0.000
1373524748.826000000 - 1373524748.826000000 = 0.000
1373524748.860000000 - 1373524748.860000000 = 0.000
1373524748.919000000 - 1373524748.919000000 = 0.000
1373524748.941000000 - 1373524748.941000000 = 0.000
1373524749.390000000 - 1373524749.390000000 = 0.000
1373524749.594000000 - 1373524749.594000000 = 0.000
1373524749.619000000 - 1373524749.619000000 = 0.000
1373524749.787000000 - 1373524749.787000000 = 0.000
1373524750.006000000 - 1373524750.006000000 = 0.000
1373524750.017000000 - 1373524750.017000000 = 0.000
1373524750.088000000 - 1373524750.088000000 = 0.000
1373524750.214000000 - 1373524750.214000000 = 0.000