Datetime to epoch conversion

Question

I have a bash question (when using awk). I'm extracting every single instance of the first and fifth column in a textfile and piping it to a new file with the following code,

cut -f4 test170201.rawtxt | awk '/stream_0/ { print $1, $5  }' > testLogFile.txt 

This is part of the file (test170201.rawtxt) I'm extracting the data from, columns Timestamp and Loss,

Timestamp                 Stream     Status     Seq         Loss Bytes    Delay
17/02/01.10:58:25.212577  stream_0     OK      80281          0  1000     38473
17/02/01.10:58:25.213401  stream_0     OK      80282          0  1000     38472
17/02/01.10:58:25.215560  stream_0     OK      80283          0  1000     38473
17/02/01.10:58:25.216645  stream_0     OK      80284          0  1000     38472

This is the result I'm getting in testLogFile.txt

17/02/01.10:58:25.212577 0
17/02/01.10:58:25.213401 0
17/02/01.10:58:25.215560 0
17/02/01.10:58:25.216645 0

However, I want the Timestamp to be written in epoch in the file above. Is there an easy way of modifying the code I already have to do this?

Answer 1

Given:

$ cat file
Timestamp                 Stream     Status     Seq         Loss Bytes    Delay
17/02/01.10:58:25.212577  stream_0     OK      80281          0  1000     38473
17/02/01.10:58:25.213401  stream_0     OK      80282          0  1000     38472
17/02/01.10:58:25.215560  stream_0     OK      80283          0  1000     38473
17/02/01.10:58:25.216645  stream_0     OK      80284          0  1000     38472

You can write a POSIX Bash script to do what you are looking for:

while IFS= read -r line || [[ -n "$line" ]]; do
    if [[ "$line" =~ ^[[:digit:]]{2}/[[:digit:]]{2}/[[:digit:]]{2} ]]
    then
        arr=($line)
        ts=${arr[0]}
        dec=${ts##*.}   # fractional seconds
        # GNU date may need different flags:
        epoch=$(date -j -f "%y/%m/%d.%H:%M:%S" "${ts%.*}" "+%s")
        printf "%s.%s\t%s\n" "$epoch" "$dec" "${arr[4]}"
    fi  
    done <file >out_file

$ cat out_file
1485975505.212577   0
1485975505.213401   0
1485975505.215560   0
1485975505.216645   0

For GNU date, try:

while IFS= read -r line || [[ -n "$line" ]]; do
    if [[ "$line" =~ ^[[:digit:]]{2}/[[:digit:]]{2}/[[:digit:]]{2} ]]
    then
        arr=($line)
        ts="20${arr[0]}"
        d="${ts%%.*}"
        tmp="${ts%.*}"
        tm="${tmp#*.}"
        dec="${ts##*.}"    # fractional seconds
        epoch=$(date +"%s" --date="$d $tm" )
        printf "%s.%s\t%s\n" "$epoch" "$dec" "${arr[4]}"
    fi  
    done <file >out_file

For an GNU awk solution, you can do:

awk 'function epoch(s){ 
                        split(s, dt, /[/:. ]/)
                        s="20" dt[1] " " dt[2] " " dt[3] " " dt[4] " " dt[5] " " dt[6]
                        return mktime(s) "." dt[7]} 
     /^[0-9][0-9]/      { print epoch($1), $5 }'    file >out_file

If you don't want the fractional second included in the epoch, they are easily removed.