Shell Script to extract logs for last 24 hrs

Question

We are trying to extract logs from a file, we need all entries which meets a pattern and the time stamp is within last 24 hrs.

my log looks like this:

2014-07-01 01:15:59,486 WARN 86c9c59c-c362-48d5-bd8c-fb4c8b616f5a 169.179.101.7 CITIKYC_164283 stence.audit.support.impl.AuditUtilsImpl: 274 - Audit updates are successful

2014-07-01 01:15:59,487 WARN 86c9c59c-c362-48d5-bd8c-fb4c8b616f5a 169.179.101.7 CITIKYC_164283 stence.audit.support.impl.AuditUtilsImpl: 173 - Duplicate reg istration, skipping...

2014-07-01 01:15:59,488 Blah Blah..

so far, we got the logs of previous day

D=$(date +"%Y-%m-%d" -d "-1 days")
cat citikyc.log | awk '/'$D' /, /'$D' / { print $0 }' | grep "Exception\|at.*\.java\:.*" | mail -s "TESTING" xxx@yyy.com

Please help us to fetch the log for last 24hours.

Thanks in Advance...!!

Answer 1

give this one-liner a try:

awk -v d="$(date -d'24 hours ago' +'%F %T,000')" '$1" "$2>=d &&/YourSearch/' log

I didn't test, I hope no typo was made.

the date -d'24 hours ago' +'%F %T,000' will give you the timestamp 24hr ago from current.

YourSearch is your search pattern (regex).

add a test to show how it worked:

#this is my current time
kent$  date +'%F %T'
2014-07-02 15:27:46

#file content, so only last 3 lines are in "last 24 hours"
kent$  cat f
2014-06-01 01:15:59,123 foo
2014-07-01 02:15:59,123 bar bar bar
2014-07-01 01:15:59,123 foo
2014-07-01 02:15:59,123 foo
2014-07-01 03:15:59,123 foo
2014-07-01 21:15:59,123 foo
2014-07-01 22:15:59,123 foo
2014-07-01 23:15:59,123 foo

#let's get them
kent$  awk -v d="$(date -d'24 hours ago' +'%F %T,000')" '$1" "$2>=d &&/foo/' f
2014-07-01 21:15:59,123 foo
2014-07-01 22:15:59,123 foo
2014-07-01 23:15:59,123 foo