I have a log file that may be very large (10+ GB). I'd like to find the last occurrence of an expression. Is it possible to do this with standard posix commands?
Here are some potential answers, from similar questions, that aren't quite suitable.
tail -n <x> <file> | grep -m 1 <expression>
: I don't know how far back the expression is, so I don't know what <x>
would be. It could be several GB previous, so then you'd be tailing the entire file. I suppose you could loop and increment <x>
until it's found, but then you'd be repeatedly reading the last part of the file.tac <file> | grep -m 1 <expression>
: tac reads the entire source file. It might be possible to chain something on to sigkill tac as soon as some output is found? Would that be efficient?If it helps, the expression is anchored at the start of a line, eg: "^foo \d+$"
.
Whatever script you write will almost certainly be slower than:
tac file | grep -m 1 '^foo [0-9][0-9]*$'