I have a Python script which needs to read a section of a very large text file, starting at line N and ending at N+X. I don't want to use "open('file')", because that will write the entire thing to the memory, which will both take too long, and waste too much memory. My script runs on a Unix machine, so I currently use the native head and tail functions, i.e.:
section = subprocess.check_output('tail -n-N {filePath} | head -n X')
but is feels like there must be a smarter way of doing it.. is there a way to get lines N through N+X of a text file in Python without opening the entire file?
Thanks!
Python's islice()
works well for doing this:
from itertools import islice
N = 2
X = 5
with open('large_file.txt') as f_input:
for row in islice(f_input, N-1, N+X):
print row.strip()
This skips over all of the initial lines and just returns the lines you are interested in.