Search code examples
pythontexttailhead

reading the first N lines in a file without opening it (Python)

I have a Python script which needs to read a section of a very large text file, starting at line N and ending at N+X. I don't want to use "open('file')", because that will write the entire thing to the memory, which will both take too long, and waste too much memory. My script runs on a Unix machine, so I currently use the native head and tail functions, i.e.:

section = subprocess.check_output('tail -n-N {filePath} | head -n X')

but is feels like there must be a smarter way of doing it.. is there a way to get lines N through N+X of a text file in Python without opening the entire file?

Thanks!

Solution

  • Python's islice() works well for doing this:

    from itertools import islice

N = 2
X = 5

with open('large_file.txt') as f_input:
    for row in islice(f_input, N-1, N+X):
        print row.strip()

    This skips over all of the initial lines and just returns the lines you are interested in.