Python list comprehension, can it be used for reading multiple files?

Question

Here is my script so far:

#!/usr/bin/env python
import fileinput
from optparse import OptionParser

op = OptionParser()
(options, files) = op.parse_args()
print options
print files
content = []
for line in fileinput.input(files):
    # print "reading from " + fileinput.filename()
    if (fileinput.isfirstline()):
        content.append([])
        # print "is reading from the first line"
    content[-1].append(line.rstrip())

As you can see I am reading all lines in all files and constructing a structure like this

[["contents", "file 1"], ["file 2; is one line"], ["file 3", "has", "3 lines in it"]]

As far as I can tell the fileinput does not let me split up based on lines and so I have resorted to calling isfirstline() on each line to see if I have progressed to the next file.

Is it the case that I cannot use list comprehensions to do this work elegantly? Since I seem to be limited to a single loop over all lines. Is there something smart I can do to get it to give me the files separated?

Answer 1

Is it the case that I cannot use list comprehensions to do this work elegantly? Since I seem to be limited to a single loop over all lines.

In general, you should resist the urge to put too much in a list comprehension. It can make the code harder to read and debug. That said, list comprehensions work great with itertools.chain, you can put multiple for-loops in one list comprehension, and you can even next list comprehensions.

So, many answers are possible. Here are a couple of them:

[map(str.rstrip, f.readlines()) for f in files]

[[line.rstrip() for line in f] for f in files]