How to open every file in a folder

Question

I have a python script parse.py, which in the script open a file, say file1, and then do something maybe print out the total number of characters.

filename = 'file1'
f = open(filename, 'r')
content = f.read()
print filename, len(content)

Right now, I am using stdout to direct the result to my output file - output

python parse.py >> output

However, I don't want to do this file by file manually, is there a way to take care of every single file automatically? Like

ls | awk '{print}' | python parse.py >> output 

Then the problem is how could I read the file name from standardin? or there are already some built-in functions to do the ls and those kind of work easily?

Thanks!

Answer 1

Os

You can list all files in the current directory using os.listdir:

import os
for filename in os.listdir(os.getcwd()):
   with open(os.path.join(os.getcwd(), filename), 'r') as f: # open in readonly mode
      # do your stuff

Glob

Or you can list only some files, depending on the file pattern using the glob module:

import os, glob
for filename in glob.glob('*.txt'):
   with open(os.path.join(os.getcwd(), filename), 'r') as f: # open in readonly mode
      # do your stuff

It doesn't have to be the current directory you can list them in any path you want:

import os, glob
path = '/some/path/to/file'
for filename in glob.glob(os.path.join(path, '*.txt')):
   with open(os.path.join(os.getcwd(), filename), 'r') as f: # open in readonly mode
      # do your stuff

Pipe

Or you can even use the pipe as you specified using fileinput

import fileinput
for line in fileinput.input():
    # do your stuff

And you can then use it with piping:

ls -1 | python parse.py