Python os.walk/fnmatch.filter does not find file when in file's current directory

Question

I'm trying to recursively walk through a directory and find files that match a certain pattern. The relevant snippet of my code is:

import sys, os, xlrd, fnmatch

for root, dirnames, filenames in os.walk('/myfilepath/'):

    for dir in dirnames:
        os.chdir(os.path.join(root, dir))

        for filename in fnmatch.filter(filenames, 'filepattern*'):
            print os.path.abspath(filename)
            print os.getcwd()
            print filename

            wb = xlrd.open_workbook(filename)

My print lines demonstrate that os.getcwd() is equal to the directory of filename, so to me it seems like the file should be found, but IOError: [Errno 2] No such file or directory is thrown for wb = xlrd.open_workbook(filename) when the first pattern is matched.

Answer 1

The dirnames returned from os.walk don't represent the directories in which filenames exist. Rather, root represents the directory in which filenames exist. For your application you can effectively ignore the directories return.

Try something like this:

import os
import fnmatch

for root, _, filenames in os.walk('/tmp'):
    print root, filenames
    for filename in fnmatch.filter(filenames, '*.py'):
        filename = os.path.join(root, filename)

        # `filename` now unambiguously refers to a file that
        # exists. Open it, delete it, xlrd.open it, whatever.
        # For example:
        if os.access(filename, os.R_OK):
            print "%s can be accessed"  % filename
        else:
            print "%s cannot be accessed"  % filename

Aside: It is probably not safe to call os.chdir() inside an os.walk() iteration. This is especially true if the parameter to os.walk() is relative.