Use a Glob() in VariantDir() einvironment to find files recursively in Python?

Question

How can one make recursive Glob() in a VariantDir() environment in Python?

The answer from the question <Use a Glob() to find files recursively in Python?> will not work, because you need use Glob() to get a list of files that is aware of VariantDir() environment.

So you need something like:

import fnmatch
import os

matches = []
for root, dirnames, filenames in os.walk('src'):
  for filename in fnmatch.filter(filenames, '*.c'):
    matches.append(os.path.join(root, filename))

matches = Glob(matches)

Will this work?

Answer 1

Your approach would work with a minor tweak as follows:

import fnmatch
import os

def RecursiveGlob(pathname)
    matches = []
    for root, dirnames, filenames in os.walk(pathname):
        for filename in fnmatch.filter(filenames, '*.c'):
            matches.append(File(os.path.join(root, filename)))

    return matches

Notice that I converted it to a File(), since the SCons Glob() function returns Nodes if the "strings" parameter is false.

To be able to handle the VariantDir, etc and to better integrate the functionality with the existing SCons Glob() functionality, you could actually incorporate a call to the existing Glob() function, like this:

# Notice the signature is similar to the SCons Glob() signature,
# look at scons-2.1.0/engine/SCons/Node/FS.py line 1403
def RecursiveGlob(pattern, ondisk=True, source=True, strings=False):
    matches = []
    # Instead of using os.getcwd() consider passing-in a path
    for root, dirnames, filenames in os.walk(os.getcwd()):
        cwd = Dir(root)
        # Glob() returns a list, so using extend() instead of append()
        # The cwd param isnt documented, (look at the code) but its 
        # how you tell SCons what directory to look in.
        matches.extend(Glob(pattern, ondisk, source, strings, cwd))

    return matches

You could take it one step further and do the following:

def MyGlob(pattern, ondisk=True, source=True, strings=False, recursive=False):
    if not recursive:
        return Glob(pattern, ondisk, source, strings)

    matches = []
    # Instead of using os.getcwd() consider passing-in a path
    for root, dirnames, filenames in os.walk(os.getcwd()):
        cwd = Dir(root)
        # Glob() returns a list, so using extend() instead of append()
        # The cwd param isnt documented, (look at the code) but its 
        # how you tell SCons what directory to look in.
        matches.extend(Glob(pattern, ondisk, source, strings, cwd))

    return matches