Skip folders to build using scons after full build

Question

I have large number of source files ~10,000 and they are scattered across several folders.

I wanted to know if there is a way to skip certain folders, I know that havent changed.

For ex, consider the following folder structure

A (Sconstruct is here)
|
->B (unchanged 1000 files)
->C (unchanged 1000 files)
->D (changed 1 file)

Once I do a complete build for the first time, I want it to compile everything (B, C, D) but when I modify a file in D (I know that), I would like to build folder D only, skip B and C and finally link them all together to form the final binary (B, C and new D).

I have been looking for quite some time now but not able to figure it out. Is it even possible? Can I specify only to look into a particular folder for changes?

Answer 1

First, I'd investigate using Decider('timestamp-match') or even building a custom Decider function. That should speed up your dependency-checking time.

But to answer your specific question, yes it is possible to not build the targets in B and C. If you don't invoke a builder for the targets in those subdirectories, you just won't build them. Just have an if that selectively chooses which env.Object() (or similar) functions to invoke.

When I fleshed out your example, I chose to have each subdirectory create a library that would be linked into the main executable, and to only invoke env.SConscript() for the directories that the user chooses. Here is one way to implement that:

A/SConstruct:

subdirs = ['B','C','D']
AddOption('--exclude', default=[], action='append', choices=subdirs)
env = Environment(EXCLUDES = GetOption('exclude'))

env.SConscript(
    dirs=[subdir for subdir in subdirs
          if subdir not in env['EXCLUDES']],
    exports='env')

env2 = env.Clone()
env2.PrependUnique(LIBPATH=subdirs,
                   LIBS=subdirs)
env2.Program('main.c')

B/SConscript:

Import('env')
env.Library('B', env.Glob('*.c'))

C/SConscript:

Import('env')
env.Library('C', env.Glob('*.c'))

D/SConscript:

Import('env')
env.Library('D', env.Glob('*.c'))

To do a global build: scons

To do a build after modifying a single file in D: scons --exclude=B --exclude=C

EDIT

Similarly, you can add a whitelist option to your SConstruct. The idea is the same: only invoke builders for certain objects.

Here is a SConstruct similar to above, but with a whitelist option:

subdirs = ['B','C','D']
AddOption('--only', default=[], action='append', choices=subdirs)
env = Environment(ONLY = GetOption('only') or subdirs)

env.SConscript(
    dirs=env['ONLY'],
    exports='env')

env2 = env.Clone()
env2.PrependUnique(LIBPATH=subdirs,
                   LIBS=subdirs)
env2.Program('main.c')

To build everything: scons

To rebuild D and relink main program: scons --only=D