Bash, excluding files in for loop and quoting

I have files in a directory that I want to process in a for loop while excluding some (previously processed) files. I'm using !() to exclude the files but how do I quote my variables when the files to exclude have space in filenames and I read the filenames from a file?

I can use !() without problems when I state the files to exclude explicitly:

$ mkdir test
$ cd test
$ touch this.txt and\ this.txt not\ this.txt nor\ this.txt

$ for THEFILE in !(not this.txt|nor this.txt); do 
    echo $THEFILE
  done

and this.txt
this.txt

But when I introduce the exclude file and there are files with space in filenames, I have a hard time of quoting my variables correctly. Here is output without quotes:

$ cd ..
$ SRCDIR=test
$ EXCLUDEFILE=excludes.txt
$ cat > excludes.txt
not this.txt|nor this.txt
^D

$ for THEFILE in $SRCDIR/!($(cat $EXCLUDEFILE)); do
  echo $THEFILE
done

test/!(not
this.txt|nor
this.txt)

Here is one with quotes:

$ for THEFILE in $SRCDIR/!("$(cat $EXCLUDEFILE)"); do
  echo $THEFILE
done

test/and this.txt
test/nor this.txt
test/not this.txt
test/this too.txt
test/this.txt

I've tried a few other variations but without success. Please SO, educate me.

Solution

A simple and portable workaround would be

filter=$(cat excludes.txt)
for THEFILE in $SRCDIR/*; do
    case "|$filter|" in *"|$THEFILE$|"*) continue;; esac
    echo "$THEFILE"
done

I would perhaps use newline as the separator in excludes.txt instead, although that makes the code a little iffier. For completely robust handling, use a literal zero byte, or (if you never need to handle directories) a slash.