Best architecture for handling file system changes?

Question

Here is the scenario:

I'm writing an app that will watch for any changes in a specific directory. This directory will be flooded with thousands of files a minute each with an "almost" unique GUID. The file format is this:

GUID.dat where GUID == xxxxxxxxxxxxxxxxxxxxxxxxxxxxx (the internal contents aren't relevant, but it's just text data)

My app will be a form that has one single text box that shows all the files that are being added and deleted in real time. Every time a new file comes in I have to update the textbox with this file, BUT I must first make sure that this semi-unique GUID is really unique, if it is, update the textbox with this new file.

When a file is removed from that directory, make sure it exists, then delete it, update textbox accordingly.

The problem is that I've been using the .NET filewatcher and it seems that there is an internal buffer that gets blown up every time the (buffersize + 1)-th file comes in. I also tried to keep an internal List in my app, and just add every single file that comes in, but do the unique-GUID check later, but no dice.

Answer 1

A couple of things that I have in my head:

• If the guid is not unique, would it not overwrite the file with the same name, or is the check based on a lookup which does some external action (e.g. check the archive)? (i.e. is this a YAGNI moment?)
• I've used FileSystemWatcher before with pretty good success, can you give us some ideas as to how your actually doing things?
• When you say "no dice" when working with your custom list, what was the problem? And how were you checking for file system changes without FileSystemWatcher?!

Sorry no answer as yet, just would like to know more about the problem :)