I have created a software package that produces a directory full of results. I would like to test the results from some standard input files. The directories should be somewhat similar on each run except for a limited number of things that change: traces of the current date/time, a run-specific UUID, current Subversion revision number. This is complicated by the fact that sometimes these variables are inserted into the body of text files, and sometimes into their file names. And sometimes the text files have been gzip-compressed. This all complicates using diff to just compare the directories.
Ideally I would like to be able to take one "golden" run of the application, edit the output files to replace the variables with some sort of regex syntax, and run some application to compare the directories, accepting things that match the regex syntax. Is there a testing framework that could do this for me easily? Best would be if it had a command-line interface, second best would be a Python interface. If not, I guess I will probably try to roll something up myself.
I sounds like you should take a look at TextTest. I haven't used it personally but based on what I know it ought to fit into your needs pretty well.