We are currently trying to handle a rather large file through our BizTalk process and are constantly gettingOutOfMemoryExceptions when processing. I have a custom disassembler that processes each record individually as to not read the entire message in memory. The environment is currently in dev and so SQL and BizTalk run on the same machine. The machine has 16 Gb of memory but BizTalk is only 32 bit architecture.
Are there any Host settings I can change to allow the file to be processed from start to finish?
Any reason you can't run 64bit BizTalk? There are Host Settings you can fiddle with, but they're related to Throttling. An OutOfMemoryException is a hard error which BizTalk really has little to no control over.
What you're describing is definitely doable. I've done it.
First look into the basics of your component. Are your message instances being properly de-referenced and all?
How are you submiting to the MessageBox? If it's through the normal Disassembler API, then them messages are going to sit in memory for a while. Using a VirtualStream can help with that.