Search code examples
c#xmlxmltextreader

read large file with XmlTextReader in C#


I am reading a large file with XmlTextReader, it takes hours to read and do some processing, during the reading and processing , what will happen if this file is changed or deleted from file server ?


Solution

  • If the file is successfully edited or deleted (i.e. you didn't have it locked for reading), your app will crash (assuming your app thought it had the file open). More likely, though: the other process will not be able to get access to the file to edit/delete it.

    If it is the processing that takes time, then consider loading the data eagerly. If, however, it is slow due to sheer volume... well, there isn't much can be done about that (assuming the volume * processing-time is the bottleneck; if it is just because of the size of the xml file, an alternative format might help). If IO is your bottleneck, you might want to try experimenting with compression (either at the file-system level, or explicitly running your xml through gzip when writing) - it might (not guaranteed) be that for a spinning platter, reducing the physical reads (trading CPU time decompressing it) speeds things up a little. Probably not the case for SSD or high-performance disks, though.