How to write xml data into multiple files by checking file size

Question

I'm creating a xml sitemap! There is a limit of 50k urls or 10MB size limit. Is there a way to check the size of file and write to a new xml file when a size limit of 10MB is reached? I'm using XmlTextWriter(C#) to create xml files

Answer 1

If you are using XmlTextWriter, you could pass in a custom 'System.IO.Stream' to handle the 10MB size limit. Your custom Stream would handle the monitoring of file size and creating a new file at a certain interval.

So you might design toward something like:

var folder = @"C:\temp";
var maxSizeInBytes = 10240;
var stream = new SegmentedStream(folder, maxSizeInBytes);

var xmlWriter = new XmlTextWriter(stream, Encoding.UTF8);

int maxRecords = 50000;
int i = 0;
foreach (var url in UrlList){
    i++;

    if (i == maxRecords){
        stream.StartNewFile(); // custom method on your SegmentedStream class
        i == 0;
    }

    // TODO: write URL using xmlWriter
    // explicitly calling Flush on the xmlWriter, may be appropriate
}

I don't know the implementation of the XmlTextWriter very well, but you may have to be clever so that you don't end up with one XML node spanning two files.

Update: A better approach may be to just write the whole XML file. And then as a post-process split them up.