Using XmlSerializer with Azure ShareFileClient creates files of incorrect size

Question

I'm trying to write XML files to an azure file share using XmlSerializer but my files are always as big as my set MaxSize.

I have an Azure Function that calls the following function that writes imported RowSet-data to an XML file and then uploads that XML to an Azure File Storage. The XML file is usually around 100-250 MiB large which means i want to use the OpenWrite function that's included in the Azure.Storage.Files.Shares namespace.

But in order to use OpenWrite you need to specify a MaxSize in ShareFileOpenWriteOptions, but this is where my troubles begin. The files that are created are always the same size as MaxSize, even when it's an empty XML. So if i give MaxSize the value "1024 * 1024 * 4" my serialized XML files will always come out as 4MiB, if i set MaxSize to "1024 * 1024 * 500" the files always come out as 500MiB.

public void WriteXmlToFile(<MyRowSetType> rs, string filePath, string fileName)
{
   try
   {
        ShareFileClient fileClient =  _shareClient.GetDirectoryClient(filePath).GetFileClient(fileName);

        ShareFileOpenWriteOptions uploadOptions = new ShareFileOpenWriteOptions
        {
            MaxSize = 1024 * 1024 * 100
        };

        using (var stream = fileClient.OpenWrite(true, 0, uploadOptions))
        {
            var serializer = new XmlSerializer(typeof(<MyRowSetType>));
            serializer.Serialize(stream, rs);
        }
    }
    catch (Exception ex)
    {
        _logger.LogError($"Error writing XML to Azure File Storage, FileName: {fileName}, FilePath: {filePath}, Exception: {ex}");
        throw;
    }
}

I don't know if I've misunderstood what MaxSize does, i thought it set the largest possible file size, not the actual size.

If i instead load the (in this case empty) XML into memory first i get the correct file size (170 bytes), but again, i Don't want to be loading large XML-files into memory before uploading them.

var serializer = new XmlSerializer(typeof(<MyRowSetType>));

using (var writer = new StringWriter())
{
    serializer.Serialize(writer, rs);
    var xml = writer.ToString();
    XmlDocument doc2 = new XmlDocument();
    doc2.LoadXml(xml);
    var len = doc2.OuterXml.Length;
    ShareFileOpenWriteOptions uploadOptions = new ShareFileOpenWriteOptions
    {
        MaxSize = len
    };

    using (var stream = fileClient.OpenWrite(true, 0, uploadOptions))
    {
        stream.Write(Encoding.UTF8.GetBytes(doc2.OuterXml), 0, len);
    }
}

So I'm looking for help how i would go about serializing XML files without loading them into memory first. Maybe I'm just overcomplicating things?

Answer 1

you are specifying a MaxSize of 1024 * 1024 * 100, which is 100 MB. This means that even if your serialized XML data is smaller than 100 MB, the file will be created with the specified maximum size.

To address this issue and create XML files with the correct size without loading them into memory first, you can follow a different approach. Instead of relying on the MaxSize property, you can serialize your data to a temporary stream (like a MemoryStream) and then copy the contents to the Azure file stream.

but if you want just write XML files you can use the condition:

public void WriteXmlToFile(<MyRowSetType> rs, string filePath, string fileName, long maxSize)
{
    try
    {
        ShareFileClient fileClient = _shareClient.GetDirectoryClient(filePath).GetFileClient(fileName);

        using (var tempMemoryStream = new MemoryStream())
        {
            var serializer = new XmlSerializer(typeof(<MyRowSetType>));
            serializer.Serialize(tempMemoryStream, rs);

            tempMemoryStream.Seek(0, SeekOrigin.Begin);

            if (tempMemoryStream.Length > maxSize)
            {
                // Handle the case where the data exceeds MaxSize
                throw new InvalidOperationException("Data size exceeds the specified MaxSize.");
            }
      
            ShareFileOpenWriteOptions uploadOptions = new ShareFileOpenWriteOptions
            {
                MaxSize = tempMemoryStream.Length
            };

            using (var fileStream = fileClient.OpenWrite(true, 0, uploadOptions))
            {
                tempMemoryStream.CopyTo(fileStream);
            }
        }
    }
    catch (Exception ex)
    {
        
        throw;
    }
}