Get UTF8.GetBytes from very large stringbuilder

Question

I have a StringBuilder of length 1,539,121,968. When calling StringBuilder .ToString() on it fails with OutOfMemoryException. I tried creating a char array but was not allowed to create such big array.

I need to store it a byte array in UTF8 format. Is it possible?

Answer 1

I'd suggest looking at the documentation for streams. As this might help.

Another way to approach it would be to split it up. As for your last comment stating that you wish to store it as a ByteArray with UTF8 you'd need a char[] as else you'd lose your encoding. I'd reccomend splitting it into many smaller strings (or char[]s) stored in separate objects that can easily be reconstructed. Something like this might suffice, create many StringSlices:

public class StringSlice()
{
     public Str {get;}
     public Index {get;}
     public StringSlice(string str, int index)
         {
              this.Str = str;
              this.Index = index;
         }

     public static List<string> ReconstructString(IEnumerable<StringSlice> parts)
         {
              //Sort input by index return list with new strings in order. Probably have to use a buffer on the disc so as not to breach 2GB obj limit.
         }
}

In essence what you would be doing here is similar to the way internet packets are split and reconstructed. I'm not entirely sure if I've answered your question but hopefully this comes some way to helping.