Search code examples
c#binary-serialization

How to Convert Primitive[] to byte[]


For Serialization of Primitive Array, i'am wondering how to convert a Primitive[] to his corresponding byte[]. (ie an int[128] to a byte[512], or a ushort[] to a byte[]...) The destination can be a Memory Stream, a network message, a file, anything. The goal is performance (Serialization & Deserialization time), to be able to write with some streams a byte[] in one shot instead of loop'ing' through all values, or allocate using some converter.

Some already solution explored:

Regular Loop to write/read

//array = any int[];
myStreamWriter.WriteInt32(array.Length);
for(int i = 0; i < array.Length; ++i)
   myStreamWriter.WriteInt32(array[i]);

This solution works for Serialization and Deserialization And is like 100 times faster than using Standard System.Runtime.Serialization combined with a BinaryFormater to Serialize/Deserialize a single int, or a couple of them.

But this solution becomes slower if array.Length contains more than 200/300 values (for Int32).

Cast?

Seems C# can't directly cast a Int[] to a byte[], or a bool[] to a byte[].

BitConverter.Getbytes()

This solution works, but it allocates a new byte[] at each call of the loop through my int[]. Performances are of course horrible

Marshal.Copy

Yup, this solution works too, but same problem as previous BitConverter one.

C++ hack

Because direct cast is not allowed in C#, i tryed some C++ hack after seeing into memory that array length is stored 4 bytes before array data starts

ARRAYCAST_API void Cast(int* input, unsigned char** output)
{
   // get the address of the input (this is a pointer to the data)
   int* count = input;
   // the size of the buffer is located just before the data (4 bytes before as this is an int)
   count--;
   // multiply the number of elements by 4 as an int is 4 bytes
   *count = *count * 4;
   // set the address of the byte array
   *output = (unsigned char*)(input);
}

and the C# that call:

byte[] arrayB = null;
int[] arrayI = new int[128];
for (int i = 0; i < 128; ++i)
   arrayI[i] = i;

// delegate call
fptr(arrayI, out arrayB);

I successfully retrieve my int[128] into C++, switch the array length, and affecting the right adress to my 'output' var, but C# is only retrieving a byte[1] as return. It seems that i can't hack a managed variable like that so easily.

So i really start to think that all theses casts i want to achieve are just impossible in C# (int[] -> byte[], bool[] -> byte[], double[] -> byte[]...) without Allocating/copying...

What am-i missing?


Solution

  • How about using Buffer.BlockCopy?

    // serialize
    var intArray = new[] { 1, 2, 3, 4, 5, 6, 7, 8 };
    var byteArray = new byte[intArray.Length * 4];
    Buffer.BlockCopy(intArray, 0, byteArray, 0, byteArray.Length);
    
    // deserialize and test
    var intArray2 = new int[byteArray.Length / 4];
    Buffer.BlockCopy(byteArray, 0, intArray2, 0, byteArray.Length);
    Console.WriteLine(intArray.SequenceEqual(intArray2));    // true
    

    Note that BlockCopy is still allocating/copying behind the scenes. I'm fairly sure that this is unavoidable in managed code, and BlockCopy is probably about as good as it gets for this.