I'm running into issues getting C# (VS2008, Compact Framework, .NET is version 3.5 SP1) to successfully deserialize nested structs. The problem only appears in CF when I'm running on the emulator for the mobile device (I'm using the "Pocket PC 2003 Second Edition" emulator), the exact same code running on my Windows box does not have the same problem.
Here's my code:
public struct Fred
{
public string Name;
}
public struct Middle
{
public Fred[] Freds;
}
public struct Top
{
public Middle Middle;
public Fred[] Freds;
}
public static void Test()
{
Top top = new Top();
top.Middle.Freds = new Fred[2];
top.Middle.Freds[0].Name = "Fred20";
top.Middle.Freds[1].Name = "Fred21";
top.Freds = new Fred[2];
top.Freds[0].Name = "Fred10";
top.Freds[1].Name = "Fred11";
StringBuilder sb = new StringBuilder();
System.Xml.Serialization.XmlSerializer x = new System.Xml.Serialization.XmlSerializer(top.GetType());
using (StringWriter sw = new StringWriter(sb))
{
x.Serialize(sw, top);
}
string xml = sb.ToString();
string[] lines = xml.Split(new char[] { '\r', '\n' });
foreach (string line in lines)
{
Debug.WriteLine(" " + line.Trim());
}
MemoryStream ms = new MemoryStream(System.Text.Encoding.ASCII.GetBytes(xml));
StreamReader sr = new StreamReader(ms);
object o = x.Deserialize(sr);
Debug.WriteLine("Deserialized into " + o);
Top go2 = (Top)o;
if (go2.Freds == null)
Debug.WriteLine(" go2.Freds is null");
else
Debug.WriteLine(" go2.Freds[0].Name is \"" + go2.Freds[0].Name + "\"");
if (go2.Middle.Freds == null)
Debug.WriteLine(" go2.Middle.Freds is null");
else
Debug.WriteLine(" go2.Middle.Freds[0].Name is \"" + go2.Middle.Freds[0].Name + "\"");
}
When I run this, the XML it creates looks good:
<?xml version="1.0" encoding="utf-16"?>
<Top xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Middle>
<Freds>
<Fred>
<Name>Fred20</Name>
</Fred>
<Fred>
<Name>Fred21</Name>
</Fred>
</Freds>
</Middle>
<Freds>
<Fred>
<Name>Fred10</Name>
</Fred>
<Fred>
<Name>Fred11</Name>
</Fred>
</Freds>
</Top>
but C# is unable to successfully deserialize this XML - the console output is this:
Deserialized into Top
go2.Freds[0].Name is "Fred10"
go2.Middle.Freds is null
xsd has similar problems:
<?xml version="1.0" encoding="utf-8"?>
<xs:schema id="Top" xmlns="" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
<xs:element name="Top" msdata:IsDataSet="true" msdata:UseCurrentLocale="true">
<xs:complexType>
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element name="Middle">
<xs:complexType>
<xs:sequence>
<xs:element name="Freds" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence>
<xs:element name="Fred" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence>
<xs:element name="Name" type="xs:string" minOccurs="0" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:complexType>
</xs:element>
</xs:schema>
Have I just encountered a C# bug? Or am I missing something obvious?
Note: It's not a problem with using the name twice, if I create a struct named George that is identical to Fred, and change the contents of Middle to public George[] George, the problem isn't any better.
Ok, I'll give it a shot.
The problem seems to be that you're using the Compact Framework, which does not have the same serialization/deserialization capabilities as the full .NET framework. So we need some custom serialization here.
Going with the philosophy of the Compact Framework, my guess is that you also want something that performs well and has a small footprint. So I picked Protobuf for the task (which is also approximately 12 times faster than XmlSerializer)
You can install it by running this command:
Install-Package protobuf-net
Let's start with the easy way - by adding attributes to your model. Configuration without attributes comes next, as you pointed out that the original model can't/shouldn't be modified. This is only for illustration.
Decorated with the appropriate attributes, your model would look like this:
I repeat, this part is only for illustration purposes - read on to "Configuration without attributes"
[ProtoContract]
public struct Fred
{
[ProtoMember(1)]
public string Name;
}
[ProtoContract]
public struct Middle
{
[ProtoMember(1)]
public Fred[] Freds;
}
[ProtoContract]
public struct Top
{
[ProtoMember(1)]
public Middle Middle;
[ProtoMember(2)]
public Fred[] Freds;
}
The only thing to note here is the usage of numbered members, called keys. It's essentially the same thing as giving them property names in the case of JSON or XML serialization, except this is the protobuf way to do it. You simply assign a unique integer value to each member within the same class, and most of the times you're done there.
For convenience let's add a simple Builder from which we can instantiate a Top which is similar to the one in your example:
public class TopTestBuilder
{
public Top BuildDefaultTestTop()
{
var top = new Top
{
Middle = new Middle
{
Freds = new[]
{
new Fred {Name = "Fred20"},
new Fred {Name = "Fred21"}
}
},
Freds = new[]
{
new Fred {Name = "Fred10"},
new Fred {Name = "Fred11"}
}
};
return top;
}
}
We can serialize it like this:
Top topIn = new TopTestBuilder().BuildDefaultTestTop();
string serialized;
using (var stream = new MemoryStream())
{
Protobuf.Serializer.Serialize(stream, topIn);
stream.Position = 0;
var reader = new StreamReader(stream);
serialized = reader.ReadToEnd();
}
// Output: "\nDC4\n\b\nACKFred20\n\b\nACKFred21DC2\b\nACKFred10DC2\b\nACKFred11"
And deserialize it like this:
Top topOut;
using (var stream = new MemoryStream())
{
var writer = new StreamWriter(stream);
writer.Write(serialized);
writer.Flush();
stream.Position = 0;
topOut = Protobuf.Serializer.Deserialize<Top>(stream);
}
As you can see there is a little bit of plumbing for the MemoryStreams, but other than that it should look familiar to how other types of serialization work. Similarly, everything can just as well be accomplished by configuring a custom TypeModel, allowing serialization to be fully decoupled from the model.
By default, Protobuf uses the attributes to define the TypeModel and then stores it in ProtoBuf.Meta.RuntimeTypeModel.Default. This property is used when you call the static Protobuf.Serializer directly.
We can also define our own. It took some fiddling around (note to self: RTFM) to get it working but it turned out to be almost as simple:
var model = TypeModel.Create();
// The first parameter (maps to ProtoContractAttribute) is the Type to be included.
// The second parameter determines whether to apply default behavior,
// based on the attributes. Since we're not using those, this has no effect.
model.Add(typeof(Fred), false);
model.Add(typeof(Middle), false);
model.Add(typeof(Top), false);
// The newly added MetaTypes can be accessed through their respective Type indices.
// The first parameter is the unique member number, similar to ProtoMemberAttribute.
// The second parameter is the name of the member as it is declared in the class.
// When the member is a list:
// The third parameter is the Type for the items.
// The fourth parameter is the Type for the list itself.
model[typeof(Fred)].Add(1, "Name");
model[typeof(Middle)].Add(1, "Freds", typeof(Fred), typeof(Fred[]));
model[typeof(Top)].Add(1, "Middle");
model[typeof(Top)].Add(2, "Freds", typeof(Fred), typeof(Fred[]));
Now all we have to do is change one line of code for both functions:
Serialize:
Top topIn = new TopTestBuilder().BuildDefaultTestTop();
string serialized;
using (var stream = new MemoryStream())
{
model.Serialize(stream, top);
stream.Position = 0;
var reader = new StreamReader(stream);
serialized = reader.ReadToEnd();
}
Deserialize:
Top topOut;
using (var stream = new MemoryStream())
{
var writer = new StreamWriter(stream);
writer.Write(serialized);
writer.Flush();
stream.Position = 0;
topOut = (Top) _model.Deserialize(stream, null, typeof (Top));
}
And it works just the same. Perhaps add a class to keep things organized - give it two public methods Serialize and Deserialize, and a private method BuildTypeModel (to call from the constructor and store in a field on the serializer?)
Your calling code would end up looking something like this:
var serializer = new CustomProtoBufSerializer();
var serialized = serializer.Serialize(someClassInput);
SomeClass someClassOutput = serializer.Deserialize(serialized);
One thing quickly became clear though - Protobuf hasn't been as thoroughly documented and tested as most JSON and XML serializers out there. This, along with the serialization results being non-readable to humans, could be a drawback in some situations. Other than that it seems like it's fast, lightweight and compatible with many different environments.
The absence of automatic type resolution bothered me a bit, so I went looking and found something that seems pretty interesting: Protobuf T4 TypeModel Generator. I haven't been able to try it yet. If people are interested, I might do that later and update the answer with a more generic solution.
Let me know if you have any trouble getting it to work.