Search code examples
c#csvlinqunity-game-enginecsvhelper

Cannot write external object inside of custom object to CSV using CSVHelper with ClassMap

I want to write a Vector3 object (see https://docs.unity3d.com/ScriptReference/Vector3.html) which is part of my custom class to a CSV file:

public class BehaviouralData
{
    public float ActionZ { get; set; }
    public float ActionX { get; set; }
    public int TargetBallAgentHashCode { get; set; }
    public Vector3 TargetBallLocalPosition { get; set; }
    public bool CollectBehaviouralData { get; set; }
    public DateTime ActionTime { get; set; }
    public DateTime Time { get; set; }
}

Therefore I am using a ClassMap

public sealed class Vector3Map : ClassMap<Vector3>
{
    public Vector3Map()
    {
        Map(m => m.x);
        Map(m => m.y);
        Map(m => m.z);
    }
}

Which I register before writing to a file:

...
using (var csv = new CsvWriter(writer, config))
{
    csv.Context.RegisterClassMap<Vector3Map>();
    csv.WriteRecords(data);
}
...

When I try to save the file, I get the following error message:

ArgumentException: Incorrect number of arguments supplied for call to method 'Single get_Item(Int32)'
Parameter name: property
System.Linq.Expressions.Expression.Property (System.Linq.Expressions.Expression expression, System.Reflection.PropertyInfo property) (at <1e18c5a6594041c9844bfd0b6618ee4a>:0)
CsvHelper.Expressions.ExpressionManager.CreateGetMemberExpression (System.Linq.Expressions.Expression recordExpression, CsvHelper.Configuration.ClassMap mapping, CsvHelper.Configuration.MemberMap memberMap) (at <9a1379e089274cc1baf4edcf480e4c6d>:0)
CsvHelper.Expressions.ExpressionManager.CreateGetMemberExpression (System.Linq.Expressions.Expression recordExpression, CsvHelper.Configuration.ClassMap mapping, CsvHelper.Configuration.MemberMap memberMap) (at <9a1379e089274cc1baf4edcf480e4c6d>:0)
CsvHelper.Expressions.ObjectRecordWriter.CreateWriteDelegate[T] (T record) (at <9a1379e089274cc1baf4edcf480e4c6d>:0)
CsvHelper.Expressions.RecordWriter.GetWriteDelegate[T] (T record) (at <9a1379e089274cc1baf4edcf480e4c6d>:0)
CsvHelper.Expressions.RecordWriter.Write[T] (T record) (at <9a1379e089274cc1baf4edcf480e4c6d>:0)
CsvHelper.Expressions.RecordManager.Write[T] (T record) (at <9a1379e089274cc1baf4edcf480e4c6d>:0)
CsvHelper.CsvWriter.WriteRecords[T] (System.Collections.Generic.IEnumerable`1[T] records) (at <9a1379e089274cc1baf4edcf480e4c6d>:0)
Rethrow as WriterException: An unexpected error occurred.
IWriter state:
Row: 2
Index: 0
HeaderRecord:
2
CsvHelper.CsvWriter.WriteRecords[T] (System.Collections.Generic.IEnumerable`1[T] records) (at <9a1379e089274cc1baf4edcf480e4c6d>:0)
Util.SaveDataToCSV[T] (System.String path, System.Collections.Generic.List`1[T] data) (at Assets/Scripts/Util/Util.cs:178)
BehaviourMeasurement.SaveRawBehavioralDataToCSV () (at Assets/Scripts/Measurement/BehaviourMeasurementBehaviour.cs:1021)
BehaviourMeasurementBehaviour.OnDisable () (at Assets/Scripts/Measurement/BehaviourMeasurementBehaviour.cs:122)

And the empty CSV file has the following columns:

ActionZ,ActionX,TargetBallAgentHashCode,Item,Item,magnitude,sqrMagnitude,magnitude,sqrMagnitude,CollectBehaviouralData,ActionTime,Time

As you can see not the coordinates are written to the file but other properties (e.g. magnitude) which I do not want to include. Why does CSVHelper not map the right properties into the CSV file?

Solution

  • Your basic problem is that, as derHugo noted in this answer, when CsvHelper automaps a type, it automaps the types of all property values, rather than using any class maps that might have been previously registered for those types. Specifically in the case of Vector3 it automaps its properties instead of the fields x, y and z, eventually leading to a crash.[1].

    So, what are your workarounds? Other than manually mapping the x, y and z fields manually as suggested in derHugo's answer, you have a couple of additional options.

    Firstly, you could automap your types containing Vector3 and fix the references afterwards. To do this, define the following generic ClassMap<T> and associated extension methods:

    public class CustomAutoMap<TClass> : ClassMap<TClass>
{
    // Automaps the given type then fixes up references to Vector3 (if any).
    public CustomAutoMap()
    {
        AutoMap(CultureInfo.InvariantCulture);
        // Now fix the Vector3 references.
        this.UpdateReferences<Vector3Map, Vector3>();
    }
}

public static partial class CsvExtensions
{
    // Update autoampped reference maps to a different map.
    // Note this should only be done within a ClassMap constructor itself.
    public static ClassMap UpdateReferences<TReferenceMap, TReferenceClass>(this ClassMap classMap, params object[] constructorArgs) where TReferenceMap : ClassMap<TReferenceClass>
    {
        // TODO: nested references of references.
        for (int i = 0, count = classMap.ReferenceMaps.Count; i < count; i++)
            if (classMap.ReferenceMaps[i].Data.Member.MemberType() == typeof(TReferenceClass))
                classMap.UpdateReference<TReferenceMap, TReferenceClass>(i, constructorArgs);
        return classMap;
    }
    
    public static MemberReferenceMap UpdateReference<TReferenceMap, TReferenceClass>(this ClassMap classMap, int index, params object[] constructorArgs) where TReferenceMap : ClassMap<TReferenceClass>
    {
        if (index < 0 || index >= classMap.ReferenceMaps.Count)
            throw new ArgumentException(nameof(index));
        var oldRef = classMap.ReferenceMaps[index];
        classMap.ReferenceMaps.RemoveAt(index);
        var newRef = classMap.References(typeof(TReferenceMap), oldRef.Data.Member, constructorArgs);
        if (oldRef.Data.Prefix != null)
            newRef.Prefix(oldRef.Data.Prefix);
        classMap.ReferenceMaps.Swap(index, classMap.ReferenceMaps.IndexOfWithHint(newRef, classMap.ReferenceMaps.Count - 1));
        return newRef;
    }
    
    static Type? MemberType(this MemberInfo m) =>
        m switch
        {
            PropertyInfo p => p.PropertyType,
            FieldInfo f => f.FieldType,
            _ => null,
        };

    static int IndexOfWithHint<T>(this IList<T> list, T item, int hint) where T : class
    {
        if (hint >= 0 && hint < list.Count && list[hint] == item)
            return hint;
        return list.IndexOf(item);
    }
                                    
    static void Swap<T>(this IList<T> list, int i, int j)
    {
        if (i != j)
        {
            T temp = list[i];
            list[i] = list[j];
            list[j] = temp;
        }
    }
}

    And now serialize your BehaviouralData as follows:

    var config = new CsvConfiguration(CultureInfo.InvariantCulture);
using (var csv = new CsvWriter(writer, config))
{
    csv.Context.RegisterClassMap<CustomAutoMap<BehaviouralData>>();
    csv.WriteRecords(data);
}

    Which results in:

    ActionZ,ActionX,TargetBallAgentHashCode,CollectBehaviouralData,ActionTime,Time,x,y,z
0,0,1,False,01/01/0001 00:00:00,01/01/0001 00:00:00,1.1,1.1,1.1
0,0,2,False,01/01/0001 00:00:00,01/01/0001 00:00:00,2.1,2.1,2.1
0,0,3,False,01/01/0001 00:00:00,01/01/0001 00:00:00,3.1,3.1,3.1

    If you would like the x, y and z fields to have some prefix, add [HeaderPrefix("prefix")] to TargetBallLocalPosition like so:

    public class BehaviouralData
{
    [CsvHelper.Configuration.Attributes.HeaderPrefix("TargetBallLocation_")]
    public Vector3 TargetBallLocalPosition { get; set; }
    // Remainder unchanged

    And you will get instead:

    ActionZ,ActionX,TargetBallAgentHashCode,CollectBehaviouralData,ActionTime,Time,TargetBallLocation_x,TargetBallLocation_y,TargetBallLocation_z
0,0,1,False,01/01/0001 00:00:00,01/01/0001 00:00:00,1.1,1.1,1.1
0,0,2,False,01/01/0001 00:00:00,01/01/0001 00:00:00,2.1,2.1,2.1
0,0,3,False,01/01/0001 00:00:00,01/01/0001 00:00:00,3.1,3.1,3.1

    If you need more manual control over the formatting of BehaviouralData you will need to create a manual class map. When you do, use the method ClassMap<TClass>.References<TClassMap>(Expression<Func<TClass, object>> expression, params object[]) rather than ClassMap<TClass>.Map() to reference the contents of Vector3 using your Vector3Map:

    public class BehaviouralDataMapExplicit : ClassMap<BehaviouralData>
{
    public BehaviouralDataMapExplicit()
    {
        Map(m => m.ActionZ);
        Map(m => m.ActionX);
        Map(m => m.TargetBallAgentHashCode);
        References<Vector3Map>(m => m.TargetBallLocalPosition)
            .Prefix("TargetBallLocation_");
        Map(m => m.CollectBehaviouralData);
        Map(m => m.ActionTime);
        Map(m => m.Time);
    }
}

    Demo fiddle here.

    [1] CsvHelper seems to have a bug here. Specifically, for classes that do not implement IEnumerable it attempts to automap indexed properties, resulting in the exception you see. This is a straightforward bug in CsvHelper, it should check that PropertyInfo.GetIndexParameters().Length == 0 before automapping any property discovered by reflection. But even if CsvHelper fixed this bug, you still wouldn't get what you want, which is the three fields of Vector3, not its properties.