Search code examples
c#.netlistalgorithmvector

Are there any existed API to split IEnumerable<T> to many Vector<T> in CSharp？

Now I have a big data structure（Array，List，or IEnumerable）and I want to split them to many Vector such as Vector64，Vector128 or 512，is there any existed vectorized API in .NET to complete this rather than writing a loop？

We assume that we don‘t need to care how to change length of IEnumerable's ‘tail’ after splitting it.

Solution

  • IEnumerable<T> is a highly abstract API. It may represent data in an array, a hashset, or data generated on demand. This makes the interface very flexible and easy to use, but it may affect performance in some situations.

    The Vector<T> classes are much closer to the hardware and are intended to be compiled to SIMD instructions whenever possible. For this to be possible all the items must be sequential in memory. I.e. arrays or Span<T>. An IEnumerable cannot guarantee any particular memory layout.

    So if you want to work on your data using vector operations you may need to convert it to an array first, for example with .ToArray(), or make sure it is stored in an array in the first place.

    If you just want to split your data into smaller chunks, see Enumerable.Chunk.