How to store and handle large arrays in unity

Question

In my unity project, I have a very large array of arrays of integers. It contains around 65 thousand arrays each of length 4. When I attempt to use this in my program, it crashes and unity shuts down.

I want to be able to quickly get the nth array within the array, and don't need to be able to edit it or anything. What is the best way to store and handle the data?

Answer 1

I sort of found a solution. I put all the information as lines of numbers in a .txt file, which actually only ended up being around 370KB big, and then used the following script to transfer this into an int[][].

using UnityEngine;
using System.IO;

public static class TranpositionTable
{
    public static string[] data;
    public static string[][] seperatedData;
    public static int[][] intData;

    static TranpositionTable()
    {
        string path = Application.dataPath + "/Scripts/TranspositionTable/table.txt";
        data = File.ReadAllText(path).Split("\n");
        seperatedData = new string[data.Length][];
        for (int i = 0; i < data.Length; i++)
        {
            seperatedData[i] = data[i].Split(",");
        }
        intData = new int[seperatedData.Length][];
        for (int i = 0; i < seperatedData.Length; i++)
        {
            if (seperatedData[i][0] == "n")
            {
                intData[i] = null;
            }
            else
            {
                intData[i] = new int[4];
                for (int j = 0; j < seperatedData[i].Length; j++)
                {
                    intData[i][j] = int.Parse(seperatedData[i][j]);
                }
            }
        }
    }

}

This is then referenced in other scripts with

using static TranpositionTable;

I thought that this would be slower then just having an int[][] with all the numbers in it, but there haven't been any performance problems, so I guess its fine.