i've a problem about saving 2d array's value to hardisk. i build 5000x5000 item similarity table (float) and i want to store it because it will be continuesly used. i've tried save them to .txt file by format index1,index2,value+\n and they're growing to 250MB of size (its approx 25million of rows). is there any ways to reduce its size? i would be glad if anyone mind to help me
Instead of storing index of each cell i.e. index1 and index2, we can write the size of the matrix at the beginning of the file itself using 2 integers(number of rows & number of columns). Use this information while reading, to know when a row of the matrix is read completely.
Instead of writing string value of float, we can use Float.floatToIntBits(float)
method to get IEEE-754 representation as an integer and serialize the integer as 4 bytes
This will reduce the size of a 5000 x 5000 matrix to ~ 100MB