Search code examples
c#stringfiledouble

How to write millions of double values into a txt file


I've made a neural network and now I need to save the results of the training process into a local file. In total, there are 7,155,264 values. I've tried with a loop like this

string weightsString = "";
string biasesString = "";

for (int l = 1; l < layers.Length; l++)
{
    for (int j = 0; j < layers[l].Length; j++)
    {
        for (int k = 0; k < layers[l - 1].Length; k++)
        {
            weightsString += weights[l][j, k] + "\n";
        }

        biasesString += biases[l][j] + "\n";
    }
}

File.WriteAllText(@"path", weightsString + "\n" + biasesString);

But it literally takes forever to go through all of the values. Is there no way to write the contents directly without having to write them in a string first?

(Weights is a double[][,] while biases is a double[][])


Solution

  • StringBuilder weightsSB = new StringBuilder();
    StringBuilder biasesSB = new StringBuilder();
    
    for (int l = 1; l < layers.Length; l++)
    {
        for (int j = 0; j < layers[l].Length; j++)
        {
            for (int k = 0; k < layers[l - 1].Length; k++)
            {
                weightsSB.Append(weights[l][j, k] + "\n");
            }
    
            biasesSB.Append(biases[l][j] + "\n");
        }
    }
    

    As suggested in the comments, I used a StringBuilder instead. Works like a charm.