Search code examples
pythondataframenumpydata-sciencelarge-data

Saving large numpy arrays using Python

I have a dataframe with only two columns, one is text and one is with numpy array values, which can have more than 170000 elements. The problem is how to save this large numpy arrays as files, so I can load them back later. When trying to save as CVS, it just saves it as a short string, loosing the real values:

[-8.0152120e-07  2.8887976e-05  3.3898741e-05 ... -1.3205040e-01
 -9.4032057e-02  0.0000000e+00]

I thought maybe to save as text one by one, but is there any elegant way to do this?

Solution

  • As suggested, you can save it as a numpy file using np.save. Later you can load it using np.load.