Search code examples
pythonnumpyfilereadlines

Reading a non-uniform file into a single array in python

I am trying to read a single non csv file into a single array in python. I have tried np.loadtxt, however the data has columns of different widths and even skipping those, loadtxt returns arrays of line values that cannot be appended together. The data file in question

Solution

  • A simple read should be sufficient for now:

    with open('data.txt') as fp:
    next(fp)  # skip first line (length)
    m = np.array(fp.read().strip().split()).astype(float)

    Output:

    >>> m
array([1.001000e-06, 1.545000e-06, 2.399000e-06, ..., 9.999984e-01,
       9.999984e-01, 1.000000e+00])

>>> m.dtype
dtype('float64')

>>> m.shape
(18050,)

    Note the first line indicates 50 and 350. The length of the array is 18050.
    This is the sum of 50 + 50 x 350. How to interpret this lines?