Search code examples
pythonnumpynangenfromtxt

numpy.genfromtxt give nan for complex numbers with minus signs?


I have some complex numbers in a file, written by np.savetxt():

(8.67272e-09+-1.64817e-07j)
(2.31263e-08+1.11916e-07j)
(9.73642e-08+-7.98195e-08j)
(1.05448e-07+7.00151e-08j)

This is in a file "test.txt". when I use `np.genfromtxt('test.txt', dtype=complex), I get:

                nan +0.00000000e+00j,
     2.31263000e-08 +1.11916000e-07j,
                nan +0.00000000e+00j,
     1.05448000e-07 +7.00151000e-08j,

Is this a bug, or is there something I can do to avoid getting nan from negative numbers?


Solution

  • This is a bug that has been reported on the numpy github repository. The problem is that savetxt writes a string that includes an extraneuous '+' when the imaginary part is negative. The '+' is extraneous from the Python point of view:

    In [95]: complex('1+-2j')
    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    <ipython-input-95-56afbb08ca8f> in <module>()
    ----> 1 complex('1+-2j')
    
    ValueError: complex() arg is a malformed string
    

    Note that 1+-2j is a valid Python expression. This suggests using a converter in genfromtxt that evaluates the expression.

    For example, here's a complex array a:

    In [109]: a
    Out[109]: array([1.0-1.j , 2.0+2.5j, 1.0-3.j , 4.5+0.j ])
    

    Save a to foo.txt:

    In [110]: np.savetxt('foo.txt', a, fmt='%.2e')
    
    In [111]: !cat foo.txt
     (1.00e+00+-1.00e+00j)
     (2.00e+00+2.50e+00j)
     (1.00e+00+-3.00e+00j)
     (4.50e+00+0.00e+00j)
    

    Read the data back using genfromtxt. For the converter, I'll use ast.literal_eval:

    In [112]: import ast
    
    In [113]: np.genfromtxt('foo.txt', dtype=np.complex128, converters={0: lambda s: ast.literal_eval(s.decode())})
    Out[113]: array([1.0-1.j , 2.0+2.5j, 1.0-3.j , 4.5+0.j ])
    

    Alternatively, you could use a converter that replaces occurrences of '+-' with '-':

    In [117]: np.genfromtxt('foo.txt', dtype=np.complex128, converters={0: lambda s: complex(s.decode().replace('+-', '-'))})
    Out[117]: array([1.0-1.j , 2.0+2.5j, 1.0-3.j , 4.5+0.j ])