Search code examples
pythonjsonnumpyserializationjson-deserialization

np.float64 and dtype=object can not be serialized


i have the below posted list and it contais lists, so it is a list of lists, and i have a webservice that returns return jsonify(resultsDict). the problem i am facing is when i run the App i recieve either of the error messages posted below. as shown in the code below, i tried to set dtype=np.float64 and dtype=object, but each of them generates an error associated to it as shown below in the code. please let me know how to fix it

attempt_1

resultsDict={
    "extras": {
        "pvTreatment":np.array(pvTreatment,dtype=np.float64).tolist(),
        ...
        ...
        ...
    }
}
    
**error associated**:

    ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (7,) + inhomogeneous part.

attempt_2

resultsDict = {
    "extras": {
        "pvTreatment":np.array(pvTreatment,dtype=object).tolist(),
        ...
        ...
        ...
    }
}   
    
**error associated**:

    raise TypeError(f'Object of type {o.__class__.__name__} '
    TypeError: Object of type float32 is not JSON serializable

Solution

  • If I try to make an array with two arrays that differ in length I get your kind of error:

    In [186]: np.array([np.ones((3)), np.zeros((4))],float)
    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    Input In [186], in <cell line: 1>()
    ----> 1 np.array([np.ones((3)), np.zeros((4))],float)
    
    ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
    

    I can make an object dtype array:

    In [187]: np.array([np.ones((3)), np.zeros((4))],object)
    Out[187]: array([array([1., 1., 1.]), array([0., 0., 0., 0.])], dtype=object)
    

    But when I use tolist I get a list with two arrays:

    In [188]: np.array([np.ones((3)), np.zeros((4))],object).tolist()
    Out[188]: [array([1., 1., 1.]), array([0., 0., 0., 0.])]
    

    Arrays are not JSON serializable.

    If the inner arrays were changed to lists, we'd get a list of lists:

    In [191]: np.array([np.ones((3)).tolist(), np.zeros((4)).tolist()],object).tolist()
    Out[191]: [[1.0, 1.0, 1.0], [0.0, 0.0, 0.0, 0.0]]
    

    For this kind of thing, avoid arrays - both for the outer list and the inner ones.