Search code examples
pythonpandasnumpygoogle-colaboratory

pd.Series.to_list() changing dtype


When I am programming on colab, I keep running into this issue:

Here is my df:

0                1
0    [2.7436598593417045e-05, 3.731542193080655e-05]
1    [8.279973504084787e-05, 2.145002145002145e-05]
2    [0.00022534319714215346, 0.0002031172259231674]
3    [3.239841667031943e-05, 2.7771297808289177e-05]
4    [0.00011311134356928321, 9.428422928088026e-05]

I want to get the data from df[1] into a list of lists so I can feed it into my model. To do so, I run:

df[1].to_list()

and i get:

['[2.7436598593417045e-05, 3.731542193080655e-05]',
 '[8.279973504084787e-05, 2.145002145002145e-05]',
 '[0.00022534319714215346, 0.00020311722592316746]',
 '[3.239841667031943e-05, 2.7771297808289177e-05]',
 '[0.00011311134356928321, 9.428422928088026e-05]'] 

which is a list of strings which I cannot use to feed into the model. I use this code all the time locally and it works fine, but on colab I get this result. Any ideas? The result I want is:

[[2.7436598593417045e-05, 3.731542193080655e-05],
 [8.279973504084787e-05, 2.145002145002145e-05],
 [0.00022534319714215346, 0.00020311722592316746],
 [3.239841667031943e-05, 2.7771297808289177e-05],
 [0.00011311134356928321, 9.428422928088026e-05]] 


Solution

  • Try ast.literal_eval

    from ast import literal_eval
    
    df[1].map(literal_eval).to_list()
    
    [[2.7436598593417045e-05, 3.731542193080655e-05],
     [8.279973504084787e-05, 2.145002145002145e-05],
     [0.00022534319714215346, 0.00020311722592316746],
     [3.239841667031943e-05, 2.7771297808289177e-05],
     [0.00011311134356928321, 9.428422928088026e-05]]