My team is migrating from Clickhouse to Azure Data Explorer (ADX). We are currently experiencing difficulties to query our data from ADX: the queried values are correct, but the data are read as a string rather than as an array of floats.
Here is an example string:
mydummystring='[1.0,2.0,3.0][4.0,5.0,6.0][6.0,7.0,8.0]'
In order to convert this string to a numpy array, I found this workaround based on list comprehension (inspired by this SO post):
import numpy as np
mynumpyarray = np.array([np.array(x) for x in eval('['+mydummystring.replace('][', '],[')+']')])
Is there a better (safer?) way to achieve this conversion? I know that it would be better to read the data correctly in the first place, but for now I am looking for a robust way to convert the output string to actual numbers.
You can use ast.literal_eval, which only parses Python literal structures and does not run arbitrary code.
from ast import literal_eval
s = '[1.0,2.0,3.0][4.0,5.0,6.0][6.0,7.0,8.0]'
np_arr = np.array([np.array(x) for x in literal_eval('['+s.replace('][', '],[')+']')])
Note that a list comprehension is not necessary to create the NumPy array.
np.array(literal_eval('['+s.replace('][', '],[')+']'))