When I try to read a space separated file using genfromtxt and using a converter function to convert numbers with a comma as decimal separator, I get a type error. It seems there is something wrong with my converter function. However, when I use it on a single value, it does work properly.
This is my code (I'm using Matplotlib/Pylab):
t = dtype([('Date', 'U12'), ('Time', 'U10'), ('Cond', 'f4'), ('Temp', 'f4')])
conv = lambda valstr: float(valstr.replace(',','.'))
c = {2:conv, 3:conv}
data = genfromtxt('Example.csv', dtype = t,
skip_header=1, delimiter = ' ', converters = c)
The data looks like this:
Date Time Cond Temp
11-10-2012 00:00:14 5,430583 29,5107
11-10-2012 00:00:15 5,431812 29,45066
11-10-2012 00:00:16 5,435501 29,43862
11-10-2012 00:00:17 5,436732 29,43862
...
And this is part of the error message:
TypeError Traceback (most recent call last)
<ipython-input-41-c65c2d17c55d> in <module>()
5 c = {2:conv, 3:conv}
6
----> 7 data = genfromtxt('Example.csv', dtype = t, skip_header=1, delimiter = ' ', converters = c)
...
<ipython-input-41-c65c2d17c55d> in <lambda>(valstr)
1 t = dtype([('Date', 'U12'), ('Time', 'U10'), ('Cond', 'f4'), ('Temp', 'f4')])
2
----> 3 conv = lambda valstr: float(valstr.replace(',','.'))
4
5 c = {2:conv, 3:conv}
TypeError: expected an object with the buffer interface
Am I doing something wrong here, or is this some kind of bug in genfromtxt?
I'am using Python 3.2 on Win7 x64. Numpy version is 1.6.2.
Apparently, genfromtxt feeds read columns to converter functions as byte strings, not as unicode strings.
The problem was solved for me by changing the code of the converter function as follows:
conv = lambda valstr: float(valstr.decode("utf-8").replace(',','.'))