Search code examples
arraysnumpydatetimepython-datetimegenfromtxt

Importing data using Numpy genfromtext and formatting column with datetime

I have a long text file that I am importing with numpy genfromtext:

00:00:01 W 348 18.2 55.9 049 1008.8 0.000 
00:00:02 W 012 12.5 55.9 049 1008.8 0.000 
00:00:03 W 012 12.5 55.9 049 1008.8 0.000
00:00:04 W 357 18.2 55.9 049 1008.8 0.000 
00:00:05 W 357 18.2 55.9 049 1008.8 0.000 
00:00:06 W 339 17.6 55.9 049 1008.8 0.000 

testdata = np.genfromtxt(itertools.islice(f_in, 0, None, 60),\
                         names=('time','ew','d12','s12','t12','p12'.....)

time = (testdata['time'])

This is organizing all the data into an array. The first column of data in the file is a timestamp for each row. In the text file it is formatted as 00:00:00 so in format (%H:%m:%s). However in the actual array that is generated, it turns it into 1900-01-01 00:00:00 . When plotting my data with time, I cannot get it to drop the Y-m-d.

I have tried time = time.strftime('%H:%M:%S') and dt.datetime.strptime(time.decode('ascii'), '%H:%M:%S')

Both do nothing. How can I turn my whole array of times to keep the original %H:%m:%s format without it adding in the %Y-%m-%d?

Solution

  • EDIT: based on data provided, you can import your file like this: 

    str2date = lambda x: datetime.strptime(x.decode("utf-8"), '%H:%M:%S').time()

data = np.genfromtxt(itertools.islice(f_in, 0, None, 60), dtype=None,names=('time','ew','d12','s12','t12','p12'.....), delimiter=' ', converters = {0: str2date})

print(data['time'])

    output: 

    00:00:01

    Note than you would need to .decode("utf-8") your input to str2date since it accepts bytes. You can set your dtype in np.genfromtxt() according to your specific file content.

    You can also use this if your data is in right format: 

    dt.datetime.strptime(time,"%H:%M:%S").time()