I’m having some issues trying to load a dataset in Azure ML Studio, a dataset containing a column that looks like a DateTime, but is in fact a string. Azure ML Studio converts the values to DateTimes internally, and no amount of wrangling seems to convince it of the that they’re in fact strings.
This is an issue, because during conversion the values lose precision and start appearing as duplicates whereas in fact they are unique. Does anybody know if ML Studio can be configured not to infer data types for columns while importing a dataset?
Now, for the long(er) story :)
I’m working here with a public dataset - specifically Kaggle’s New York City Fare Prediction competition. I wanted to see if I could do a quick-and-dirty solution using Azure ML Studio, however the dataset’s unique key values are of the form
2015-01-27 13:08:24.0000003
2015-01-27 13:08:24.0000002
2011-10-06 12:10:20.0000001
and so on.
When importing them in my experiment the key values get converted to DateTime, making them no longer unique, even though they’re unique in the csv. Needless to say, this prevents me from submitting any solution to Kaggle, since I can’t identify the rows uniquely :).
I’ve tried the following:
I’ve reproduced this behavior with both the big, 5.5GB train dataset, but also with the more manageable sample_submission dataset.
Curious to know if there is some sort of workaround to tell ML Studio not to try converting this column while loading the dataset. I'm looking here specifically for Azure ML Studio-only solutions, as I don't want to do any preprocessing on the dataset.
I have tried with you sample data and here is my quick and dirty solution: 1) Add any symbol (I've added the '#') in front of each date 2) Load it to AML Studio (it is now considered as a string feature) 3) Add a Python/R component to remove the '#' symbol and explicitly convert the column to string (as.string(columnname) or str(columnname))
Hope this helps