i have a data column like this:
df['zone'].unique()
out[4]:
array(['BROOKLYN', 'BRONX', '07 BRONX', 'Unspecified', '05 BRONX',
'QUEENS', 'MANHATTAN', '07 MANHATTAN', 'STATEN ISLAND',
'17 BROOKLYN', '0 Unspecified', 'Unspecified MANHATTAN',
'12 BROOKLYN', '07 BROOKLYN', '09 MANHATTAN', '01 STATEN ISLAND',
'12 MANHATTAN', '04 QUEENS', '06 BROOKLYN',
'01/04/2016 01:45:00 PM', '01/02/2016 05:43:34 AM', '07 QUEENS',
'11 BRONX', '01/04/2016 03:45:00 PM', '10 MANHATTAN', '03 BRONX',
'04 BRONX', ' or 311 Online."', '01/13/2016 12:00:00 AM',
'04 BROOKLYN', '03 BROOKLYN', '01 QUEENS',
'01/04/2016 03:34:55 PM', '08 MANHATTAN', '14 BROOKLYN',
'10 QUEENS', 'Unspecified STATEN ISLAND', '02 BRONX', '09 BRONX',
'08 QUEENS', '10 BRONX', '03 MANHATTAN', '12 QUEENS',
' please call (212) NEW-YORK (212-639-9675)."',
'Unspecified BROOKLYN', '01/11/2016 04:45:00 PM', '04 MANHATTAN',
'01 BRONX', '09 BROOKLYN', '01/05/2016 07:00:00 AM', '18 BROOKLYN',
'01/08/2016 09:00:00 AM', '01 BROOKLYN', '06 BRONX',
'01 MANHATTAN', '01/06/2016 12:15:00 PM', '02/04/2016 08:45:00 PM',
'01/05/2016 12:45:00 PM', ' no action was taken."', '05 BROOKLYN',
'08 BROOKLYN', 'Unspecified QUEENS', '01/08/2016 03:00:00 PM',
'08/22/2016 12:00:00 AM', '13 BROOKLYN', '02 QUEENS', '14 QUEENS',
'01/05/2016 08:45:00 AM', '11 QUEENS', '02 MANHATTAN',
'01/08/2016 10:05:00 AM', '01/05/2016 01:05:00 PM',
'Unspecified BRONX', '06 QUEENS', '09 QUEENS', '15 BROOKLYN',
'01/07/2016 09:25:00 AM', '02 STATEN ISLAND',
'01/02/2016 12:00:00 PM', '01/06/2016 08:45:00 PM',
'04/04/2016 12:00:00 AM', '01/06/2016 08:30:00 AM'])
as you can see, i have a lot of mixed types there, everything is being categorized by pandas as string object. I have tried already some parameters in the pd.read_csv command like low_memory = False, chunksize, etc... without any success.
What i really need to do here though is to kind of map this column into the following format:
(Manhattan -> 1, Brooklyn -> 2, Queens -> 3, Staten Island -> 4, Bronx -> 5, Other -> 0)
i also need to include the string '07 BRONX' as bronx and not as other or unknown.
I have thought about the .map() method as the way to go, but since the column is a real mess of mixed types, i'm not sure anymore about what my options are.
I will appreciate any suggestions here.
Thanks a lot in advance
Create dictionary for mapping values by extract keys of dictionary with | for OR by map, last fillna all unmatched values to 0:
a = np.array(['BROOKLYN', 'BRONX', '07 BRONX', 'Unspecified', '05 BRONX',
'QUEENS', 'MANHATTAN', '07 MANHATTAN', 'STATEN ISLAND',
'17 BROOKLYN', '0 Unspecified', 'Unspecified MANHATTAN',
'12 BROOKLYN', '07 BROOKLYN', '09 MANHATTAN', '01 STATEN ISLAND',
'12 MANHATTAN', '04 QUEENS', '06 BROOKLYN',
'01/04/2016 01:45:00 PM', '01/02/2016 05:43:34 AM', '07 QUEENS',
'11 BRONX', '01/04/2016 03:45:00 PM', '10 MANHATTAN', '03 BRONX',
'04 BRONX', ' or 311 Online."', '01/13/2016 12:00:00 AM',
'04 BROOKLYN', '03 BROOKLYN', '01 QUEENS',
'01/04/2016 03:34:55 PM', '08 MANHATTAN', '14 BROOKLYN',
'10 QUEENS', 'Unspecified STATEN ISLAND', '02 BRONX', '09 BRONX',
'08 QUEENS', '10 BRONX', '03 MANHATTAN', '12 QUEENS',
' please call (212) NEW-YORK (212-639-9675)."',
'Unspecified BROOKLYN', '01/11/2016 04:45:00 PM', '04 MANHATTAN',
'01 BRONX', '09 BROOKLYN', '01/05/2016 07:00:00 AM', '18 BROOKLYN',
'01/08/2016 09:00:00 AM', '01 BROOKLYN', '06 BRONX',
'01 MANHATTAN', '01/06/2016 12:15:00 PM', '02/04/2016 08:45:00 PM',
'01/05/2016 12:45:00 PM', ' no action was taken."', '05 BROOKLYN',
'08 BROOKLYN', 'Unspecified QUEENS', '01/08/2016 03:00:00 PM',
'08/22/2016 12:00:00 AM', '13 BROOKLYN', '02 QUEENS', '14 QUEENS',
'01/05/2016 08:45:00 AM', '11 QUEENS', '02 MANHATTAN',
'01/08/2016 10:05:00 AM', '01/05/2016 01:05:00 PM',
'Unspecified BRONX', '06 QUEENS', '09 QUEENS', '15 BROOKLYN',
'01/07/2016 09:25:00 AM', '02 STATEN ISLAND',
'01/02/2016 12:00:00 PM', '01/06/2016 08:45:00 PM',
'04/04/2016 12:00:00 AM', '01/06/2016 08:30:00 AM'])
df=pd.DataFrame({ 'zone':a })
d = {'MANHATTAN':1, 'BROOKLYN':2, 'QUEENS' : 3, 'STATEN ISLAND' : 4, 'BRONX' : 5}
pat = '(' + '|'.join(d.keys()) + ')'
df['code'] = df['zone'].str.extract(pat, expand=False).map(d).fillna(0, downcast='int')
print (df.head(10))
zone code
0 BROOKLYN 2
1 BRONX 5
2 07 BRONX 5
3 Unspecified 0
4 05 BRONX 5
5 QUEENS 3
6 MANHATTAN 1
7 07 MANHATTAN 1
8 STATEN ISLAND 4
9 17 BROOKLYN 2