There are similar questions asked before on stackoverflow, however, none of them could fix my problem. I don't understand why info() clearly doesn't output a "bool" but sklearn is outputting an error saying I have boolean values in my dataframe. Can anyone help me debug this thanks!
X = df.drop("Transported", axis=1)
y = df.Transported
X.info()
"""
output:
>>> <class 'pandas.core.frame.DataFrame'>
RangeIndex: 8693 entries, 0 to 8692
Data columns (total 11 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 HomePlanet 8492 non-null object
1 CryoSleep 8476 non-null object
2 Cabin 8494 non-null object
3 Destination 8511 non-null object
4 Age 8514 non-null float64
5 VIP 8490 non-null object
6 RoomService 8512 non-null float64
7 FoodCourt 8510 non-null float64
8 ShoppingMall 8485 non-null float64
9 Spa 8510 non-null float64
10 VRDeck 8505 non-null float64
dtypes: float64(6), object(5)
memory usage: 747.2+ KB
"""
categorical_features = ["HomePlanet", "CryoSleep", "Cabin", "Destination", "VIP"]
categorical_transformer = Pipeline(steps=[
("imputer", SimpleImputer(strategy="constant", fill_value="missing")),
("encoder", OneHotEncoder(handle_unknown="ignore"))
])
numerical_features = ["Age", "RoomService", "FoodCourt", "ShoppingMall", "Spa", "VRDeck"]
numerical_transformer = Pipeline(steps=[
("imputer", SimpleImputer(strategy="median")),
("scaler", StandardScaler())
])
preprocessor = ColumnTransformer(transformers=[
("cat", categorical_transformer, categorical_features),
("num", numerical_transformer, numerical_features)
])
model = Pipeline(steps=[("preprocessor", preprocessor), ("model", RandomForestRegressor())])
X = df.drop("Transported", axis=1)
y = df["Transported"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
model.fit(X_train, y_train)
model.score(X_test, y_test)
error message:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
File D:\Programming\python\machine_learning\ml\lib\site-packages\sklearn\utils\_encode.py:173, in _unique_python(values, return_inverse, return_counts)
171 uniques_set, missing_values = _extract_missing(uniques_set)
--> 173 uniques = sorted(uniques_set)
174 uniques.extend(missing_values.to_list())
TypeError: '<' not supported between instances of 'str' and 'bool'
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
Input In [68], in <cell line: 19>()
17 y = df["Transported"]
18 X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
---> 19 model.fit(X_train, y_train)
...
Thing is, columns like CryoSleep and VIP are actually boolean (I assume this is the original Kaggle ST dataset). They're shown as object because of missing values (resulting in a mixed type).
Try explicitly changing the values first, e.g.:
df['CryoSleep'] = str(df['CryoSleep'])
df['VIP'] = str(df['VIP'])
On a minor note, you probably meant using RandomForestClassifier().