I am using California housing data, which has latitude and longitude. Is it good practice to remove them (latitude & longitude)before I continue to train my model?
If you are just using raw lat/long information, then yes, you should remove them. That's because the values of lat/long are not meaningful in and of themselves, conditional on your model not having any "understanding" of what a change in lat/long means. For instance, what would a change in 1 degree in latitude mean in terms of a change in your target variable? If there is nothing related that way, then you introduce either noise or potentially spurious relationships.
Lat/long are more often useful as a way to calculate a feature that does have a meaningful impact on your response variable: distance between points, area withing boundries, or anything where you can say "the change in this feature that I calculate from spatial information correlates with a change in the response".
In short: either take it out, or use it to calculate a feature you do want to include.