Search code examples
pythonpandasread-csv

pd.read_csv: error when "," comma is present in string column

I am reading in a csv file locally.The sep for the data is "|", so I specify pd.read_csv(repo, sep="|"). The data loads.

However, when I examine the data, any time there is a comma in the first feature, all of that row’s data is squished into that first feature. My guess is that pandas is still seeing ‘,’ as the sep.

import pandas as pd

df = pd.read_csv(repo, sep="|", names=cols, encoding="latin-1")

df.iloc[10:15, ::]

Is there any way to handle this in pandas, or will I have to break out excel and replace the data from there?

Solution

  • Got it; had to add the "quotechar" parameter to pd.read_csv().