Search code examples
pandascsv

Pandas Error: need to escape, but no escapechar set

I am creating a dataset by concatenating different other datasets in order to get a large one. This is in the purpose of training a generative AI models on it in the field of NLP

df = pd.concat([df, df_1, df_2, df_3])

Then I try to save it into csv file to use it when I need

df.to_csv('java_dataset.csv', index=False)

But I got this error

Error
Traceback (most recent call last)
in <cell line: 1>()
----> 1 df.to_csv('java_dataset.csv', index=False)

5 frames /usr/local/lib/python3.10/dist-packages/pandas/io/formats/csvs.py in _save_chunk(self, start_i, end_i)
318
319 ix = self.data_index[slicer]._format_native_types(**self._number_format)
--> 320 libwriters.write_csv_rows(
321 data,
322 ix,

writers.pyx in pandas._libs.writers.write_csv_rows()

Error: need to escape, but no escapechar set

Solution

  • Try:

    df.to_csv('java_dataset.csv',index=False, quoting=3, escapechar='\\')