Search code examples
pythoncsvpandaspywin32

How to get the system wide list separator on Windows


I use pandas to convert csv to more usable datatypes.
The problem is that csv files doesn't need to be separated by commas. The separator (between cells) is defined by the operating system (at least under Windows), and when the system wide list separator differs from comma, pandas (or anything else I tried) cannot determine what separator should be used.
Sure, I can make a twocell csv file with pywin or something, and get the separator, but isn't there a direct way to do this?

Below you can see the txt representation of two csv files, which are displayed identically on Windows machines, where respectively "," and ";" are the default list separators.

  • test,1,2,"My name is Bond, James Bond",My age is; 41
  • test;1;2;My name is Bond, James Bond;"My age is; 41"

Solution

  • Use Python's built in Sniffer class to detect the CSV's delimiters. Here's an example from the docs:

    with open('example.csv', 'rb') as csvfile:
    
        #detect the delimiter used
        dialect = csv.Sniffer().sniff(csvfile.read(1024))
    
        #return to the beginning of the file
        csvfile.seek(0)
    
        #file should now open with the correct delimiter.
        reader = csv.reader(csvfile, dialect)