Export json from python to C# file encoding issue

Question

I have a python script that generates a json file that is further used on a C# application. This is the code creating the export:

def put(data, filename):
    with open(filename, 'w') as outfile:
        json.dump(data, outfile, indent=4, ensure_ascii=False, encoding='utf8')

Then in the C# application I attempt to read it the following way:

public static string readFile(string filename)
{
    using (StreamReader r = new StreamReader(filename))
    {
        return r.ReadToEnd();
    }
}

//different file
string json_result = funcs.readFile(out_file);
VirtualMachine newvm = JsonConvert.DeserializeObject<MyClass>(json_result);

The problem is json_result contains the characters \r\n at each line break and fails to Deserialize the json.

I tried to check the encoding produced by the python script and according to SublimeText is u'Undefined'

How can I make python correctly encode the file for C# or make C# load it with the right encoding?

Answer 1

What I imagine is happening is that you are on a Windows system, and the python script's open command is automatically making newlines ends with \r\n and the C# reader does not expect them.

One way to fix this is to write to the file as binary instead of as text:

def put(data, filename):
    with open(filename, 'wb') as outfile:
        outfile.write(json.dumps(data, indent=4, ensure_ascii=False).encode("utf-8"))

However the issue may also inside of the JSON library itself while using indent, which could be fixed by either removing the argument (but I assume you want it pretty for a reason) or edit the JSON string after generating it and do a replace /r/n with /n before writing to the file.