getting a UnicodeEncodeError when trying to write a JSON file

Question

Given input file with content:

{ "symbol": "°C" }

And this code:

import sys
import json

with open(sys.argv[1], 'r') as ifile, open(sys.argv[2], 'w') as ofile:
    json.dump(json.load(ifile), ofile, indent=4, ensure_ascii=False)

I get an error:

$ python2.7 play.py input.json output.json
Traceback (most recent call last):
  File "play.py", line 5, in <module>
    json.dump(json.load(ifile), ofile, indent=4, ensure_ascii=False)
  File "/usr/lib/python2.7/json/__init__.py", line 190, in dump
    fp.write(chunk)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xb0' in position 1: ordinal not in range(128)

But Python 3 works fine:

$ python3.3 play.py input.json output.json
$ cat output.json 
{
    "symbol": "°C"
}

Answer 1

You can use the codecs module to deal with it by declaring the encoding of the file:

import sys
import json
import codecs

with codecs.open(sys.argv[1], 'r', 'utf-8') as ifile, codecs.open(sys.argv[2], 'w', 'utf-8') as ofile:
    json.dump(json.load(ifile), ofile, indent=4, ensure_ascii=False)