Why is does the git end-of-file-fixer keeps adding a unknown character at the end of my file?

Question

I have a .json file on Windows which I tried to commit to my git repo. We have the end-of-file-fixer pre-commit hook activated, to make sure there is always a new line at the end of each file. However, each time I commit my file it adds some unknown character to the end of my file:

If I later try to read this file with Python, it complains that there is an unknown character in the json and it can't decode the file.

Traceback (most recent call last):
  File "C:\Users\rruiter\Miniconda3\envs\nest39\lib\site-packages\IPython\core\interactiveshell.py", line 3251, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-7-cd035d98e6fe>", line 2, in <module>
    data = json.loads(f.read())
  File "C:\Users\rruiter\Miniconda3\envs\nest39\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\Users\rruiter\Miniconda3\envs\nest39\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\rruiter\Miniconda3\envs\nest39\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Answer 1

Turns out my my .json file was encoded as UTF-16 LE BOM. When I converted it into UTF-8 everything worked again.