Search code examples
djangopostgresqldatabase-restoreheroku-postgres

Django Heroku Postgresql ERROR: invalid byte sequence for encoding "UTF8"

I am using Django with Heroku Postgreql. I use both English and Turkish languages on default DB settings. My DB works with no issues both locally and on Heroku. However, I get an error when I try to dump and restore a db from local to production.

psql -U user -d db_name -f "b003.dump"

psql:b003.dump:270: ERROR:  invalid byte sequence for encoding "UTF8": 0xa3
Solution

  • First guess: Encoding mismatch

    Your file b003.dump is not in the UTF-8 encoding.

    You will need to override the encoding settings, specifying the correct text encoding for the file.

    It's probably iso-8859-9 if it has Turkish text, though 0xa3 is the pound symbol (£) in many of the ISO-8859 encodings.

    Try:

    PGCLIENTENCODING="iso-8859-9" psql ....

    I also suggest checking the output of the locale command to see what your system default encoding is, and of file -bi b003.dump to try to guess the encoding.

    Second guess, after comments revealed file output

    Your file isn't an SQL script style dump. It a PostgreSQL custom database dump. That's why file says:

    b003.dump: PostgreSQL custom database dump - v1.12-0

    Restore it with the pg_restore command.

    pg_restore -f b003.dump -d db_name -U user

    At some point I want to enhance psql so it tells you this, rather than failing with an error.

    The weird thing here is that you must've had a lot of previous errors, before the one you showed here, if it's a custom dump.