UnicodeDecodeError: 'ascii' codec can't decode byte 0xe0 in position 2: ordinal not in range(128)

Question

I installed pywikibot-core (version 2.0b3) for my Mediawiki installation. I got an error when i tried to run a command which contains Unicode text.

I run the following command:

python pwb.py replace.py -regex -start:! "\[মুয়ায্যম হুসায়ন খান\]" "[মুয়ায্‌যম হুসায়ন খান]"  -summary:"fix: মুয়ায্যম > মুয়ায্‌যম"

Here is the error i got:

Traceback (most recent call last):
  File "pwb.py", line 161, in <module>
    import pywikibot  # noqa
  File "/var/www/html/banglapedia_bn/core/pywikibot/__init__.py", line 32, in <module>
    from pywikibot import config2 as config
  File "/var/www/html/banglapedia_bn/core/pywikibot/config2.py", line 285, in <module>
    if arg.startswith("-verbose") or arg == "-v":
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe0 in position 2: ordinal not in range(128)

Answer 1

Use python3 instead of python.

---

You are seeing that error because the module config2.py uses from __future__ import unicode_literals, making all strings in the module unicode objects. However, sys.args is a bytestring, and is not affected by __future__ imports.

Therefore, because arg is a byte string, but "-verbose" and "-v" are two unicode strings, arg gets implicitly promoted to unicode, but this is failing because implicit conversion only works with ASCII.

Instead, in Python 3, all strings are unicode by default, including sys.args.