Search code examples
pythonopensusesystem-administration

No module named 'encodings' on OpenSuse


A whole host of actions keep returning to this problem:

pip install encodings

Fatal Python error: Py_Initialize: Unable to get the locale encoding

ModuleNotFoundError: No module named 'encodings'


python3

Fatal Python error: Py_Initialize: Unable to get the locale encoding

ModuleNotFoundError: No module named 'encodings'


libreoffice --safe-mode

Fatal Python error: Py_Initialize: Unable to get the locale encoding

ModuleNotFoundError: No module named 'encodings'


zypper se python |grep '^i '

i  | libpython3_6m1_0                                         | Python Interpreter shared library                                                     | package
i  | libsamba-policy0-python3                                 | Active Directory Group Policy library                                                 | package
i  | python-qt3d-qt5-sip                                      | Sip files for python-qt3d-qt5                                                         | package
i  | python-qt5-utils                                         | Common files for PyQt5 for python2 and python3                                        | package
i  | python-qtdatavis3d-qt5-sip                               | Sip files for python-qtdatavis3d-qt5                                                  | package
i  | python-qtwebengine-qt5-sip                               | Sip files for python-qtwebengine-qt5                                                  | package
i  | python-rpm-generators                                    | Dependency generator dependencies for Python RPMs                                     | package
i  | python-rpm-macros                                        | RPM macros for building of Python modules                                             | package
i  | python-sip-common                                        | SIP tool to create python bindings -- common files                                    | package
i  | python-websockify-common                                 | Common data files for the Websockify TCP proxy/bridge                                 | package
i  | python3-Automat                                          | Self-service finite-state machines for the programmer on the go                       | package
i  | python3-Babel                                            | Internationalization utilities                                                        | package
i  | python3-Bottleneck                                       | A collection of fast NumPy array functions                                            | package
i  | python3-Brotli                                           | Python bindings for the Brotli compression library                                    | package
i  | python3-Cycler                                           | Composable style cycles                                                               | package
i  | python3-Cython                                           | The Cython compiler for writing C extensions for the Python language                  | package
i  | python3-Genshi                                           | A toolkit for generation of output for the web                                        | package
i  | python3-HeapDict                                         | A heap with decrease-key and increase-key operations                                  | package
i  | python3-Jinja2                                           | A fast and easy to use template engine written in pure Python                         | package
i  | python3-MarkupSafe                                       | Implements a XML/HTML/XHTML Markup safe string for Python                             | package
i  | python3-Pillow                                           | Python Imaging Library (Fork)                                                         | package
i  | python3-PyHamcrest                                       | Hamcrest framework for matcher objects                                                | package
i  | python3-PyJWT                                            | JSON Web Token implementation in Python                                               | package
i  | python3-PyNaCl                                           | Python binding to the Networking and Cryptography (NaCl) library                      | package
i  | python3-PySocks                                          | A Python SOCKS client module                                                          | package
i  | python3-PyWavelets                                       | PyWavelets is a Python wavelet transforms module                                      | package
i  | python3-PyYAML                                           | YAML parser and emitter for Python                                                    | package
i  | python3-Pygments                                         | A syntax highlighting package written in Python                                       | package
i  | python3-QtPy                                             | Abstraction layer on top of Qt bindings                                               | package
i  | python3-SQLAlchemy                                       | Database Abstraction Library                                                          | package
etc...

I think I've only got python 3.6 installed but the only suggestion I've seen is where there is more than one version installed.

Any suggestions on what I should test now to find the source of this problem?

My Env:


uname -a

Linux localhost.localdomain 5.3.18-150300.59.49-preempt #1 SMP PREEMPT Mon Feb 7 14:40:20 UTC 2022 (77d9d02) x86_64 x86_64 x86_64 GNU/Linux


python

-bash: /usr/bin/python: No such file or directory


python3

Python 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> (then exited from python console)


echo $PYTHONHOME returns nothing

echo $PYTHONPATH returns nothing


I have subsequently reinstalled the OS with no different result (surprising to me) and I've tried strace -o pythonstrace.txt python3.6 The (presumably) relevant part of the strace output is:

munmap(0x7f780442e000, 262144)          = 0
mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f780442e000
fstat(0, {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x7), ...}) = 0
stat("/sbin/python3.6", 0x7ffff6521070) = -1 ENOENT (No such file or directory)
stat("/usr/sbin/python3.6", 0x7ffff6521070) = -1 ENOENT (No such file or directory)
stat("/usr/local/sbin/python3.6", 0x7ffff6521070) = -1 ENOENT (No such file or directory)
stat("/root/bin/python3.6", 0x7ffff6521070) = -1 ENOENT (No such file or directory)
stat("/usr/local/bin/python3.6", 0x7ffff6521070) = -1 ENOENT (No such file or directory)
stat("/usr/bin/python3.6", {st_mode=S_IFREG|0755, st_size=10472, ...}) = 0
readlink("/usr/bin/python3.6", 0x7ffff6517fa0, 4096) = -1 EINVAL (Invalid argument)
openat(AT_FDCWD, "/usr/bin/pyvenv.cfg", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/pyvenv.cfg", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/usr/bin/Modules/Setup", 0x7ffff6521070) = -1 ENOENT (No such file or directory)
stat("/usr/bin/lib64/python3.6/os.py", 0x7ffff6518f40) = -1 ENOENT (No such file or directory)
stat("/usr/bin/lib64/python3.6/os.pyc", 0x7ffff6518f40) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python3.6/os.py", {st_mode=S_IFREG|0644, st_size=37526, ...}) = 0
stat("/usr/bin/pybuilddir.txt", 0x7ffff6521070) = -1 ENOENT (No such file or directory)
stat("/usr/bin/lib64/python3.6/lib-dynload", 0x7ffff6521070) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python3.6/lib-dynload", {st_mode=S_IFDIR|0755, st_size=4802, ...}) = 0
brk(0x55a4968ff000)                     = 0x55a4968ff000
mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f78043ee000
sysinfo({uptime=132115, loads=[4256, 3872, 448], totalram=16583553024, freeram=1984724992, sharedram=68530176, bufferram=122511360, totalswap=2146430976, freeswap=2146430976, procs=990, totalhigh=0, freehigh=0, mem_unit=1}) = 0
brk(0x55a496921000)                     = 0x55a496921000
sigaltstack({ss_sp=0x55a4968d9c20, ss_flags=0, ss_size=8192}, {ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=0}) = 0
stat("/usr/lib/python36.zip", 0x7ffff6529bd0) = -1 ENOENT (No such file or directory)
stat("/usr/lib", {st_mode=S_IFDIR|0755, st_size=10808, ...}) = 0
stat("/usr/lib/python36.zip", 0x7ffff65293b0) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python3.6", {st_mode=S_IFDIR|0755, st_size=4054, ...}) = 0
stat("/usr/lib64/python3.6", {st_mode=S_IFDIR|0755, st_size=4054, ...}) = 0
stat("/usr/lib64/python3.6", {st_mode=S_IFDIR|0755, st_size=4054, ...}) = 0
openat(AT_FDCWD, "/usr/lib64/python3.6", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
fstat(3, {st_mode=S_IFDIR|0755, st_size=4054, ...}) = 0
getdents64(3, /* 204 entries */, 32768) = 6800
getdents64(3, /* 0 entries */, 32768)   = 0
close(3)                                = 0
stat("/usr/lib64/python3.6/encodings/__init__.cpython-36m-x86_64-linux-gnu.so", 0x7ffff65296f0) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python3.6/encodings/__init__.abi3.so", 0x7ffff65296f0) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python3.6/encodings/__init__.so", 0x7ffff65296f0) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python3.6/encodings/__init__.py", {st_mode=S_IFREG|0644, st_size=5642, ...}) = 0
mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f78043ae000
stat("/usr/lib64/python3.6/encodings/__init__.py", {st_mode=S_IFREG|0644, st_size=5642, ...}) = 0
openat(AT_FDCWD, "/usr/lib64/python3.6/encodings/__pycache__/__init__.cpython-36.pyc", O_RDONLY|O_CLOEXEC) = 3
fcntl(3, F_GETFD)                       = 0x1 (flags FD_CLOEXEC)
fstat(3, {st_mode=S_IFREG|0644, st_size=3922, ...}) = 0
lseek(3, 0, SEEK_CUR)                   = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=3922, ...}) = 0
read(3, "3\r\r\n@*\273a\n\26\0\0\343\0\0\0\0\0\0\0\0\0\0\0\0\5\0\0\0@\0\0"..., 3923) = 3922
read(3, "", 1)                          = 0
close(3)                                = 0
stat("/usr/lib64/python3.6", {st_mode=S_IFDIR|0755, st_size=4054, ...}) = 0
stat("/usr/lib64/python3.6/codecs.py", {st_mode=S_IFREG|0644, st_size=36276, ...}) = 0
stat("/usr/lib64/python3.6/codecs.py", {st_mode=S_IFREG|0644, st_size=36276, ...}) = 0
openat(AT_FDCWD, "/usr/lib64/python3.6/__pycache__/codecs.cpython-36.pyc", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=33902, ...}) = 0
lseek(3, 0, SEEK_CUR)                   = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=33902, ...}) = 0
read(3, "3\r\r\n@*\273a\264\215\0\0\343\0\0\0\0\0\0\0\0\0\0\0\0:\0\0\0@\0\0"..., 33903) = 33902
read(3, "", 1)                          = 0
close(3)                                = 0
stat("/usr/lib64/python3.6/encodings", {st_mode=S_IFDIR|0755, st_size=2708, ...}) = 0
stat("/usr/lib64/python3.6/encodings", {st_mode=S_IFDIR|0755, st_size=2708, ...}) = 0
stat("/usr/lib64/python3.6/encodings", {st_mode=S_IFDIR|0755, st_size=2708, ...}) = 0
openat(AT_FDCWD, "/usr/lib64/python3.6/encodings", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
fstat(3, {st_mode=S_IFDIR|0755, st_size=2708, ...}) = 0
getdents64(3, /* 128 entries */, 32768) = 4336
getdents64(3, /* 0 entries */, 32768)   = 0
close(3)                                = 0
stat("/usr/lib64/python3.6/encodings/aliases.py", {st_mode=S_IFREG|0644, st_size=15577, ...}) = 0
stat("/usr/lib64/python3.6/encodings/aliases.py", {st_mode=S_IFREG|0644, st_size=15577, ...}) = 0
openat(AT_FDCWD, "/usr/lib64/python3.6/encodings/__pycache__/aliases.cpython-36.pyc", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=6266, ...}) = 0
lseek(3, 0, SEEK_CUR)                   = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=6266, ...}) = 0
read(3, "3\r\r\n@*\273a\331<\0\0\343\0\0\0\0\0\0\0\0\0\0\0\0D\1\0\0@\0\0"..., 6267) = 6266
read(3, "", 1)                          = 0
close(3)                                = 0
stat("/usr/lib64/python3.6/encodings", {st_mode=S_IFDIR|0755, st_size=2708, ...}) = 0
stat("/usr/lib64/python3.6/encodings/utf_8.py", {st_mode=S_IFREG|0644, st_size=1005, ...}) = 0
stat("/usr/lib64/python3.6/encodings/utf_8.py", {st_mode=S_IFREG|0644, st_size=1005, ...}) = 0
openat(AT_FDCWD, "/usr/lib64/python3.6/encodings/__pycache__/utf_8.cpython-36.pyc", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=1584, ...}) = 0
lseek(3, 0, SEEK_CUR)                   = 0

This seems to show that it can't find a whole bunch of files, from a variety of different directories and they're not even even all in a common directory branch! a selection of examples: /sbin/python3.6, /usr/sbin/python3.6, /usr/local/sbin/python3.6, /usr/local/bin/python3.6, /usr/lib64/python3.6

Just also tried from my home bash prompt:

me@myhost:~> python
ImportError: No module named site
me@myhost:~> python2
ImportError: No module named site
me@myhost:~> python2.7
ImportError: No module named site
me@myhost:~> python3
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007f5e446c0740 (most recent call first):
Aborted (core dumped)
me@myhost:~> python3.6
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007f0e8c018740 (most recent call first):
Aborted (core dumped)
greg@TRANQUILITY:~> 

And as root:

HOST:~ # python
Python 2.7.18 (default, Mar 04 2021, 23:25:57) [GCC] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 
[2]+  Stopped                 python
HOST:~ # python2
Python 2.7.18 (default, Mar 04 2021, 23:25:57) [GCC] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 
[3]+  Stopped                 python2
HOST:~ # python2.7
Python 2.7.18 (default, Mar 04 2021, 23:25:57) [GCC] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 
[4]+  Stopped                 python2.7
HOST:~ # python3
Python 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
[5]+  Stopped                 python3
HOST:~ # python3.6
Python 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

I checked env and added export LC_CTYPE=en_GB.UTF-8 to ~/.bashrc (mine, not root's) - made no difference from a new terminal

I also used 'strace python3.6as both me and root and saved the output to two files and then in VSCode diffed them. I don't have the tech knowledge to make sense of it but the 'root' strace is much longer and works, the 'greg' strace is shorter and on line 216, sayswrite(2, "No module named 'encodings'", 27No module named 'encodings') = 27`

As greg https://www.dropbox.com/s/p55uen19njil5zb/stracePython3.6-greg.lua?dl=0

As root https://www.dropbox.com/s/dlxu43yeqnb7li2/stracePython3.6-root.lua?dl=0


Solution

  • Looking at the strace output for both root and greg, the problem seems clear.

    For the root user, python 3.6 finds the libraries in /usr/lib64/python3.6.

    However, for greg, it only looks under /usr/bin/python3 for subdirectories. That doesn't work because /usr/bin/python3 is a file.

    I suspect that the user greg has PYTOHNHOME set erroneously to the location of the Python binary , and that is causing the issue.

    Remove PYTOHNHOME from your environment, log out and log in again.

    Note: the stuff below is probably barking up the wrong tree. I'll leave it for information.


    The encodings module is an (undocumented) part of the python standard library. It is used by the locale module.

    Based on the output I suspect that your Python installation has been damaged or corrupted. Try re-installing python.

    EDIT:

    If a forced re-install doesn't fix the problem, check that the directory encodings exist in your Python stdlib directory, and is accessible for all users.

    To find out which directory that is:

    python
    >>> import sysconfig
    >>> sysconfig.get_path('stdlib')
    '/usr/local/lib/python3.9'
    

    Check accessability:

    ls -ld /usr/local/lib/python3.9/encodings
    drwxr-xr-x  3 root  wheel  5632 Dec 11 14:34 /usr/local/lib/python3.9/encodings/
    

    Read and execute for all users, so that is OK.

    EDIT2:

    Looking at the trace results.

    I don't think the missing files are something to worry about. That's just Python looking for the different locations/names that files can have, given that Python is a multiplatform program.

    The interesting bit is near the end:

    stat("/usr/lib64/python3.6/encodings/__init__.py", {st_mode=S_IFREG|0644, st_size=5642, ...}) = 0
    mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f78043ae000
    

    This shows that it can find the directory, and map __init__.py into memory.

    EDIT3:

    Since both Python 2.7 and 3.6 are end-of-life now, I would suggest removing them, and install python 3.10 instead.

    EDIT4:

    The directory structure is determined by how you install Python. WRT permissions, directories usually have permissions 0755 and files 0644. Traditionally on UNIX, files in /usr/ are owned by root, while the group differs among systems. The last number (permission for "others" being neither the owner nor the group) are probably most relevant.

    You can fix permissions with a combination of find and chmod. For example:

    find /usr/lib64/python3.6 -type d -exec chmod 755 {} \;
    find /usr/lib64/python3.6 -type f -exec chmod 644 {} \;
    

    should set a good set of permissions for both files and directories in the Python tree.