So I'm struggling with what will be, I'm sure, an easy fix: No matter what I import in my Python scripts, I always get the error message
from flask import Flask;
ModuleNotFoundError: No module named 'flask'
Replace "flask" here by any other module (tried "psycopg2"), nothing gets imported. I believe that it has something to do with my shell somehow but I'm not sure where the problem comes from and especially, how to fix it.
Here's more information about my config:
Pip list:
Package Version
---------------- -------
click 8.1.3
Flask 2.1.3
Flask-SQLAlchemy 2.5.1
itsdangerous 2.1.2
Jinja2 3.1.2
MarkupSafe 2.1.1
pip 22.1.2
psycopg2 2.9.3
psycopg2-binary 2.9.3
setuptools 58.1.0
SQLAlchemy 1.4.39
Werkzeug 2.2.0
any ideas?
EDIT:
where python: python: aliased to /usr/bin/python3
where python3:
/Library/Frameworks/Python.framework/Versions/3.10/bin/python3 /opt/homebrew/bin/python3 /usr/local/bin/python3 /usr/bin/python3 /Library/Frameworks/Python.framework/Versions/3.10/bin/python3 /opt/homebrew/bin/python3
where pip
/Library/Frameworks/Python.framework/Versions/3.10/bin/pip /Library/Frameworks/Python.framework/Versions/3.10/bin/pip
where pip3
/Library/Frameworks/Python.framework/Versions/3.10/bin/pip3 /opt/homebrew/bin/pip3 /usr/local/bin/pip3 /usr/bin/pip3 /Library/Frameworks/Python.framework/Versions/3.10/bin/pip3 /opt/homebrew/bin/pip3
EDIT2:
Sys code:
➜ ~ python3
Python 3.10.5 (v3.10.5:f377153967, Jun 6 2022, 12:36:10) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; print(sys.path)
['', '/Library/Frameworks/Python.framework/Versions/3.10/lib/python310.zip', '/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10', '/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/lib-dynload', '/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages']
pip show psycopg2:
➜ ~ pip show psycopg2
Name: psycopg2
Version: 2.9.3
Summary: psycopg2 - Python-PostgreSQL Database Adapter
Home-page: https://psycopg.org/
Author: Federico Di Gregorio
Author-email: [email protected]
License: LGPL with exceptions
Location: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages
Requires:
Required-by:
EDIT 3: profiles and shell related files.
~ folder: 1 staff 150 Jun 29 18:57 .bash_profile 1 staff 131 May 30 16:47 .bashrc 22 staff 704 May 30 13:32 .oh-my-zsh 1 staff 292 Jun 29 18:32 .zprofile 1 staff 126 Jun 17 10:19 .zprofile.pysave 1 staff 18097 Jul 28 09:53 .zsh_history 7 staff 224 Jul 13 13:38 .zsh_sessions 1 staff 4163 Jul 21 11:39 .zshrc
cat bash_profile:
source ~/.bashrc
export PATH="$PATH:/Applications/Visual Studio Code.app/Contents/Resources/app/bin"
export PYTHONPATH="$PATH:/usr/local/bin/python"
cat .zprofile
export PATH="$PATH:/Applications/Visual Studio Code.app/Contents/Resources/app/bin"
eval "$(/opt/homebrew/bin/brew shellenv)"
# Setting PATH for Python 3.10
# The original version is saved in .zprofile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.10/bin:${PATH}"
export PATH
cat .zshrc
# If you come from bash you might have to change your $PATH.
# export PATH=$HOME/bin:/usr/local/bin:$PATH
# Path to your oh-my-zsh installation.
export ZSH="$HOME/.oh-my-zsh"
# Set name of the theme to load --- if set to "random", it will
# load a random theme each time oh-my-zsh is loaded, in which case,
# to know which specific one was loaded, run: echo $RANDOM_THEME
# See https://github.com/ohmyzsh/ohmyzsh/wiki/Themes
ZSH_THEME="robbyrussell"
# Set list of themes to pick from when loading at random
# Setting this variable when ZSH_THEME=random will cause zsh to load
# a theme from this variable instead of looking in $ZSH/themes/
# If set to an empty array, this variable will have no effect.
# ZSH_THEME_RANDOM_CANDIDATES=( "robbyrussell" "agnoster" )
# Uncomment the following line to use case-sensitive completion.
# CASE_SENSITIVE="true"
# Uncomment the following line to use hyphen-insensitive completion.
# Case-sensitive completion must be off. _ and - will be interchangeable.
# HYPHEN_INSENSITIVE="true"
# Uncomment one of the following lines to change the auto-update behavior
# zstyle ':omz:update' mode disabled # disable automatic updates
# zstyle ':omz:update' mode auto # update automatically without asking
# zstyle ':omz:update' mode reminder # just remind me to update when it's time
# Uncomment the following line to change how often to auto-update (in days).
# zstyle ':omz:update' frequency 13
# Uncomment the following line if pasting URLs and other text is messed up.
# DISABLE_MAGIC_FUNCTIONS="true"
# Uncomment the following line to disable colors in ls.
# DISABLE_LS_COLORS="true"
# Uncomment the following line to disable auto-setting terminal title.
# DISABLE_AUTO_TITLE="true"
# Uncomment the following line to enable command auto-correction.
# ENABLE_CORRECTION="true"
# Uncomment the following line to display red dots whilst waiting for completion.
# You can also set it to another string to have that shown instead of the default red dots.
# e.g. COMPLETION_WAITING_DOTS="%F{yellow}waiting...%f"
# Caution: this setting can cause issues with multiline prompts in zsh < 5.7.1 (see #5765)
# COMPLETION_WAITING_DOTS="true"
# Uncomment the following line if you want to disable marking untracked files
# under VCS as dirty. This makes repository status check for large repositories
# much, much faster.
# DISABLE_UNTRACKED_FILES_DIRTY="true"
# Uncomment the following line if you want to change the command execution time
# stamp shown in the history command output.
# You can set one of the optional three formats:
# "mm/dd/yyyy"|"dd.mm.yyyy"|"yyyy-mm-dd"
# or set a custom format using the strftime function format specifications,
# see 'man strftime' for details.
# HIST_STAMPS="mm/dd/yyyy"
# Would you like to use another custom folder than $ZSH/custom?
# ZSH_CUSTOM=/path/to/new-custom-folder
# Which plugins would you like to load?
# Standard plugins can be found in $ZSH/plugins/
# Custom plugins may be added to $ZSH_CUSTOM/plugins/
# Example format: plugins=(rails git textmate ruby lighthouse)
# Add wisely, as too many plugins slow down shell startup.
plugins=(git)
source $ZSH/oh-my-zsh.sh
A
# User configuration
# export MANPATH="/usr/local/man:$MANPATH"
# You may need to manually set your language environment
# export LANG=en_US.UTF-8
# Preferred editor for local and remote sessions
# if [[ -n $SSH_CONNECTION ]]; then
# export EDITOR='vim'
# else
# export EDITOR='mvim'
# fi
# Compilation flags
# export ARCHFLAGS="-arch x86_64"
# Set personal aliases, overriding those provided by oh-my-zsh libs,
# plugins, and themes. Aliases can be placed here, though oh-my-zsh
# users are encouraged to define aliases within the ZSH_CUSTOM folder.
# For a full list of active aliases, run `alias`.
#
# Example aliases
# alias zshconfig="mate ~/.zshrc"
# alias ohmyzsh="mate ~/.oh-my-zsh"
alias python=/usr/bin/python3
export PATH="/opt/homebrew/opt/libpq/bin:$PATH"
export PATH="/opt/homebrew/opt/openssl@3/bin:$PATH"
export LIBRARY_PATH=$LIBRARY_PATH:/opt/homebrew/opt/openssl@3/lib/
export LDFLAGS="-L/opt/homebrew/opt/libpq/lib"
export CPPFLAGS="-I/opt/homebrew/opt/libpq/include"
Summary: use
python -m pip install <packages>
to guarantee that python can find the installed packages.
This is a problem that comes up often when there are multiple Python installations on the same system. For example, there may be a default (system) Python, and one that a user themselves has installed, because that one is more recent (or they need a specific version; like, Guido forbid, Python 2, because they have to run an old Python script or use an old Python package).
As a result, and at times with the help of some aliasing, you may end up in a situation where python refers to the system Python, as here: /usr/bin/python. While e.g. python3 may refer to the user installed Python, as here /opt/homebrew/bin/python3. That is still fine, because these are different commands.
Enter pip, the Python package manager: it may be installed for only one of the multiple Python versions, or it may be installed for all of them, but each time with a different name. As a result, pip may install packages for /usr/bin/python, or it may install packages for e.g. /opt/homebrew/bin/python3. Depending on the setup (and on the use of sudo, but more on that below), these packages may end up in different places. And if the two Python versions are different enough, the installed packages will not be compatible, and will also be installed in different subdirectories of the package installation directory (on Linux, it would go in $HOME/.local/lib/pythonx.y, with x.y indicating the different versions. On macOS, somewhere in $HOME/Library. Note that this applies to a user, i.e. local, installation, without sudo).
To complicate things, the python 2-to-3 transition resulted in systems naming executables specifically python3 or pip3, while python would still be Python 2. Now that most Python 2 (system) scripts have been ported to Python 3, that habit has worn off, Python 2 was ditched, and python now refers to Python 3. But you may still be caught in the middle of this, and some names remain. Confusing! (Python 3 should just have been named completely different, for example, monty. That would have avoided a lot of confusion.)
So if you're lucky, everything works. If you're unlucky, the Pip installation installed for the wrong Python, and it's a long search which Python to use (or how to re-install the packages for the correct Python).
python, the executable, does have the option to run a package as a script (to phrase it simply): python -m <package> will do that. That will guarantee that the package/script uses that python executable. Pip is one of those packages that can do that (in fact, the pip or pip3 script is a short wrapper around that functionality).
So using python -m pip install <package> guarantees that a package installed by pip, will be found by python as well. No messing with incompatible python and pip executables.
I don't recommend the use of sudo on a single user system. It may mess up the system, because plenty of people don't know the details of their system. sudo should preferably only be used by system administrators to install software for other users on the system.
What about Pip? Pip has an option, --user, that will install packages in a directory that Python will automatically find, but is user related, in the user's home directory. In fact, recent versions of Pip will automatically use that directory if they find they can't install packages in the normal (system) place. There'll be a note in the output, and then pip implicitly installs with the --user option. This is good in 90% (or more) of the cases, since this is very likely the intention of the person using Pip, and not too many people know about --user.
For (properly installed and activated) virtual environment, this problem doesn't exist. python and pip are unique within that virtual environment. This includes a Conda environment.
And, in fact, the built-in virtual environment creation for Python is another example of a runnable module:
python -m venv <name>
This also guarantees that your virtual environment uses (a copy of) that specific python executable. If you had wanted a virtual environment with another Python version, use e.g.
python3.8 -m venv <name>
(don't forget to activate the relevant v-env afterwards!).
When not activated, you'll be using the default python, but when activated, python3.8. Easy switching!
This doesn't work for all python executables: Python 2 doesn't have the venv package, and python executables on some systems are very barebones, and no venv package will exist. Often, that can be amended (if you are using the system Python), by installing the relevant package: sudo apt install python-venv or something similar(*). (Another case where it's debatable if sudo should actually be used on a single user system, but harder to avoid.)
Note that to a point, Python versions are always distinguishable by using the major.minor version added to the python executable. So, when done well, a system can have python2.7, python3.6, python3.7, python3.8, python3.9 and python3.10 all work independently from each other.
I actually use this to occasionally switch and test versions with pyenv. Just be careful that you don't open another can of worms if you're not used to managing Python versions.
(*) In my option, it's a shame that some systems separate pip and venv from the default Python package, and/or don't install these by default. Installing these by default requires very little extra disk space, and provide a huge convenience. If you want a barebones systems, it should be a simple uninstall of system packages like python-pip and python-venv, instead of the other way around.
Caveat: while this technically applies everywhere, IDEs such as Visual Studio Code may still cause new people problems, because they have both a terminal, people may use another terminal, and then there is the configuration/settings themselves, where you can specify a Python interpreter (and sometimes a Pip executable as well).
The above solution should work fine on a command line by itself. Within an IDE, I can't guarantee this, but do check your settings for both Python and Pip, and beware of virtual environments set up (automatically) by your IDE, which may make things even worse, since the v-env will need to be activated before installing packages (and then again in a next session, when running Python).