How To Fix pip Install Error: externally-managed-environment

The next Debian/Ubuntu releases will likely no longer allow pip install outside a virtual environment or separate Python installs because of conflicts between pip and the OS package manager. It can still be forced, but that's strongly not recommended.

Software shipped with a Linux distribution can be (quite easily, I might add) broken by installing packages using pip. All the user has to do is install a package (or some of its dependencies) that's newer and backwards-incompatible with a version that was installed from the Linux distribution official repositories. And this isn't even about breaking some application—because Python is so widely used nowadays, users can easily break critical distro packages.

The breakage can occur for both system-wide pip installations (sudo pip install), as well as user installations (pip install --user), but especially the latter, since trying to recover from this might result in removing (using pip, not the distro package manager) packages installed using the Linux distribution's package manager.

For this reason, the next Debian (Debian 12 Bookworm) and Ubuntu (Ubuntu 23.04 Lunar Lobster) releases will likely adopt PEP668 (PEP = Python Enhancement Proposal), marking the Python base environments as "externally managed", and no longer allowing regular pip install usage for both user and system installations. This can still be forced though, and obviously, there are alternatives—see below.

The change is already live in Debian Testing and Ubuntu 23.04 Lunar Lobster (which will have a beta release on March 30, with the final release expected on April 20). There's also a proposal to include this in Fedora 38 (which had a beta release today), but this has not landed in Fedora 38 for now.

I said "likely adopt" because, even though this change is already present in Debian Testing and Ubuntu 23.04 Lunar Lobster, Stefano Rivera, one of the Debian/Ubuntu Python maintainers, mentioned that "if necessary, we can roll back EXTERNALLY-MANAGED in our python3.11 for bookworm’s release, but I’d like to make this happen…". 

[[Edit]] This change has made it into the Ubuntu 23.04 Lunar Lobster release! When trying to install a Python package using pip, users will now see this message: "error: externally-managed-environment" / "This environment is externally managed".

So what are the alternatives to using pip install directly and solve this externally managed environment pip error? Take a look below:

1. If possible, try to install the Python application or library module from the Debian/Ubuntu repositories.

2. For Python applications that aren't available in the Debian/Ubuntu repositories (or for which you want to install a newer version), you can use pipx to install them. This sets up an isolated environment and installs the application and its dependencies into it. 

For example, instead of running:

pip install --user some-python-app

You can use:

pipx install some-python-app

3. For Python library modules that aren't available in Debian (or for which you want to install a newer version), you can use virtualenv (package called virtualenv on Debian/Ubuntu) / venv (package called python3-venv on Debian/Ubuntu), which creates an isolated environment where you can install Python applications and libraries. 

For example, instead of running:

pip install --user some-python-module-or-app #or
pip install --user -r requirements.txt

You can use:

python3 -m venv .venv/some-python-module-or-app
source .venv/some-python-module-or-app/bin/activate
python3 -m pip install some-python-module-or-app #or
python3 -m pip install -r requirements.txt

If needed, the isolated environment can also have access to system Python modules, with the --system-site-packages flag.

4. Install your own Python (from source) in e.g. /usr/local.

I don't care, I want to use pip install like before

Like I mentioned above, if you really need it, you can force install Python packages using pip outside a virtual environment, just like before, but do this at your own risk! You can force install packages using pip by passing the --break-system-packages option to pip, exporting PIP_BREAK_SYSTEM_PACKAGES=1, or adding the following to the ~/.config/pip/pip.conf or /etc/pip.conf file:

[global]
break-system-packages = true