Develop Reference

pip version is old in virtual environment [duplicate]

As part of the compilation step for a new python version, I fetch and run get-pip.py, to have the latest pip installed next to the python executable:
$ /opt/python/3.7.0/bin/python --version
Python 3.7.0
$ /opt/python/3.7.0/bin/pip --version
pip 18.0 from /opt/python/3.7.0/lib/python3.7/site-packages/pip (python 3.7)
I have 25 such versions under /opt/python, although I mostly use the five latest versions of each major.minor version that is not EOL. To setup an invironment I used to run virtualenv or my virtualenvutils with the -p /opt/python/X.Y.Z/bin/python option to get a virtual environment with a specific version.
With Python 3.7 this gives the imp module deprecation warning:
$ virtualenv -p /opt/python/3.7.0/bin/python /tmp/py37virtualenv
Running virtualenv with interpreter /opt/python/3.7.0/bin/python
Using base prefix '/opt/python/3.7.0'
/opt/util/virtualenvutils/lib/python3.6/site-packages/virtualenv.py:1041: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
New python executable in /tmp/py37virtualenv/bin/python
Installing setuptools, pip, wheel...done.
I have little hope this will be solved in virtualenv, as this has had a PendingDeprecationWarning at least since 2014 (as can be seen from the output in this question)
While investigating replacing virtualenv with python -m venv in virtualenvutils, I first created a new venv based virtual environment by hand:
$ /opt/python/3.7.0/bin/python -m venv /tmp/py37venv
$ /tmp/py37venv/bin/pip --version
pip 10.0.1 from /tmp/py37venv/lib/python3.7/site-packages/pip (python 3.7)
That has an old pip version! If you use it, you'll get:
You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command
In the virtual environment created with virtualenv you immediately get the latest version:
$ /tmp/py37virtualenv/bin/pip --version
pip 18.0 from /tmp/py37virtualenv/lib/python3.7/site-packages/pip (python 3.7)
I can run a post-creation step:
/tmp/py37venv/bin/pip install -U --disable-pip-version-check pip
which will take extra time. And if there was a some security update for pip, this would imply running the non-secure version to get a secure version, an ideal point of attack.
From virtualenvutils it is trivial to do the multiple steps to create a pip-less virtualenv and then add pip using get-pip.py. From the command-line this is not so simple:
$ /opt/python/3.7.0/bin/python -m venv --without-pip /tmp/py37venvnopip
$ /tmp/py37venvnopip/bin/python -c "from urllib.request import urlopen; response = urlopen('https://bootstrap.pypa.io/get-pip'); open('/tmp/tmp_get_pip.py', 'w').write(response.read())"
$ /opt/python/3.7.0/bin/python /tmp/tmp_get_pip.py
......
$ /opt/python/3.7.0/bin/pip --version
pip 18.0 from /opt/python/3.7.0/lib/python3.7/site-packages/pip (python 3.7)
What is causing /opt/python/3.7.0/bin/python -m venv to take that old pip version? Is that the version available when 3.7.0 was released?
How can I update my install under /opt/python/3.7.0 in some way so that using /opt/python/3.7.0/bin/python -m venv creates a virtualenv with the latest pip version without reverting to scripts, aliases or using multiple commands? Having the latest pip installed under /opt/python/3.7.0 obviously is not enough.
There are two bundled wheels:
/opt/python/3.7.0/lib/python3.7/ensurepip/_bundled/setuptools-39.0.1-py2.py3-none-any.whl
/opt/python/3.7.0/lib/python3.7/ensurepip/_bundled/pip-10.0.1-py2.py3-none-any.whl
I suspect I need to update those. Is there a better way than updating those by hand? Some option for /some/python -m venv would be nice.
(And running /some/python -m ensurepip --upgrade doesn't do the trick)
Running the deprecated /opt/python/3.7.0/bin/pyvenv has the same old pip version problem.

The trick is not to install the bundled version of pip (which will almost always be out of date), but to use it to install the most current version from the internet.
Standard library venv offers a --without-pip flag that can help here. After creating the virtual environment without pip, you can then you can "execute" ensurepip's wheel directly thanks to Python's zip importer. This is both faster and less hacky than installing pip and then immediately using that same pip installation to uninstall itself and upgrade.
Code speaks louder than words, so here's an example bash function for the process I've described:
# in ~/.bashrc or wherever
function ve() {
local py="python3"
if [ ! -d ./.venv ]; then
echo "creating venv..."
if ! $py -m venv .venv --prompt=$(basename $PWD) --without-pip; then
echo "ERROR: Problem creating venv" >&2
return 1
else
local whl=$($py -c "import pathlib, ensurepip; whl = list(pathlib.Path(ensurepip.__path__[0]).glob('_bundled/pip*.whl'))[0]; print(whl)")
echo "boostrapping pip using $whl"
.venv/bin/python $whl/pip install --upgrade pip setuptools wheel
source .venv/bin/activate
fi
else
source .venv/bin/activate
fi
}
If you prefer the older project virtualenv, it also offers --no-pip, --no-setuptools, and --no-wheel flags to achieve the same on Python 2.7.
Note: Python 3.9+ venv has an --upgrade-deps option to immediately upgrade the pip/setuptools versions after creating an environment, see https://bugs.python.org/issue34556 for more info about that. I don't use this option because it still goes through an unnecessary install/uninstall of the vendored versions, which is inferior to the method of creating an environment with the latest versions directly as shown above.

I use upgrade-ensurepip to update those pip and setuptools wheel files that are part of the ensurepip package. It's not as elegant as being able to upgrade ensurepip via pip, but it's still preferable to doing it manually.
https://pypi.org/project/upgrade-ensurepip/

It is an expected behavior. python -m venv calls python -m ensurepip to install pip and This answer shows that ensurepip would only install the bundled version even with --upgrade option. There isn't any official option to update the bundled pip and setuptools.
Well I have also no good idea to fix this problem as it just is the designed behavior. I would like to give two suggestions:
Use pipenv. It is really good! And it will be the next-generation official package manager in the future(Although there is a big problem related to current Pypi's structure. In short, a package manager can only decide the dependencies with downloading the whole package. This gives a huge difficulty to building dependencies graph.).
Implement your custom EnvBuilder, actually there is an official example about this. And in the example, it also use get-pip.py to install the latest pip.

pip freeze gives different version from pip --version

I am on Windows 10 and just installed Python 3.7.4. I installed virtualenv using pip install virtualenv, and i got the message that i installed 16.7.2 while 19.2.1 is available. I then used python -m pip install --upgrade pip to upgrade.
Here is the issue: when i type pip freeze on Powershell, i am getting this:
virtualenv==16.7.2
but when i type pip --version, i am getting this:
pip 19.2.1 from c:\users\user1\python\lib\site-packages\pip (python 3.7)
Can you please let me know why pip freeze still shows the old version?
Thanks.

pip and virtualenv are 2 different cases where pip is mainly used for installing python packages and virtualenv for utilizing virtual environments while development.
The following command updates only pip not virtual env
python -m pip install --upgrade pip
If you think you still need to upgrade virtualenv (of course if a new version is available). Please follow the following syntax without specifying a version.
pip install [package] --upgrade

Problems of VirtualEnv and pip3

I used VirtualEnv to create a python2 environment without system site packages like this:
virtualenv -p /usr/bin/python2.7 --no-site-packages ENV2.7
And I want to install packages in this environment.
However, I found that my python code is still trying to look for packages out of this environment.
For example, after activate this env, I used:
pip install matplotlib
And in my demo.py, there is
import matplotlib
But this raised an error, and can not find this package
However, when I use python in the terminal and enter the interactive python, import matplotlib dose not raise an error.
Then I started another terminal and tried to install this package out of the environment by pip3:
pip3 install matplotlib
It turned out that my demo.py just work well.
Any idea? Many Thanks!

It sounds like your virtualenv pip version may be using pip3 instead of pip2:
Make sure you are using the correct python version in your project that you mean to, and using the same version of pip in your virtualenv. (Note that you use pip above once, then you used pip3 outside your virtualenv.)
Check your pip version from inside the virtualenv:
workon (your env name)
which pip
pip -V
Output should look something like:
$ which pip
/home/yourname/.virtualenvs/testenv/bin/pip
$ pip -V
pip 9.0.1 from /home/yourname/.virtualenvs/testenv/local/lib/python2.7/site-packages (python 2.7)
It should tell you you're using pip inside your virtualenv, and the correct python version.
If that looks correct, install your packages.
pip install (whatever)
Check they are installed with pip freeze.
Run your project. :)

How to override the pip command to Python3.x instead of Python2.7?

I am using OSX and I have pip installed for both Python3.5 and Python2.7. I know I can run the command pip2 to use Python2 and when I use the command pip3 Python3.x will be used.
The problem is that the default of pip is set to Python2.7 and I want it to be Python3.x.
How can I change that?
edit:
No, I am not running a virtual environment yet. If it was a virtual environment I could just run Python3.x and forget all about Python2.7, unfortunately since OSX requires Python2.7 for it's use I can't do that. Hence why I'm asking this.
Thanks for the answer. I however don't want to change what running python does. Instead I would like to change the path that running pip takes. At the moment pip -V shows me pip 8.1.2 from /Library/Python/2.7/site-packages (python 2.7), but I am looking for pip 8.1.2 from /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages (python 3.5) I am sure there has to be a way to do this. Any ideas?

Run this:
pip3 install --upgrade --force pip
or even more explicit:
python3 -m pip install --upgrade --force pip
This will install pip for Python 3 and make Python 3 version of pip default.
Validate with:
pip -V

I always just run it via Python itself, this way:
python3 -m pip install some_module
or
python2 -m pip install some_module
The -m calls the __main__.py module of a specified package. Pip supports this.

Can't you alias pip='pip3' in your ~/.bash_profile?
In Terminal, run nano ~/.bash_profile, then add a line to the end that reads alias pip='pip3'. This is safe; it won't affect system processes, only your terminal.

For your projects, you should be using a virtualenv.
You can choose which python will be that of the virtualenv at creation time, by specifying it on the command line:
virtualenv -p python3 env
# then
. env/bin/activate
python # ← will run python3
That python interpreter will be the one used when you run python or pip while the virtualenv is active.
Under the hood, activating the virtualenv will:
modify your PATH environment setting so binaries in env/bin
override those from your system.
modify your PYTHONHOME
environment setting so python modules are loaded from env/lib.
So python, pip and any other package you install with pip will be run from the virtualenv, with the python version you chose and the package versions you installed in the virtualenv.
Other than this, running python without using virtualenv will just run the default python of the system, which you cannot usually change as it would break a lot of system scripts.

It works for me:
As super-user
Uninstall pip
sudo pip uninstall pip
Install pip
sudo python3 -m pip install --upgrade --force pip
Check install path
sudo pip -V
As local-user
Uninstall pip
pip uninstall pip
Install pip
python3 -m pip install --upgrade --force pip
Check install path
pip -V

Although PEP 394 does not specifically mention pip, it does discuss a number of other Python-related commands (including python itself). The short version is that, for reasons of backwards compatibility, the unversioned commands should refer to Python 2.x for the immediate future on most reasonable systems.
Generally, these aliases are implemented as symbolic links, and you can just flip the symlink to point at the version you want (e.g. with ln -f -s $(which pip3) $(which pip) as root). But it may not be a good idea if you have any software that expects to interact with Python 2 (which may be more than you think since a lot of software interacts with Python).
The saner option is to set up a Virtualenv with Python 3. Then, within the Virtualenv, all Python-related commands will refer to 3.x instead of 2.x. This will not break the system, unlike the previous paragraph which could well break things.

Since you have specified in the comments you want syntax like pip install [package] to work, here is a solution:
Install setuptools for Python3: apt-get install python3-setuptools
Now pip for Python3 could be installed by: python3 -m easy_install pip
Now you can use pip with the specific version of Python to
install package for Python 3 by: pip-3.2 install [package]

Why not just repoint the link /bin/python to python3? It seems like the easiest solution. Especially if you want it for all users of your system.

Pip suddenly using wrong version of Python

Having a weird problem with pip on os x.
As far as I can recall (and a quick look at my .bash_history seems to confirm) I have not made any recent changes to my configuration. Alas, the pip command seems to be suddenly using a different version of python than it was previously. Up until now I was using the command pip to manage my python2 libraries and pip3 to manage by python3 libraries. Suddenly, any attempts at running pip install fails with errors like missing parenthesis around print statements.
Here is the result of a few commands I attempted to figure out the problem:
which pip > /usr/local/bin/pip
which pip3 > /usr/local/bin/pip3
which python > /usr/local/bin/python
python version > Python 2.7.11
pip --version > pip 8.1.1 from /usr/local/lib/python3.5/site-packages (python 3.5)
So for some reason the pip command seems to be running from the PyPi2 database but in python3 now? Any ideas how to fix this?

I run with multiple Python versions and thus multiple pip versions as well.
Everytime, however, you update pip, you'll replace the standard pip command with the version you updated. So even pip3 install --upgrade pip will put a /usr/local/bin/pip in your system, messing up the Python 2 version.
Instead, I run pip as an (executable) module:
python3 -m pip search <package>
or
python2 -m pip search <package>
or even
python3.5 -m pip search <package>
This guarantees that your pip version always matches the Python version you want to use it for. It's somewhat longer to type, but I prefer the expliciteness of it (which, I guess, follows the Zen of Python).
Note that updating pip:
python3.5 -m pip install --upgrade pip
will still install a Python 3.5 version in /usr/local/bin/pip, but I'm simply ignoring that. Just beware of (shell) scripts that execute pip directly.

Find absolute path to Python you'd like to use:
which python
Open your default pip executable script:
vi $(which pip)
You will see a shebang line at the top which may point to wrong Python (i had that once too).
Point to the Python you want (see step 1), e.g.:
#!/usr/local/bin/python3.7

Try setting aliases by running the following commands in Terminal,
alias pip="/usr/local/bin/pip"
alias pip2="/usr/local/bin/pip"
alias pip3="/usr/local/bin/pip3"
If this solves your problem then you need to add the aliases in your bash profile.
Look How do I create a Bash alias? for more info.
Alternatively, you have to reinstall pip using python2 get-pip.py first and then python3 get-pip.py get-pip.py can be downloaded here https://bootstrap.pypa.io/get-pip.py

I had exactly the same problem!
I reinstall python2 by brew brew reinstall python#2
after reinstall, pip install packagename works!

None of these worked for me so what I did was navigate to
C:\Users(User)\AppData\Local\Programs\Python\
and deleted all the old python versions I wasn't using. (Worked)