I have a script that runs pip install -r requirements.txt -t folder and executes scripts in "folder" for me.
Every time I run it pip install runs, collects all the packages and installs them with setup.py. It looks like it's not making any changes for packages that are already there and up to date but it still takes a while to collect them and run setup.py for each one. Often I'm not modifying the packages.
How can I have it skip ones that are already there? is there a way to speed this up apart from just not running pip install every time?
You can apparently select the upgrade strategy like this:
python -m pip install --requirement requirements.txt --upgrade --upgrade-strategy=only-if-needed
Source: Ignoring some requirements when installing pip requirements
In my case it still doesn't completely do what I expect since dependencies of the updated packages will still be reinstalled, disregarding whether they are already installed and the version matches.
But it should do the trick for most cases.
You could take a look at :
-I, --ignore-installed Ignore the installed packages (reinstalling instead).
or
--upgrade-strategy Determines how dependency upgrading should be handled [default: only-if-needed]. "eager" -
dependencies are upgraded regardless of whether the currently
installed version satisfies the requirements of the upgraded
package(s). "only-if-needed" - are upgraded only when they do not
satisfy the requirements of the upgraded package(s).
Never get into this problem before, but
https://pip.pypa.io/en/stable/man/commands/install/?highlight=--ignore-installed
it might help you.
Or you can do a pre-setup script that make some check before.
I use pip to manage my packages, but how can I know a package's newest version?
I use the
pip install --upgrade eventlet
find the result the version did not change. still is eventlet (0.21.0).
But in the GitHub, you see the doc the newest version is v0.22.1.
If a package is not updated on pypi, you cannot pip install packagename to get the latest version. If you absolutely want the latest version you can install it directly from git
git clone <git URL of library>
cd into the directory and install the package:
sudo python setup.py install
if you want to get the list of all the modules that can be updated i.e. the modules with updated available you can use:
pip list --outdated
This will give you the output like
....
ipython (6.2.1) - Latest: 6.3.1 [wheel]
....
allowing you to chose the packages you want to update
I used pyenv, pyenv-virtualenv for managing python virtual environment.
I have a project working in Python 3.4 virtual environment.
So all installed packages(pandas, numpy etc) are not newest version.
What I want to do is to upgrade Python version from 3.4 to 3.6 as well as upgrade other package version to higher one.
How can I do this easily?
Here is how you can switch to 3.9.0 for a given virtual environement venv-name:
pip freeze > requirements-lock.txt
pyenv virtualenv-delete venv-name
pyenv virtualenv 3.9.0 venv-name
pip install -r requirements-lock.txt
Once everything works correctly you can safely remove the temporary requirements lock file:
rm requirements-lock.txt
Note that using pip freeze > requirements.txt is usually not a good idea as this file is often used to handle your package requirements (not necessarily pip freeze output). It's better to use a different (temporary) file just to be sure.
Use pip freeze > requirements.txt to save a list of installed packages.
Create a new venv with python 3.6.
Install saved packages with pip install -r requirements.txt. When pip founds an universal wheel in its cache it installs the package from the cache. Other packages will be downloaded, cached, built and installed.
OP asked to upgrade the packages alongside Python. No other answers address the upgrade of packages. Lock files are not the answer here.
Save your packages to a requirements file without the version.
pip freeze | cut -d"=" -f1 > requirements-to-upgrade.txt
Delete your environment, create a new one with the upgraded Python version, then install the requirements file.
pyenv virtualenv-delete venv-name
pyenv virtualenv 3.6.8 venv-name
pip install -r requirements-to-upgrade.txt
The dependency resolver in pip should try to find the latest package. This assumes you have the upgrade Python version installed (e.g., pyenv install 3.6.8).
If you use anaconda, just type
conda install python==$pythonversion$
pip install --upgrade -r requirements.txt
repeats the installation process for all the previously installed dependencies which can be a pain when I have a huge list of dependencies (like more than 30?)
Isn't there a way to check an updated requirements.txt and install only specific dependencies that have been included into the requirements.txt file since the previous installation attempt?
I find this to be a real shortcoming of pip (or using pip in virtualenv for that matter). Do not like the repetitive installation nature of pip at all.
As mentioned by Piotr in the comments above, if "--upgrade" is not included in the command, already installed python packages are left alone.
I am trying to install version 1.2.2 of MySQL_python, using a fresh virtualenv created with the --no-site-packages option. The current version shown in PyPi is 1.2.3. Is there a way to install the older version? I have tried:
pip install MySQL_python==1.2.2
However, when installed, it still shows MySQL_python-1.2.3-py2.6.egg-info in the site packages. Is this a problem specific to this package, or am I doing something wrong?
TL;DR:
Update as of 2022-12-28:
pip install --force-reinstall -v
For example: pip install --force-reinstall -v "MySQL_python==1.2.2"
What these options mean:
--force-reinstall is an option to reinstall all packages even if they are already up-to-date.
-v is for verbose. You can combine for even more verbosity (i.e. -vv) up to 3 times (e.g. --force-reinstall -vvv).
Thanks to #Peter for highlighting this (and it seems that the context of the question has broadened given the time when the question was first asked!), the documentation for Python discusses a caveat with using -I, in that it can break your installation if it was installed with a different package manager or if if your package is/was a different version.
Original answer:
pip install -Iv (i.e. pip install -Iv MySQL_python==1.2.2)
What these options mean:
-I stands for --ignore-installed which will ignore the installed packages, overwriting them.
-v is for verbose. You can combine for even more verbosity (i.e. -vv) up to 3 times (e.g. -Ivvv).
For more information, see pip install --help
First, I see two issues with what you're trying to do. Since you already have an installed version, you should either uninstall the current existing driver or use pip install -I MySQL_python==1.2.2
However, you'll soon find out that this doesn't work. If you look at pip's installation log, or if you do a pip install -Iv MySQL_python==1.2.2 you'll find that the PyPI URL link does not work for MySQL_python v1.2.2. You can verify this here: http://pypi.python.org/pypi/MySQL-python/1.2.2
The download link 404s and the fallback URL links are re-directing infinitely due to sourceforge.net's recent upgrade and PyPI's stale URL.
So to properly install the driver, you can follow these steps:
pip uninstall MySQL_python
pip install -Iv http://sourceforge.net/projects/mysql-python/files/mysql-python/1.2.2/MySQL-python-1.2.2.tar.gz/download
You can even use a version range with pip install command. Something like this:
pip install 'stevedore>=1.3.0,<1.4.0'
And if the package is already installed and you want to downgrade it add --force-reinstall like this:
pip install 'stevedore>=1.3.0,<1.4.0' --force-reinstall
One way, as suggested in this post, is to mention version in pip as:
pip install -Iv MySQL_python==1.2.2
i.e. Use == and mention the version number to install only that version. -I, --ignore-installed ignores already installed packages.
To install a specific python package version whether it is the first time, an upgrade or a downgrade use:
pip install --force-reinstall MySQL_python==1.2.4
MySQL_python version 1.2.2 is not available so I used a different version. To view all available package versions from an index exclude the version:
pip install MySQL_python==
I believe that if you already have a package it installed, pip will not overwrite it with another version. Use -I to ignore previous versions.
Sometimes, the previously installed version is cached.
~$ pip install pillow==5.2.0
It returns the followings:
Requirement already satisfied: pillow==5.2.0 in /home/ubuntu/anaconda3/lib/python3.6/site-packages (5.2.0)
We can use --no-cache-dir together with -I to overwrite this
~$ pip install --no-cache-dir -I pillow==5.2.0
Since this appeared to be a breaking change introduced in version 10 of pip, I downgraded to a compatible version:
pip install 'pip<10'
This command tells pip to install a version of the module lower than version 10. Do this in a virutalenv so you don't screw up your site installation of Python.
This below command worked for me
Python version - 2.7
package - python-jenkins
command - $ pip install 'python-jenkins>=1.1.1'
I recently ran into an issue when using pip's -I flag that I wanted to document somewhere:
-I will not uninstall the existing package before proceeding; it will just install it on top of the old one. This means that any files that should be deleted between versions will instead be left in place. This can cause weird behavior if those files share names with other installed modules.
For example, let's say there's a package named package. In one of packages files, they use import datetime. Now, in package#2.0.0, this points to the standard library datetime module, but in package#3.0.0, they added a local datetime.py as a replacement for the standard library version (for whatever reason).
Now lets say I run pip install package==3.0.0, but then later realize that I actually wanted version 2.0.0. If I now run pip install -I package==2.0.0, the old datetime.py file will not be removed, so any calls to import datetime will import the wrong module.
In my case, this manifested with strange syntax errors because the newer version of the package added a file that was only compatible with Python 3, and when I downgraded package versions to support Python 2, I continued importing the Python-3-only module.
Based on this, I would argue that uninstalling the old package is always preferable to using -I when updating installed package versions.
There are 2 ways you may install any package with version:-
A). pip install -Iv package-name == version
B). pip install -v package-name == version
For A
Here, if you're using -I option while installing(when you don't know if the package is already installed) (like 'pip install -Iv pyreadline == 2.* 'or something), you would be installing a new separate package with the same existing package having some different version.
For B
At first, you may want to check for no broken requirements.
pip check
2.and then see what's already installed by
pip list
3.if the list of the packages contain any package that you wish to install with specific version then the better option is to uninstall the package of this version first, by
pip uninstall package-name
4.And now you can go ahead to reinstall the same package with a specific version, by
pip install -v package-name==version
e.g. pip install -v pyreadline == 2.*
If you want to update to latest version and you don't know what is the latest version you can type.
pip install MySQL_python --upgrade
This will update the MySQL_python for latest version available, you can use for any other package version.
dependency packaging has had a new release, wherein it has dropped LegacyVersion from its codebase
The quick solution might be pin packaging==21.3