I recently switched to Apple Silicon and am trying to now install all packages in a Python project in the requirements.txt. However, since the architecture is different, I have trouble satisfying all requirements, specifically sub-dependencies.
requirements.txt
ethnicolr==0.8.1
# and many more, but this one is causing issues
Installing it with pip install -r requirements.txt yields
ERROR: Could not find a version that satisfies the requirement
tensorflow==2.5.2 (from ethnicolr) (from versions: none)
TensorFlow 2.4.0 is available for this setup, and it should work with ethnicolr. So how do I override the version of the sub-dependency in requirements.txt?
I am using Mambaforge for virtual environments btw. because it is recommended to run a PyData Stack on Apple M1
A way to workaround that is to explicitly set your dependencies and run pip install with a --no-deps parameter.
After setting my requirements.txt as:
ethnicolr==0.8.1
# and many more, but this one is causing issues
tensorflow==2.5.0 # You set this to whatever version you want to
I ran:
> pip install --no-deps -r requirements.txt
Collecting ethnicolr==0.8.1
Using cached ethnicolr-0.8.1-py2.py3-none-any.whl (36.1 MB)
Collecting tensorflow==2.5.0
Using cached tensorflow-2.5.0-cp39-cp39-manylinux2010_x86_64.whl (454.4 MB)
Installing collected packages: tensorflow, ethnicolr
Successfully installed ethnicolr-0.8.1 tensorflow-2.5.0
This implies a clear issue: You'll have to know beforehand which deps your packages use. To easily find out those, you can use pip-tools
You should install a version of ethnicolr that doesn't have that specific requirement for the tensorflow version. pip install ethnicolr==0.2.0could work.
In conda I can have the following environment.yml:
# name of the environment
name: rust_env
# channels from which the dependencies will be sourced. The order defines the priority.
channels:
- anaconda
- conda-forge
- defaults
# conda dependencies
dependencies:
# name==version==hash?
- anyio=3.3.4=py39hcbf5309_1
# ...
- pip:
# regular pip dependency
# name==version
- jupyter-contrib-core==0.3.3
# dependency sourced from git repo
# name#git+link
- nbconvert#git+https://github.com/JohnScience/nbconvert.git#0b84bb8c3d94cfa229d12c1069e426d907e98d05
Related
I'm experimenting with my python package. I use anaconda and, inside my environment, I have packages that I need already installed:
conda list | egrep "rdkit|numpy"
returns
numpy 1.19.2 py39h89c1606_0
numpy-base 1.19.2 py39h2ae0177_0
rdkit 2022.03.1b1.0 py39he30056e_1 rdkit/label/beta
My setup.py has two dependencies without specifying versions:
install_requires=[
'rdkit',
'numpy',
],
But when I run pip install . it installs another version of rdkit
Collecting rdkit
Using cached rdkit-2022.3.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.9 MB)
The question is, how to prevent pip from such behavior? I want something like: if a package has already been installed, whatever the version, just keep it and do nothing.
You could try:
pip install --no-cache-dir
to avoid cached versions.
Check this link, it might have what you need:
pip uses incorrect cached package version, instead of the user-specified version
Trying to pip install a repo's specific branch. Google tells me to
pip install https://github.com/user/repo.git#branch
The branch's name is issue/34/oscar-0.6 so I did pip install https://github.com/tangentlabs/django-oscar-paypal.git#/issue/34/oscar-0.6 but its returning a 404.
How do I install this branch?
Prepend the url prefix git+ (See VCS Support):
pip install git+https://github.com/tangentlabs/django-oscar-paypal.git#issue/34/oscar-0.6
And specify the branch name without the leading /.
Using pip with git+ to clone a repository can be extremely slow (test with https://github.com/django/django#stable/1.6.x for example, it will take a few minutes). The fastest thing I've found, which works with GitHub and BitBucket, is:
pip install https://github.com/user/repository/archive/branch.zip
which becomes for Django master:
pip install https://github.com/django/django/archive/master.zip
for Django stable/1.7.x:
pip install https://github.com/django/django/archive/stable/1.7.x.zip
With BitBucket it's about the same predictable pattern:
pip install https://bitbucket.org/izi/django-admin-tools/get/default.zip
Here, the master branch is generally named default.
This will make your requirements.txt installing much faster.
Some other answers mention variations required when placing the package to be installed into your requirements.txt. Note that with this archive syntax, the leading -e and trailing #egg=blah-blah are not required, and you can just simply paste the URL, so your requirements.txt looks like:
https://github.com/user/repository/archive/branch.zip
Instructions to install from private repo using ssh credentials:
$ pip install git+ssh://git#github.com/myuser/foo.git#my_version
To install a package from a subdirectory, say stackoverflow
$ pip install git+ssh://git#github.com/myuser/foo.git#my_version#subdirectory=stackoverflow
https://pip.pypa.io/en/stable/topics/vcs-support/
Just to add an extra, if you want to install it in your pip file it can be added like this:
-e git+https://github.com/tangentlabs/django-oscar-paypal.git#issue/34/oscar-0.6#egg=django-oscar-paypal
It will be saved as an egg though.
This worked like charm:
pip3 install git+https://github.com/deepak1725/fabric8-analytics-worker.git#develop
Where :
develop: Branch
fabric8-analytics-worker.git : Repo
deepak1725: user
You used the egg files install procedure.
This procedure supports installing over git, git+http, git+https, git+ssh, git+git and git+file. Some of these are mentioned in other answers.
It's good. You can use branches, tags, or hashes to install.
Steve_K noted it can be slow to install with "git+" and proposed installing via zip file:
pip install https://github.com/user/repository/archive/branch.zip
Alternatively, I suggest you may install using the .whl file if this exists.
pip install https://github.com/user/repository/archive/branch.whl
It's pretty new format, newer than egg files. It requires wheel and setuptools>=0.8 packages. You can find more in the documentation.
to me your suggestion from question work e.g.
pip install https://github.com/user/repo.git#branch
translating concretely to doing
pip install -U git+https://github.com/moskomule/anatome.git#dev
worked. Perhaps remove the extra / is redundant. My output:
(original_anatome_env) brando~/ultimate-anatome ❯ pip install -U git+https://github.com/moskomule/anatome.git#dev
Collecting git+https://github.com/moskomule/anatome.git#dev
Cloning https://github.com/moskomule/anatome.git (to revision dev) to /private/var/folders/x4/0xq0brj57xz3dbhbmblypbm00000gr/T/pip-req-build-62d_ghd2
Running command git clone -q https://github.com/moskomule/anatome.git /private/var/folders/x4/0xq0brj57xz3dbhbmblypbm00000gr/T/pip-req-build-62d_ghd2
Running command git checkout -b dev --track origin/dev
Switched to a new branch 'dev'
Branch 'dev' set up to track remote branch 'dev' from 'origin'.
Resolved https://github.com/moskomule/anatome.git to commit 4b576e51cb1824a57ea04974e0f92b5a6143294d
Requirement already satisfied: torch>=1.10.0 in /Users/brando/anaconda3/envs/metalearning/envs/original_anatome_env/lib/python3.9/site-packages (from anatome==0.0.6) (1.10.0)
Requirement already satisfied: torchvision>=0.11.1 in /Users/brando/anaconda3/envs/metalearning/envs/original_anatome_env/lib/python3.9/site-packages (from anatome==0.0.6) (0.11.1)
Requirement already satisfied: typing-extensions in /Users/brando/anaconda3/envs/metalearning/envs/original_anatome_env/lib/python3.9/site-packages (from torch>=1.10.0->anatome==0.0.6) (3.10.0.2)
Requirement already satisfied: pillow!=8.3.0,>=5.3.0 in /Users/brando/anaconda3/envs/metalearning/envs/original_anatome_env/lib/python3.9/site-packages (from torchvision>=0.11.1->anatome==0.0.6) (8.4.0)
Requirement already satisfied: numpy in /Users/brando/anaconda3/envs/metalearning/envs/original_anatome_env/lib/python3.9/site-packages (from torchvision>=0.11.1->anatome==0.0.6) (1.21.4)
Building wheels for collected packages: anatome
Building wheel for anatome (setup.py) ... done
Created wheel for anatome: filename=anatome-0.0.6-py3-none-any.whl size=10167 sha256=63b12a36f33deb8313bfe7756be60bd08915b8ba36711be47e292b590df70f61
Stored in directory: /private/var/folders/x4/0xq0brj57xz3dbhbmblypbm00000gr/T/pip-ephem-wheel-cache-rde_ngug/wheels/19/e4/be/01479e8cba62ae8cdcd501cd3bf49e199f2bb94732a6a1b006
Successfully built anatome
Installing collected packages: anatome
Attempting uninstall: anatome
Found existing installation: anatome 0.0.5
Uninstalling anatome-0.0.5:
Successfully uninstalled anatome-0.0.5
Successfully installed anatome-0.0.6
0.6.0 is the dev branch version number and 0.5.0 is the master, so it worked!
For windows & pycharm setup:
If you are using pycharm and If you want to use pip3 install git+https://github.com/...
firstly, you should download git from https://git-scm.com/downloads
then restart pycharm
and you can use pycharm terminal to install what you want
I need to update offline a library in Python.
I have downloaded the library with pip download and then I try to update the library with the command:
pip install --no-index --user --find-links /tmp/pip/ --upgrade Werkzeug==0.15.5
which gives:
Ignoring indexes: https://...
Collecting Werkzeug==0.15.5
Installing collected packages: Werkzeug
Successfully installed Werkzeug-0.11.15
and then the library stays in the same version!
pip freeze | grep Wer
Werkzeug==0.11.15
Any ideas why this happens?
UPDATE: After the comment from #hoefling I rerun with the -vvv option and this is what I got:
pip install --no-index --user --find-links /tmp/pip2/ -vvv Werkzeug==0.15.5
Ignoring indexes: https://pypi:pypi#..../simple/
Collecting Werkzeug==0.15.5
0 location(s) to search for versions of Werkzeug:
Skipping link /tmp/pip2/werk/ (from -f); not a file
Found link file:///tmp/pip2/werk/Werkzeug-0.15.5-py2.py3-none-any.whl, version:
0.15.5
Local files found: /tmp/pip2/werk/Werkzeug-0.15.5-py2.py3-none-any.whl
Using version 0.15.5 (newest of versions: 0.15.5)
Installing collected packages: Werkzeug
Successfully installed Werkzeug-0.11.15
Cleaning up...
Try this command:
pip install Werkzeug-0.15.5.tar.gz
and the result must be like this:
Processing ./Werkzeug-0.15.5.tar.gz
Installing collected packages: Werkzeug
Running setup.py install for Werkzeug ... done
Successfully installed Werkzeug-0.15.5
This behaviour can happen because pip by default works with system Python which is located in /usr/bin/ on Linux. When installing the package, by giving Python --user flag your package is installed in your user's version of Python, probably located somewhere in ~/.local/.
To solve the problem you can install the package to your system Python, which is generally not recommended without --user flag. Another option is to use virtual environments and have the distribution that is made specifically for your project. Currently the recommended way is using venv.
$ python -m venv env
$ source env/bin/activate
(env) $ pip install ... (packages you need to install without --user flag)
(env) $ pip freeze
# should give you the packages you installed
This can help you not only with this example, but it can always keep your system Python installation clean and if you mess something up, you will only mess the environment you are having for specific project.
I type
sudo pip install "line_profiler"
and I get
Downloading/unpacking line-profiler
Could not find a version that satisfies the requirement line-profiler (from versions: 1.0b1, 1.0b2, 1.0b3)
Cleaning up...
No distributions matching the version for line-profiler
Storing debug log for failure in /home/milia/.pip/pip.log
When I search for line_profile using
sudo pip search "line_profiler"
I get:
django-debug-toolbar-line-profiler - A panel for django-debug-toolbar that integrates
information from line_profiler
line_profiler - Line-by-line profiler.
tracerbullet - A line-by-line profiler that doesn't suck.
Somehow the underscore gets turned to "-". How can I bypass that?
The problem is not in the fact that pip converts _ into the - to meet the package naming requirements, but the thing is: the package is in beta state, there is no stable package versions. In other words, there are only beta package version links available on the package PyPI page. As you see, pip sees it:
Could not find a version that satisfies the requirement line-profiler (from versions: 1.0b1, 1.0b2, 1.0b3)
According to the Pre-release Versions documentation page:
Starting with v1.4, pip will only install stable versions as specified
by PEP426 by default. If a version cannot be parsed as a compliant
PEP426 version then it is assumed to be a pre-release.
Pass --pre argument to the pip install:
--pre
Include pre-release and development versions. By default, pip only finds stable versions.
sudo pip install --pre line_profiler
Or, install a specific version:
sudo pip install line_profiler==1.0b3
I installed miniconda and run these:
$ conda install -c anaconda line_profiler
Trying to pip install a repo's specific branch. Google tells me to
pip install https://github.com/user/repo.git#branch
The branch's name is issue/34/oscar-0.6 so I did pip install https://github.com/tangentlabs/django-oscar-paypal.git#/issue/34/oscar-0.6 but its returning a 404.
How do I install this branch?
Prepend the url prefix git+ (See VCS Support):
pip install git+https://github.com/tangentlabs/django-oscar-paypal.git#issue/34/oscar-0.6
And specify the branch name without the leading /.
Using pip with git+ to clone a repository can be extremely slow (test with https://github.com/django/django#stable/1.6.x for example, it will take a few minutes). The fastest thing I've found, which works with GitHub and BitBucket, is:
pip install https://github.com/user/repository/archive/branch.zip
which becomes for Django master:
pip install https://github.com/django/django/archive/master.zip
for Django stable/1.7.x:
pip install https://github.com/django/django/archive/stable/1.7.x.zip
With BitBucket it's about the same predictable pattern:
pip install https://bitbucket.org/izi/django-admin-tools/get/default.zip
Here, the master branch is generally named default.
This will make your requirements.txt installing much faster.
Some other answers mention variations required when placing the package to be installed into your requirements.txt. Note that with this archive syntax, the leading -e and trailing #egg=blah-blah are not required, and you can just simply paste the URL, so your requirements.txt looks like:
https://github.com/user/repository/archive/branch.zip
Instructions to install from private repo using ssh credentials:
$ pip install git+ssh://git#github.com/myuser/foo.git#my_version
To install a package from a subdirectory, say stackoverflow
$ pip install git+ssh://git#github.com/myuser/foo.git#my_version#subdirectory=stackoverflow
https://pip.pypa.io/en/stable/topics/vcs-support/
Just to add an extra, if you want to install it in your pip file it can be added like this:
-e git+https://github.com/tangentlabs/django-oscar-paypal.git#issue/34/oscar-0.6#egg=django-oscar-paypal
It will be saved as an egg though.
This worked like charm:
pip3 install git+https://github.com/deepak1725/fabric8-analytics-worker.git#develop
Where :
develop: Branch
fabric8-analytics-worker.git : Repo
deepak1725: user
You used the egg files install procedure.
This procedure supports installing over git, git+http, git+https, git+ssh, git+git and git+file. Some of these are mentioned in other answers.
It's good. You can use branches, tags, or hashes to install.
Steve_K noted it can be slow to install with "git+" and proposed installing via zip file:
pip install https://github.com/user/repository/archive/branch.zip
Alternatively, I suggest you may install using the .whl file if this exists.
pip install https://github.com/user/repository/archive/branch.whl
It's pretty new format, newer than egg files. It requires wheel and setuptools>=0.8 packages. You can find more in the documentation.
to me your suggestion from question work e.g.
pip install https://github.com/user/repo.git#branch
translating concretely to doing
pip install -U git+https://github.com/moskomule/anatome.git#dev
worked. Perhaps remove the extra / is redundant. My output:
(original_anatome_env) brando~/ultimate-anatome ❯ pip install -U git+https://github.com/moskomule/anatome.git#dev
Collecting git+https://github.com/moskomule/anatome.git#dev
Cloning https://github.com/moskomule/anatome.git (to revision dev) to /private/var/folders/x4/0xq0brj57xz3dbhbmblypbm00000gr/T/pip-req-build-62d_ghd2
Running command git clone -q https://github.com/moskomule/anatome.git /private/var/folders/x4/0xq0brj57xz3dbhbmblypbm00000gr/T/pip-req-build-62d_ghd2
Running command git checkout -b dev --track origin/dev
Switched to a new branch 'dev'
Branch 'dev' set up to track remote branch 'dev' from 'origin'.
Resolved https://github.com/moskomule/anatome.git to commit 4b576e51cb1824a57ea04974e0f92b5a6143294d
Requirement already satisfied: torch>=1.10.0 in /Users/brando/anaconda3/envs/metalearning/envs/original_anatome_env/lib/python3.9/site-packages (from anatome==0.0.6) (1.10.0)
Requirement already satisfied: torchvision>=0.11.1 in /Users/brando/anaconda3/envs/metalearning/envs/original_anatome_env/lib/python3.9/site-packages (from anatome==0.0.6) (0.11.1)
Requirement already satisfied: typing-extensions in /Users/brando/anaconda3/envs/metalearning/envs/original_anatome_env/lib/python3.9/site-packages (from torch>=1.10.0->anatome==0.0.6) (3.10.0.2)
Requirement already satisfied: pillow!=8.3.0,>=5.3.0 in /Users/brando/anaconda3/envs/metalearning/envs/original_anatome_env/lib/python3.9/site-packages (from torchvision>=0.11.1->anatome==0.0.6) (8.4.0)
Requirement already satisfied: numpy in /Users/brando/anaconda3/envs/metalearning/envs/original_anatome_env/lib/python3.9/site-packages (from torchvision>=0.11.1->anatome==0.0.6) (1.21.4)
Building wheels for collected packages: anatome
Building wheel for anatome (setup.py) ... done
Created wheel for anatome: filename=anatome-0.0.6-py3-none-any.whl size=10167 sha256=63b12a36f33deb8313bfe7756be60bd08915b8ba36711be47e292b590df70f61
Stored in directory: /private/var/folders/x4/0xq0brj57xz3dbhbmblypbm00000gr/T/pip-ephem-wheel-cache-rde_ngug/wheels/19/e4/be/01479e8cba62ae8cdcd501cd3bf49e199f2bb94732a6a1b006
Successfully built anatome
Installing collected packages: anatome
Attempting uninstall: anatome
Found existing installation: anatome 0.0.5
Uninstalling anatome-0.0.5:
Successfully uninstalled anatome-0.0.5
Successfully installed anatome-0.0.6
0.6.0 is the dev branch version number and 0.5.0 is the master, so it worked!
For windows & pycharm setup:
If you are using pycharm and If you want to use pip3 install git+https://github.com/...
firstly, you should download git from https://git-scm.com/downloads
then restart pycharm
and you can use pycharm terminal to install what you want