Specify extras_require with pip install -e - python

How can one manage to install extras_requires with pip when installing from a git repository ?
I know that you can do pip install project[extra] when the project is on pypi.
And you have to do pip install -e git+https://github.com/user/project.git#egg=project for a git repo but I didn't manage to find how to link these two options together.

This should work, per example #6
For remote repos:
pip install -e git+https://github.com/user/project.git#egg=project[extra]
And this for local ones (thanks to #Kurt-Bourbaki):
pip install -e .[extra]
As per #Jurt-Bourbaki:
If you are using zsh you need to escape square brackets or use quotes:
pip install -e .\[extra\]
# or
pip install -e ".[extra]"

Important to notice: you should not have whitespaces around or within brackets. I.e. this will work: -e ".[extra1,extra2]" but this won't: -e ". [extra1, extra2]" - and even as a row in requirements.txt file, where it is not so obvious. The worst thing about it is that when you have whitespace, extras are just silently ignored.

It may not be obvious for some users, and wasn't for me, so thought to highlight that extra in the following command
pip install -e ".[extra]"
needs to be replaced by the actual name of the extra requirements.
Example:
You add options.extras_require section to your setup.cfg as follows:
[options.extras_require]
test =
pre-commit>=2.10.1,<3.0
pylint>=2.7.2,<3.0
pytest>=6.2.2,<7.0
pytest-pspec>=0.0.4,<1.0
Then you install the test extra as follows
pip install -e ".[test]"

This also works when installing from a whl file so, for example, you can do:
pip install path/to/myapp-0.0.1-py3-none-any.whl[extra1]
This is very far from clear from the docs, and not particularly intuitive.

Using git + ssh to install packages with extras from private repositories:
pip install -e 'git+ssh://git#github.com/user/project.git#egg=project[extra1,extra2]'

Related

Can I have an editable entry in my requirements.txt?

I have an environment which I previously installed into an editable package:
virtualenv venv
. venv/bin/activate
pip install -e ...
pip freeze | grep <pkg_name>
-e git+ssh://git#bitbucket.org/SPACE/REPO.git#HASH#egg=NAME&subdirectory=PATH
I copeid the pip freeze result to a req.txt file and installed it into a new environment, and it works.
My question is - how can I make it pull the code to build and install, not from a remote server, but from my local project (like done when running pip install -e)
It would obviously only work on my machine, assuming that project still is there, but this is what I want...
According to pip documentation (1, 2), yes, you can have an entry like this in your requirements.txt:
-e git+ssh://git#example.com/repo.git
Also as chepner has pointed out, one could just specify local URL:
-e file:///home/someone/repo
-e file://C:\Users\someone\repo

Fixing python packages mis-installed by `sudo pip` / `sudo pip3` [duplicate]

How do I uninstall all packages installed by pip from my currently activated virtual environment?
I've found this snippet as an alternative solution. It's a more graceful removal of libraries than remaking the virtualenv:
pip freeze | xargs pip uninstall -y
In case you have packages installed via VCS, you need to exclude those lines and remove the packages manually (elevated from the comments below):
pip freeze | grep -v "^-e" | xargs pip uninstall -y
If you have packages installed directly from github/gitlab, those will have #.
Like:
django # git+https://github.com/django.git#<sha>
You can add cut -d "#" -f1 to get just the package name that is required to uninstall it.
pip freeze | cut -d "#" -f1 | xargs pip uninstall -y
This will work for all Mac, Windows, and Linux systems.
To get the list of all pip packages in the requirements.txt file (Note: This will overwrite requirements.txt if exist else will create the new one, also if you don't want to replace old requirements.txt then give different file name in the all following command in place requirements.txt).
pip freeze > requirements.txt
Now to remove one by one
pip uninstall -r requirements.txt
If we want to remove all at once then
pip uninstall -r requirements.txt -y
If you're working on an existing project that has a requirements.txt file and your environment has diverged, simply replace requirements.txt from the above examples with toberemoved.txt. Then, once you have gone through the steps above, you can use the requirements.txt to update your now clean environment.
And For single command without creating any file (As #joeb suggested).
pip uninstall -y -r <(pip freeze)
I wanted to elevate this answer out of a comment section because it's one of the most elegant solutions in the thread. Full credit for this answer goes to #joeb.
pip uninstall -y -r <(pip freeze)
This worked great for me for the use case of clearing my user packages folder outside the context of a virtualenv which many of the above answers don't handle.
Edit: Anyone know how to make this command work in a Makefile?
Bonus: A bash alias
I add this to my bash profile for convenience:
alias pipuninstallall="pip uninstall -y -r <(pip freeze)"
Then run:
pipuninstallall
Alternative for Pipenv
If you are using pipenv, you can run:
pipenv uninstall --all
Alternative for Poetry
If you are using Poetry, run:
poetry env remove --python3.9
(Note that you need to change the version number there to match whatever your Python version is.)
This works with the latest. I think it's the shortest and most declarative way to do it.
virtualenv --clear MYENV
But why not just delete and recreate the virtualenv?
Immutability rules. Besides it's hard to remember all those piping and grepping the other solutions use.
Other answers that use pip list or pip freeze must include --local else it will also uninstall packages that are found in the common namespaces.
So here are the snippet I regularly use
pip freeze --local | xargs pip uninstall -y
Ref: pip freeze --help
I managed it by doing the following:
Create the requirements file called reqs.txt with currently installed packages list
pip freeze > reqs.txt
Then uninstall all the packages from reqs.txt
# -y means remove the package without prompting for confirmation
pip uninstall -y -r reqs.txt
I like this method as you always have a pip requirements file to fall back on should you make a mistake. It's also repeatable, and it's cross-platform (Windows, Linux, MacOs).
Method 1 (with pip freeze)
pip freeze | xargs pip uninstall -y
Method 2 (with pip list)
pip list | awk '{print $1}' | xargs pip uninstall -y
Method 3 (with virtualenv)
virtualenv --clear MYENV
On Windows if your path is configured correctly, you can use:
pip freeze > unins && pip uninstall -y -r unins && del unins
It should be a similar case for Unix-like systems:
pip freeze > unins && pip uninstall -y -r unins && rm unins
Just a warning that this isn't completely solid as you may run into issues such as 'File not found' but it may work in some cases nonetheless
EDIT: For clarity: unins is an arbitrary file which has data written out to it when this command executes: pip freeze > unins
That file that it written in turn is then used to uninstall the aforementioned packages with implied consent/prior approval via pip uninstall -y -r unins
The file is finally deleted upon completion.
I use the --user option to uninstall all the packages installed in the user site.
pip3 freeze --user | xargs pip3 uninstall -y
For Windows users, this is what I use on Windows PowerShell
pip uninstall -y (pip freeze)
First, add all package to requirements.txt
pip freeze > requirements.txt
Then remove all
pip uninstall -y -r requirements.txt
The quickest way is to remake the virtualenv completely. I'm assuming you have a requirements.txt file that matches production, if not:
# On production:
pip freeze > reqs.txt
# On your machine:
rm $VIRTUALENV_DIRECTORY
mkdir $VIRTUALENV_DIRECTORY
pip install -r reqs.txt
Using virtualenvwrapper function:
wipeenv
See wipeenv documentation
Its an old question I know but I did stumble across it so for future reference you can now do this:
pip uninstall [options] <package> ...
pip uninstall [options] -r <requirements file> ...
-r, --requirement file
Uninstall all the packages listed in the given requirements file. This option can be used multiple times.
from the pip documentation version 8.1
(adding this as an answer, because I do not have enough reputation to comment on #blueberryfields 's answer)
#blueberryfields 's answer works well, but fails if there is no package to uninstall (which can be a problem if this "uninstall all" is part of a script or makefile). This can be solved with xargs -r when using GNU's version of xargs:
pip freeze --exclude-editable | xargs -r pip uninstall -y
from man xargs:
-r, --no-run-if-empty
If the standard input does not contain any nonblanks, do not run the command. Normally, the command is run once even if there
is no input. This option is a GNU extension.
pip3 freeze --local | xargs pip3 uninstall -y
The case might be that one has to run this command several times to get an empty pip3 freeze --local.
Best way to remove all packages from the virtual environment.
Windows PowerShell:
pip freeze > unins ; pip uninstall -y -r unins ; del unins
Windows Command Prompt:
pip freeze > unins && pip uninstall -y -r unins && del unins
Linux:
pip3 freeze > unins ; pip3 uninstall -y -r unins ; rm unins
This was the easiest way for me to uninstall all python packages.
from pip import get_installed_distributions
from os import system
for i in get_installed_distributions():
system("pip3 uninstall {} -y -q".format(i.key))
the easy robust way
cross-platform
and work in pipenv as well is:
pip freeze
pip uninstall -r requirement
by pipenv:
pipenv run pip freeze
pipenv run pip uninstall -r requirement
but won't update piplock or pipfile so be aware
Cross-platform support by using only pip:
#!/usr/bin/env python
from sys import stderr
from pip.commands.uninstall import UninstallCommand
from pip import get_installed_distributions
pip_uninstall = UninstallCommand()
options, args = pip_uninstall.parse_args([
package.project_name
for package in
get_installed_distributions()
if not package.location.endswith('dist-packages')
])
options.yes = True # Don't confirm before uninstall
# set `options.require_venv` to True for virtualenv restriction
try:
print pip_uninstall.run(options, args)
except OSError as e:
if e.errno != 13:
raise e
print >> stderr, "You lack permissions to uninstall this package.
Perhaps run with sudo? Exiting."
exit(13)
# Plenty of other exceptions can be thrown, e.g.: `InstallationError`
# handle them if you want to.
On Windows if your path is configured correctly, you can use:
pip freeze > unins && pip uninstall -y -r unins && del unins
This works on my windows system
pip freeze > packages.txt && pip uninstall -y -r packages.txt && del packages.txt
The first part pip freeze > packages.txt creates a text file with list of packages installed using pip along with the version number
The second part pip uninstall -y -r packages.txt deletes all the packages installed without asking for a confirmation prompt.
The third part del packages.txt deletes the just now created packages.txt.
This is the command that works for me:
pip list | awk '{print $1}' | xargs pip uninstall -y
If you're running virtualenv:
virtualenv --clear </path/to/your/virtualenv>
for example, if your virtualenv is /Users/you/.virtualenvs/projectx, then you'd run:
virtualenv --clear /Users/you/.virtualenvs/projectx
if you don't know where your virtual env is located, you can run which python from within an activated virtual env to get the path
In Command Shell of Windows, the command pip freeze | xargs pip uninstall -y won't work. So for those of you using Windows, I've figured out an alternative way to do so.
Copy all the names of the installed packages of pip from the pip freeze command to a .txt file.
Then, go the location of your .txt file and run the command pip uninstall -r *textfile.txt*
If you are using pew, you can use the wipeenv command:
pew wipeenv [env]
I simply wanted to remove packages installed by the project, and not other packages I've installed (things like neovim, mypy and pudb which I use for local dev but are not included in the app requirements). So I did:
cat requirements.txt| sed 's/=.*//g' | xargs pip uninstall -y
which worked well for me.
Select Libraries To Delete From This Folder:
C:\Users\User\AppData\Local\Programs\Python\Python310\Lib\site-packages
Why not just rm -r .venv and start over?
pip uninstall `pip freeze --user`
The --user option prevents system-installed packages from being included in the listing, thereby avoiding /usr/lib and distutils permission errors.

How to setup pip to download from mirror repository by default?

I am forced to download python packages from local mirror PyPi repository. I do this by using the -i and --trusted-host options. Whole installation command looks like this:
pip install -i https://sampleurl.com/pypi-remote/simple --trusted-host sample.host.com package_name
Having to type in that options each time is kinda annoying though (in reality those are long URL's). I've tried to create get_package.bat file (I'm working on Windows 10) with following content:
pip install -i https://sampleurl.com/pypi-remote/simple --trusted-host sample.host.com "%1"
It works perfectly fine, although when I wanted to execute pip search command, it turned out to be useless since it has hard-coded install command and there is no way to use it with search.
Is there any way in which I can setup pip to download from mirror repository by default, so that I can execute pip install package_name or pip search package_name without any additional options?
Eventually I could try making .bat file that would take 2 parameters like this:
pip %1 -i https://sampleurl.com/pypi-remote/simple --trusted-host sample.host.com "%2"
But I wonder if there's more "elegant" way to do this.
using pip config, on user or global level. I have /etc/pip.conf configured like this:
[global]
index=https://my-company/nexus/repository/pypi-group/pypi
index-url=https://my-company/nexus/repository/pypi-group/simple
trusted-host=my-company
but you can configure this using pip config on user or global level, something like:
pip config --user set global.index https://my-company/nexus/repository/pypi-group/pypi
pip config --user set global.index-url https://my-company/nexus/repository/pypi-group/simple
pip config --user set global.trusted-host my-company
#NOTES
--index-url is used by pip install
--index is used by pip search
Use pip3 config list -v to get list of locations where your pip.conf is located. Then go to one of the location(I prefer user) and add your URL.
The file should look like this, if empty then add the lines.
[global]
index-url=https://pypi.org/simple
extra-index-url=<your_url>
In case if you want pip to look into your URL first then switch the places of url on above options.
[global]
index-url=<your_url>
extra-index-url=https://pypi.org/simple
You could also add the -i and --trusted-host options to your requirements.txt like this:
-i https://sampleurl.com/pypi-remote/simple --trusted-host sample.host.com
numpy==1.23.4
pandas==1.5.1
And then, just do:
pip install -r requirements.txt
which gives:
Looking in indexes: https://sampleurl.com/pypi-remote/simple
Collecting numpy==1.23.4
Downloading numpy-1.23.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
...
I found this solution to work well in case you don't necessary want to create a pip.conf file (for example, in CIs).

pip -U -r requirements.txt with a URL keeps reinstalling

I'm using a beta version of Django which the download page suggests to use a URL.
The requirements.txt entry is simply the URL:
https://www.djangoproject.com/download/1.7b3/tarball/
When I run pip install -U -r requirements.txt it always reinstalls Django. Is there a way to specify the version in the requirements.txt line, e.g. ...tarball/#egg=Django==1.7b3?
I prefer to be at the latest version of each package when developing, so I use -U.
Maybe there is a better way around this?
You should try adding one of these lines into your requirements.txt
-e https://github.com/django/django.git#egg=django
Also point to specific commit
-e https://github.com/django/django.git#b8d255071ead897cf68120cd2fae7c91326ca2cc#egg=django
or tag
-e git+https://github.com/django/django.git#1.7b3
Read the pip's documentation there's a lot of other examples

How to point pip at a Mercurial branch?

I'm trying to install my application via pip to a virtualenv for testing.
Works fine for installing the default or tip like so:
pip install -e hg+https://username#bitbucket.org/username/app_name#egg=app_name
But is there any way to point to a branch, rather than just getting the tip. Not sure if this would be a mercurial thing, bitbucket, or pip.
Bitbucket allows for downloading of a tagged version of the code, but I can only get it to work while logged into the browser. I tried installing from a tag tar.gz like so:
pip install https://username#bitbucket.org/username/app_name/get/bbc4286a75db.tar.gz
but even after entering my password it returns a 401 Unauthorized (Its a Private Repo)
In official pip documentation in section VCS Support:
Mercurial
The supported schemes are: hg+http, hg+https, hg+static-http and
hg+ssh:
-e hg+http://hg.myproject.org/MyProject/#egg=MyProject
-e hg+https://hg.myproject.org/MyProject/#egg=MyProject
-e hg+ssh://hg#myproject.org/MyProject/#egg=MyProject
You can also specify a revision number, a revision hash, a tag name or
a local branch name:
-e hg+http://hg.myproject.org/MyProject/#da39a3ee5e6b#egg=MyProject
-e hg+http://hg.myproject.org/MyProject/#2019#egg=MyProject
-e hg+http://hg.myproject.org/MyProject/#v1.0#egg=MyProject
-e hg+http://hg.myproject.org/MyProject/#special_feature#egg=MyProject
The syntax is the same when specifying repo at the command line
pip install -e hg+http://hg.myproject.org/MyProject/#special_feature#egg=MyProject
and it works when not using -e option starting from version 0.8.2.

Categories

Resources