Develop Reference

Can I have an editable entry in my requirements.txt?

I have an environment which I previously installed into an editable package:
virtualenv venv
. venv/bin/activate
pip install -e ...
pip freeze | grep <pkg_name>
-e git+ssh://git#bitbucket.org/SPACE/REPO.git#HASH#egg=NAME&subdirectory=PATH
I copeid the pip freeze result to a req.txt file and installed it into a new environment, and it works.
My question is - how can I make it pull the code to build and install, not from a remote server, but from my local project (like done when running pip install -e)
It would obviously only work on my machine, assuming that project still is there, but this is what I want...

According to pip documentation (1, 2), yes, you can have an entry like this in your requirements.txt:
-e git+ssh://git#example.com/repo.git
Also as chepner has pointed out, one could just specify local URL:
-e file:///home/someone/repo
-e file://C:\Users\someone\repo

How to install local python packages when building jobs under Github Actions?

I am building a python project -- potion. I want to use Github actions to automate some linting & testing before merging a new branch to master.
To do that, I am using a slight modification of a Github recommended python actions starter workflow -- Python Application.
During the step of "Install dependencies" within the job, I am getting an error. This is because pip is trying to install my local package potion and failing.
The code that is failing if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
The corresponding error is:
ERROR: git+https#github.com:<github_username>/potion.git#82210990ac6190306ab1183d5e5b9962545f7714#egg=potion is not a valid editable requirement. It should either be a path to a local project or a VCS URL (beginning with bzr+http, bzr+https, bzr+ssh, bzr+sftp, bzr+ftp, bzr+lp, bzr+file, git+http, git+https, git+ssh, git+git, git+file, hg+file, hg+http, hg+https, hg+ssh, hg+static-http, svn+ssh, svn+http, svn+https, svn+svn, svn+file).
Error: Process completed with exit code 1.
Most likely, the job is not able install the package potion because it is not able to find it. I installed it on my own computer using pip install -e . and later used pip freeze > requirements.txt to create the requirements file.
Since I use this package for testing therefore I need to install this package so that pytest can run its tests properly.
How can I install a local package (which is under active development) on Github Actions?
Here is part of the Github workflow file python-app.yml
...
steps:
- uses: actions/checkout#v2
- name: Set up Python 3.8
uses: actions/setup-python#v2
with:
python-version: 3.8
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
...
Note 1: I have already tried changing from git+git#github.com:<github_username>... to git_git#github.com/<github_username>.... Pay attention to / instead of :.
Note 2: I have also tried using other protocols such as git+https, git+ssh, etc.
Note 3: I have also tried to remove the alphanumeric #8221... after git url ...potion.git

The "package under test", potion in your case, should not be part of the requirements.txt. Instead, simply add your line
pip install -e .
after the line with pip install -r requirements.txt in it. That installs the already checked out package in development mode and makes it available locally for an import.
Alternatively, you could put that line at the latest needed point, i.e. right before you run pytest.

pip install & uninstall recursively in non-interactive mode

As a part of my bash script, I want to install and uninstall pip dependencies that I have their names in a file in a non-interactive mode. I was able to search around and find these commands:
pip3 uninstall --yes -r host-requirements.txt
pip3 install --no-input -r host-requirements.txt
I wasn't able to find --yes & --no-input options in the help doc of pip, and I'm not sure if they are officially supported.

For uninstall, you can use the --yes or -y flag as described here: https://pip.pypa.io/en/stable/cli/pip_uninstall/
For installation, you can pass a yes | pip install -r requirements.txt as described here: python pip silent install
Hope this helps.

There are more interactive questions that expect other answers than "yes". For instance:
Directory /opt/services/spam/egg already exists, and is not a git clone.
What to do? (i)gnore, (w)ipe, (b)ackup`
In such scenario I found calling echo "i" | pip install ... was sufficient.

A common issue on install is if there is a private repository dependency that has to be resolved and the key of the remote server has to be initially added.
Obtaining file://...
Collecting your_private_package# git+ssh://...
Cloning ssh://****#.../
Running command git clone -q 'ssh://****#.../
The authenticity of host can't be established.
RSA key fingerprint is ...
Are you sure you want to continue connecting (yes/no)?
For this, ssh StrictHostKeyChecking would be temporarily set to no. This can be done at the host or user level at the risk of less security.

Specify extras_require with pip install -e

How can one manage to install extras_requires with pip when installing from a git repository ?
I know that you can do pip install project[extra] when the project is on pypi.
And you have to do pip install -e git+https://github.com/user/project.git#egg=project for a git repo but I didn't manage to find how to link these two options together.

This should work, per example #6
For remote repos:
pip install -e git+https://github.com/user/project.git#egg=project[extra]
And this for local ones (thanks to #Kurt-Bourbaki):
pip install -e .[extra]
As per #Jurt-Bourbaki:
If you are using zsh you need to escape square brackets or use quotes:
pip install -e .\[extra\]
# or
pip install -e ".[extra]"

Important to notice: you should not have whitespaces around or within brackets. I.e. this will work: -e ".[extra1,extra2]" but this won't: -e ". [extra1, extra2]" - and even as a row in requirements.txt file, where it is not so obvious. The worst thing about it is that when you have whitespace, extras are just silently ignored.

It may not be obvious for some users, and wasn't for me, so thought to highlight that extra in the following command
pip install -e ".[extra]"
needs to be replaced by the actual name of the extra requirements.
Example:
You add options.extras_require section to your setup.cfg as follows:
[options.extras_require]
test =
pre-commit>=2.10.1,<3.0
pylint>=2.7.2,<3.0
pytest>=6.2.2,<7.0
pytest-pspec>=0.0.4,<1.0
Then you install the test extra as follows
pip install -e ".[test]"

This also works when installing from a whl file so, for example, you can do:
pip install path/to/myapp-0.0.1-py3-none-any.whl[extra1]
This is very far from clear from the docs, and not particularly intuitive.

Using git + ssh to install packages with extras from private repositories:
pip install -e 'git+ssh://git#github.com/user/project.git#egg=project[extra1,extra2]'

pip -U -r requirements.txt with a URL keeps reinstalling

I'm using a beta version of Django which the download page suggests to use a URL.
The requirements.txt entry is simply the URL:
https://www.djangoproject.com/download/1.7b3/tarball/
When I run pip install -U -r requirements.txt it always reinstalls Django. Is there a way to specify the version in the requirements.txt line, e.g. ...tarball/#egg=Django==1.7b3?
I prefer to be at the latest version of each package when developing, so I use -U.
Maybe there is a better way around this?

You should try adding one of these lines into your requirements.txt
-e https://github.com/django/django.git#egg=django
Also point to specific commit
-e https://github.com/django/django.git#b8d255071ead897cf68120cd2fae7c91326ca2cc#egg=django
or tag
-e git+https://github.com/django/django.git#1.7b3
Read the pip's documentation there's a lot of other examples