In order to make packages installed offline, I use the -d (or --download) option to pip install. For instance, pip install --download dependencies -r requirements.txt will download the packages for all required dependencies mentioned in requirements.txt to dependencies dir (but will not install them). Then I use pip install --no-index --find-links dependencies -r requirements.txt to install those downloaded packages without accessing the network.
Most of the time it works fine, but sometimes installation fails with error "Could not find a version that satisfies the requirement xyz". After doing pip install --user xyz --find-links dependencies manually (xyz IS present in the dependencies folder), installation fails with the same "Could not find a version that satisfies the requirement abc" error, but with different package 'abc'. It repeats several times until I manually resolve all failed dependencies.
How could I make run pip install --no-index --find-links dependencies -r requirements.txt without those weird dependency errors not finding packages that are already there?
Make sure of two things:
The pip version is the same in the offline server and in the online one.
To find out: pip -V
To update (if needed): pip install --upgrade pip
The python version is the same in both virtual enviroments or servers.
To find out: python (the header will have the version info)
In my case I was calling pip install --download outside the virtual environment (using default python version - 2.7) and then installing in a virtual environment with python 3 and the error I got was exactly the one you mentioned.
I am trying to install waferslim (A python port of the fitnesse slim server and protocols) into my python environment. Waferslim is in the Python package library PyPI, however when I try to install it using pip it doesn't work, I get a package not found error:
(test) C:\Python27\VirtualEnvs\test\Scripts>pip install waferslim
Downloading/unpacking waferslim
Could not find any downloads that satisfy the requirement waferslim
Cleaning up...
No distributions at all found for waferslim
But if I try installing it with easy_install, it is installed correctly. Why does this happen? Is there a way I can install it using Pip instead of easy_install? I wrote a requirements.txt file that holds all the dependencies for my project, but running pip install -r requirements.txt fails because of waferslim.
I'm working on a Python library, which I have installed in my local virtualenv for testing. I have several dependencies installed with pip. When I do
$ pip freeze > requirements.txt
it adds my current project like so:
-e git+git#github.com:path/to/my/project#somehash#egg=lib-master
Which I have to manually remove - my project doesn't actually depend on itself. Is it possible to pass a parameter to pip that says, "Hey, ignore this/these kinds of packages?"
The simplest solution would be to pipe the result of pip freeze to grep with -v (invert-match):
pip freeze | grep -v 'project_name' > requirements.txt
Demo:
$ mkvirtualenv test
New python executable in test/bin/python
Installing Setuptools...done.
Installing Pip...done.
(test)$ pip freeze
wsgiref==0.1.2
(test)$ pip install requests
Downloading/unpacking requests
Downloading requests-2.2.1.tar.gz (421kB): 421kB downloaded
Running setup.py egg_info for package requests
Installing collected packages: requests
Running setup.py install for requests
Successfully installed requests
Cleaning up...
(test)$ pip freeze
requests==2.2.1
wsgiref==0.1.2
(test)$ pip freeze | grep -v 'requests'
wsgiref==0.1.2
(test)$ pip freeze | grep -v 'requests' > requirements.txt
(test)$ cat requirements.txt
wsgiref==0.1.2
Also see: Negative matching using grep (match lines that do not contain foo).
Hope that helps.
This is wrong question. You should not try to establish requirements based on what you have installed in the moment. Your project should specify its requirements and based on this information (and requirements of requirements) the final set of requirements should be calculated. Please note, that development requirements should be specified as such so that they could be installed separately as needed – see Setuptools “development” Requirements and How to customize a requirements.txt for multiple environments?.
Unfortunately pip is not able to calculate this yet. You can calculate this using pip-tools which will write the result into requirements.txt file. See my answer for complete example how to use pip-tools.
I'm new to virtualenv (on windows). I'm trying to use pip (1.5) install a local wheel file, but it is failing.
The command is:
pip install --no-index -f C:/Users/<User>/Download openpyxl
In the pip.log, I can see where it finds the correct file, but then doesn't try to install it:
Skipping link file:///C:/Users/<User>/Download/openpyxl-1.7.0-py2.py3-none-any.whl; unknown archive format: .whl
I have wheel (version 0.22) install globally as well as in the virtual environment. Any idea how I can get .whl to be a recognized format?
It appears wheel support is disabled.
Make sure that you have setuptools version 0.8 or newer installed, and that the use-wheel option is not set to false in $HOME/.pip/pip.conf.
Upgrading setuptools is easy enough if pip is already working:
pip install --upgrade setuptools
but note that older virtualenv versions can depend on older setuptools versions; you'll need to make sure that virtualenv is also up to date.
I have bumped into the same problem with wheel when downloaded requirements with:
pip install --download /pip_mirror six django_debug_toolbar
dir2pi /pip_mirror/
and tried to install them with:
pip install six-1.7.3-py2.py3-none-any.whl
Even though there is no any config at $HOME/.pip/pip.conf and
$ easy_install --version
setuptools 5.4.1
I still get:
unknown archive format: .whl
I have managed to avoid the problem by adding --no-use-wheel like this, so got only tar.gz files (instead of .whl)
pip install --no-use-wheel --download /pip_mirror six django_debug_toolbar
dir2pi /pip_mirror/
After this pip install --index-url=file:///pip_mirror/simple/ six went without any problems
Why does pip list generate a more comprehensive list than pip freeze?
$ pip list
feedparser (5.1.3)
pip (1.4.1)
setuptools (1.1.5)
wsgiref (0.1.2)
$ pip freeze
feedparser==5.1.3
wsgiref==0.1.2
Pip's documentation states:
freeze
Output installed packages in requirements format.
list
List installed packages.
What is a "requirements format"?
One may generate a requirements.txt via:
$ pip freeze > requirements.txt
A user can use this requirements.txt file to install all the dependencies. For instance:
$ pip install -r requirements.txt
The packages need to be in a specific format for pip to understand, such as:
# requirements.txt
feedparser==5.1.3
wsgiref==0.1.2
django==1.4.2
...
That is the "requirements format".
Here, django==1.4.2 implies install django version 1.4.2 (even though the latest is 1.6.x).
If you do not specify ==1.4.2, the latest version available would be installed.
You can read more in "Virtualenv and pip Basics",
and the official "Requirements File Format" documentation.
To answer the second part of this question, the two packages shown in pip list but not pip freeze are setuptools (which is easy_install) and pip itself.
It looks like pip freeze just doesn't list packages that pip itself depends on. You may use the --all flag to show also those packages.
From the documentation:
--all
Do not skip these packages in the output: pip, setuptools, distribute, wheel
The main difference is that the output of pip freeze can be dumped into a requirements.txt file and used later to re-construct the "frozen" environment.
In other words you can run:
pip freeze > frozen-requirements.txt on one machine and then later on a different machine or on a clean environment you can do:
pip install -r frozen-requirements.txt
and you'll get the an identical environment with the exact same dependencies installed as you had in the original environment where you generated the frozen-requirements.txt.
Look at the pip documentation, which describes the functionality of both as:
pip list
List installed packages, including editables.
pip freeze
Output installed packages in requirements format.
So there are two differences:
Output format, freeze gives us the standard requirement format that may be used later with pip install -r to install requirements from.
Output content, pip list include editables which pip freeze does not.
pip list shows ALL installed packages.
pip freeze shows packages YOU installed via pip (or pipenv if using that tool) command in a requirements format.
Remark below that setuptools, pip, wheel are installed when pipenv shell creates my virtual envelope. These packages were NOT installed by me using pip:
test1 % pipenv shell
Creating a virtualenv for this project…
Pipfile: /Users/terrence/Development/Python/Projects/test1/Pipfile
Using /usr/local/Cellar/pipenv/2018.11.26_3/libexec/bin/python3.8 (3.8.1) to create virtualenv…
⠹ Creating virtual environment...
<SNIP>
Installing setuptools, pip, wheel...
done.
✔ Successfully created virtual environment!
<SNIP>
Now review & compare the output of the respective commands where I've only installed cool-lib and sampleproject (of which peppercorn is a dependency):
test1 % pip freeze <== Packages I'VE installed w/ pip
-e git+https://github.com/gdamjan/hello-world-python-package.git#10<snip>71#egg=cool_lib
peppercorn==0.6
sampleproject==1.3.1
test1 % pip list <== All packages, incl. ones I've NOT installed w/ pip
Package Version Location
------------- ------- --------------------------------------------------------------------------
cool-lib 0.1 /Users/terrence/.local/share/virtualenvs/test1-y2Zgz1D2/src/cool-lib <== Installed w/ `pip` command
peppercorn 0.6 <== Dependency of "sampleproject"
pip 20.0.2
sampleproject 1.3.1 <== Installed w/ `pip` command
setuptools 45.1.0
wheel 0.34.2
My preferred method of generating a requirements file is:
pip list --format=freeze > requirements.txt
This method keeps just the package names and package versions without potentially linking to local file paths which 'pip freeze' alone will sometimes give me. Local file paths in a requirements file make your codebase harder to use for other users and some developers don't know how to fix this so I prefer this method for ease of adoptability.
pip list
List installed packages: show ALL installed packages that even pip installed implictly
pip freeze
List installed packages: - list of packages that are installed using pip command
pip freeze has --all flag to show all the packages.
Other difference is the output it renders, that you can check by running the commands.
For those looking for a solution. If you accidentally made pip requirements with pip list instead of pip freeze, and want to convert into pip freeze format. I wrote this R script to do so.
library(tidyverse)
pip_list = read_lines("requirements.txt")
pip_freeze = pip_list %>%
str_replace_all(" \\(", "==") %>%
str_replace_all("\\)$", "")
pip_freeze %>% write_lines("requirements.txt")