I notice that if I install a package with optional dependencies (setup.py extras_require):
$ pip install mypackage[optional]==1.0.0
I do not see the optional dependency in the output of pip freeze:
$ pip freeze | grep mypackage
mypackage==1.0.0
However, I'm using a requirements-to-freeze workflow that involves using pip freeze to update the requirements file with exact versions I use in production. So, what I would like to see is:
mypackage[optional]==1.0.0
Is there a way to achieve this?
Update:
Someone pointed out to me that I actually don't need this in the requirements-to-freeze workflow. In that workflow (at least the version I'm using), you first run pip install -r requirements-to-freeze.txt, which has mypackage[option]. This will find the latest version and also install the optional dependencies. Later, when pip freeze is run, the optional dependencies will be in the list. So actually, the fact that mypackage shows up in pip freeze without the square bracket notation is fine. The desired outcome will still be achieved.
According to the pip freeze documentation, there is no option to do such that thing
Related
my requirements.txt:
google-api-python-client>=1.6.2
httplib2==0.9.2
...
In this case google-api-python-client installs the httplib2=0.14.0. But I would be overwritten to the older version on the next line.
Of course, in this case, I could act. But there are some cases that a lib depends on another, that depends on another, etc.
So I'm wondering whether there's a way to tell pip to always keep the most recent version of the libs.
like: pip install -r --keep-most-recent requirements.txt
This would mean that pip should install the latest version of the requirements and their dependencies no matter what the version constraints in requirements.txt are, is that right? Impossible as far as I know.
Not sure exactly what you are trying to achieve but maybe you would prefer working with one of these tools:
poetry update
pip-compile --upgrade
pipenv update
I have a situation where I am doing the following in CI:
pip3 wheel -r requirements.txt
I would like to get a list of all the deps that requires (including transitive ones).
The only way I can think of is to make a throwaway virtual environment pip install from the wheel and then do pip freeze from that virtual environment.
This seems undesirable in a CI process so I am hoping there is a way to remove that pip install step and instead get a list of all the deps directly from the wheel / requirements.txt.
There is no other way because of recursive ("transitive") dependencies. There is no way to query for dependencies remotely so pip has at least to download packages to inspect their list of dependencies.
And where from pip freeze could get a list of installed packages? Your transient virtualenv seems to be the only way.
I want to install the 2.3 branch of PyMC.
I have tried with:
pip install -e git+git:://github.com/pymc-devs/pymc#2.3
and
pip install git+git:://github.com/pymc-devs/pymc#2.3
without luck. Do I need to specify egg information? If so why?
You have too many colons in your URL:
git+git:://github.com/pymc-devs/pymc#2.3
# ----^
Just one colon is required:
pip install git+git://github.com/pymc-devs/pymc#2.3
or, in editable mode:
pip install -e git+git://github.com/pymc-devs/pymc#2.3
Provided you have a Fortran compiler installed that Just Works.
The #egg=<projectname> part is optional and only used to test for dependencies before downloading. It lets pip test if the package is already installed without needing to clone the whole repository.
`pip freeze > requirements.txt`
automatically writes my dependencies in an apparently alphabetically order, like this:-
matplotlib==1.2.0
numpy==1.6.2
pandas==0.9.1
The problem with this is that pip install -r requirements.txt (when I deploy my code with its dependencies listed in requirements.txt) will end up failing because matplotlib needs numpy to be installed first.
How can I ensure that matplotlib is listed after numpy in the requirements.txt file when I pip freeze it?
For your case it does not matter, because pip builds every requirements (calling python setup.py egg_info for each) and then install them all. For your specific case, it does not matter, because numpy is currently required to be installed while building matplotlib.
It is a problem with matplotlib, and they created a proposal to fix it: https://github.com/matplotlib/matplotlib/wiki/MEP11
See comments from this issue at pip issue tracker: https://github.com/pypa/pip/issues/25
This question is a duplicate of Matplotlib requirements with pip install in virtualenv.
You can try command
pip install --no-deps -r requirements.txt
This installs the packages without dependencies and possibly you will get rid above written problems.
A file containing the packages in the desired order can be used like so:
pip freeze -r sorted-package-list.txt > requirements.txt
Where sorted-package-list.txt contains
numpy
matplotlib
Note: Packages not included in the sorted-package-list.txt file are appended at the end of the requirements file.
Example result:
numpy==1.14.1
matplotlib==2.2.3
## The following requirements were added by pip freeze:
pandas==0.23.4
Note that h5py (the HDF5 Python wrapper) has the same problem.
My workaround is to split the output of pip freeze into two: into a short requirements file containing only numpy's version ${NUMPY_REQS}, and a long one ${REQS} containing all other packages. Note the -v switch of the second grep, the "inverse match".
pip freeze | tee >( grep '^numpy' > ${NUMPY_REQS} ) | grep -v '^numpy' > ${REQS}
And then invoke pip install twice (e.g. when installing a virtual env):
# this installs numpy
pip install -r ${NUMPY_REQS}
# this installs everything else, h5py and/or matplotlib are happy
pip install -r ${REQS}
Note that this tee / grep magic combo works on Unix-like systems only. No idea how to achieve the same thing on Windows.
How can I specify optional dependencies in a pip requirements file?
According to the pip documentation this is possible, but the documentation doesn't explain how to do it, and I can't find any examples on the web.
Instead of specifying optional dependencies in the same file as the hard requirements, you can create a optional-requirements.txt and a requirements.txt.
To export your current environment's packages into a text file, you can do this:
pip freeze > requirements.txt
If necessary, modify the contents of the requirements.txt to accurately represent your project's dependencies. Then, to install all the packages in this file, run:
pip install -U -r requirements.txt
-U tells pip to upgrade packages to the latest version, and -r tells it to install all packages in requirements.txt.
In 2015 PEP-0508 defined a way to specify optional dependencies in requirements.txt:
requests[security]
That means that yourpackage needs requests for its security option. You can install it as:
pip install yourpackage[security]
You are misunderstanding the documentation; it's not as clear as it could be. The point in the documentation is that with a requirements file you can feel free to specify your full recommended working set of packages, including both necessary dependencies and optional ones.
You can add comments (lines beginning with #) to distinguish the two to humans, but pip makes no distinction. You can also have two requirements files, as Daniel suggests.