Is setuptools always installed with Python?
I would like to invoke setuptools at runtime outside of a setup.py script.
In other words, should I include setuptools inside my package's requirements.txt and setup.py's install_requires list?
Background
I have noticed when creating a new virtual environment (with Python 3.7.9) that both pip and setuptools are installed by default:
python -m venv venv
source ./venv/bin/activate
pip list
Package Version
---------- -------
pip 20.1.1
setuptools 47.1.0
This is documented here: Creating Virtual Environments:
venv is available by default in Python 3.3 and later, and installs pip and setuptools into created virtual environments in Python 3.4 and later.
Even in a vanilla version of Python 3.7.6 (installed via pyenv), the packages installed by default are both pip and setuptools.
Research
Should setuptools be included in setup_requires in Python?
Informs that setuptools should not be included in setup_requires, but does not talk about it being included in package requirements for runtime use.
TL;DR
Formally, No. Usually, Yes.
the setuptools is not part of the python vanilla codebase, hence not a vanilla modules.
python.org installers or mac homebrew will install it for you, but if someone compile the python by himself or install it on some linux distribution he may not get it and will need to install it by himself.
Related
I'm a strong pyenv and poetry user that's beginning to use pipx and looking to learn more about how it works. Specifically, I'd like to understand how it determines which Python version to use when installing.
I've noticed that it seems to search PATH for existing references to applications when you pipx install <package> to make existing installations accessible globally. This, for instance, works well with pyenv where it pipx will find a version of the package you install across any Python versions installed via pyenv.
pipx install cookiecutter
⚠️ Note: cookiecutter was already on your PATH at ~/.pyenv/shims/cookiecutter
installed package cookiecutter 1.7.3, Python 3.9.6
These apps are now globally available
- cookiecutter
done! ✨ 🌟 ✨
But what if it's a package you've never installed before? And there's a package version compatible with 3.6, 3.7, 3.8 – how will it determine with Python version to use when installing this package?
I've found you can provide the Python version PipX should install the package into by providing it as an argument to the install command (see below).
More specifically, when using pyenv, you can switch to the version you desire to install into and then provide $(which python) to automatically provide the path to that python version...
pyenv shell 3.X.X
pipx install <package-name> --python $(which python)
In a package I'm maintaining, I have a requirements.txt containing an unpinned dependency pandas. Installing my package shows different behavior locally vs on Travis:
Running pip install on the package locally leads to installing Pandas version 0.24.2.
Running the same pip install within Travis CI leads to installing Pandas version 0.25.0rc0 (link to job).
In both cases the cases the versions of pip and setuptools are the same (pip==19.1.1, setuptools==41.0.1).
I'm not specifying an explicit index (and locally I don't have a global pip config), so I'm assuming both should use the standard PyPI index. Looking into the available versions on PyPi I can see that both 0.24.2 and 0.25.0rc0 are available.
How is it possible that I'm getting a different version locally compared to Travis? Does Travis have some kind of global pip config that would e.g. allow RC versions to be installed?
Oops, an error in my Travis config happened to execute python setup.py install where I was expecting to see the output of pip install -e ..
Apparently, python setup.py install has a different logic to interpret unpinned dependencies in a requirements.txt and picks the RC version (see Q/A difference between python setup.py and pip install for general differences). I thought I can support both python setup.py install and pip install, but given the discrepancies in dependency resolution it's probably best to support pip only.
I used pyenv, pyenv-virtualenv for managing python virtual environment.
I have a project working in Python 3.4 virtual environment.
So all installed packages(pandas, numpy etc) are not newest version.
What I want to do is to upgrade Python version from 3.4 to 3.6 as well as upgrade other package version to higher one.
How can I do this easily?
Here is how you can switch to 3.9.0 for a given virtual environement venv-name:
pip freeze > requirements-lock.txt
pyenv virtualenv-delete venv-name
pyenv virtualenv 3.9.0 venv-name
pip install -r requirements-lock.txt
Once everything works correctly you can safely remove the temporary requirements lock file:
rm requirements-lock.txt
Note that using pip freeze > requirements.txt is usually not a good idea as this file is often used to handle your package requirements (not necessarily pip freeze output). It's better to use a different (temporary) file just to be sure.
Use pip freeze > requirements.txt to save a list of installed packages.
Create a new venv with python 3.6.
Install saved packages with pip install -r requirements.txt. When pip founds an universal wheel in its cache it installs the package from the cache. Other packages will be downloaded, cached, built and installed.
OP asked to upgrade the packages alongside Python. No other answers address the upgrade of packages. Lock files are not the answer here.
Save your packages to a requirements file without the version.
pip freeze | cut -d"=" -f1 > requirements-to-upgrade.txt
Delete your environment, create a new one with the upgraded Python version, then install the requirements file.
pyenv virtualenv-delete venv-name
pyenv virtualenv 3.6.8 venv-name
pip install -r requirements-to-upgrade.txt
The dependency resolver in pip should try to find the latest package. This assumes you have the upgrade Python version installed (e.g., pyenv install 3.6.8).
If you use anaconda, just type
conda install python==$pythonversion$
I have some problems with jenkins and creating a virtualenv. I'm using the shiningpanda plugin and the "Virtualenv Builder" build step combined with pyenv.
I can install packages with "pip install package" but I cannot install requirements from a requirements file, because the subsequent packages cannot find the installed packages, e.g. numexpr cannot find/import numpy.
As I was typing my question, I found the answer to that problem: The current version (v0.21) of the shiningpanda plugin does NOT support pip's requirements.txt in virtualenv builders.
https://wiki.jenkins-ci.org/display/JENKINS/ShiningPanda+Plugin
Current version (0.23) works in our setup like this (in Build-Virtualenv Builder, with Nature: Shell):
pushd %run_dir%
SET PYTHONPATH=%CD%
python -m pip install --upgrade -r configurations/requirements.txt
This has worked well even if libraries require each other.
So I heard about the proper way to install packages into python by creating a new virtual environment for every project. Being on a mac (10.8) I have installed python3 using Homebrew, then I installed pip and virtualenv on this copy.
Now here comes the problem:
I create a new virtualenv, and activate it using:
virtualenv testing
source testing/bin/activate
When I type
which python
/Users/mik/Desktop/testing/bin/python
But typing
which pip
/usr/local/bin/pip
(learned of this when trying to install a package in the virtual environment, and it installed in the system wide installation in /usr/local/)
Inside the folder testing there is no file referring to pip
Extra Question: How does pip know which python to install the files to, for example pip list (which I believe refers to python 2.7) outputs the names of packages installed on python 3.3
I'll start with the last question as it explains what is happening.
The commands pip and easy_install are python scripts which are made executable on the filesystem. The python they use is the python that the first line tells to run the script. e.g. in /usr/bin/easy_install it is #!/usr/bin/python This will be Apple's python. So easy_install will install the 2.7 version of pip and virtualenv and will ignore your python3.3 setup.
The way to instal into python 3 is to install the 3.3 version of pip and virtualenv, the easiest way would be to install the Homebrew package for them. I think it is easier and less confusing to use just one package manager (Homebrew here) and not two (i.e. Homebrew and python).
You can also install easy_install directly. The way to do this is install the distribute package using python3.3 explicitly.
Python 3.4 will make this much easier as pip will always be available