I used pyenv, pyenv-virtualenv for managing python virtual environment.
I have a project working in Python 3.4 virtual environment.
So all installed packages(pandas, numpy etc) are not newest version.
What I want to do is to upgrade Python version from 3.4 to 3.6 as well as upgrade other package version to higher one.
How can I do this easily?
Here is how you can switch to 3.9.0 for a given virtual environement venv-name:
pip freeze > requirements-lock.txt
pyenv virtualenv-delete venv-name
pyenv virtualenv 3.9.0 venv-name
pip install -r requirements-lock.txt
Once everything works correctly you can safely remove the temporary requirements lock file:
rm requirements-lock.txt
Note that using pip freeze > requirements.txt is usually not a good idea as this file is often used to handle your package requirements (not necessarily pip freeze output). It's better to use a different (temporary) file just to be sure.
Use pip freeze > requirements.txt to save a list of installed packages.
Create a new venv with python 3.6.
Install saved packages with pip install -r requirements.txt. When pip founds an universal wheel in its cache it installs the package from the cache. Other packages will be downloaded, cached, built and installed.
OP asked to upgrade the packages alongside Python. No other answers address the upgrade of packages. Lock files are not the answer here.
Save your packages to a requirements file without the version.
pip freeze | cut -d"=" -f1 > requirements-to-upgrade.txt
Delete your environment, create a new one with the upgraded Python version, then install the requirements file.
pyenv virtualenv-delete venv-name
pyenv virtualenv 3.6.8 venv-name
pip install -r requirements-to-upgrade.txt
The dependency resolver in pip should try to find the latest package. This assumes you have the upgrade Python version installed (e.g., pyenv install 3.6.8).
If you use anaconda, just type
conda install python==$pythonversion$
Related
Hey everyone I installed Python3.9 on my Mac using homebrew package manager but now I do not know how to install packages to it for use
Can anyone please tell me, thanks !
You should first do some research on the Python virtual environment, but the final answer to your question is to use pip install for installing Python packages. Be aware that there are other options out there, but pip is the most prevalent.
When you installed python, it has pip installed by default. pip comes with python. You can check pip version by
pip --version
OR
pip3 --version
Now, in order to install any other package, you can install by
pip install <package-name>
It would be better if you install a virtual environment, and install all other packages inside the virtual environment so that you can install packages according to your project requirements and with different versions.
To install virtual environment, do
pip install virtualenv
Once the virtual environment is installed, you can create your virtualenv according to your project requirement by:
virtualenv -p python3 venv
Here venv is your virtualenv name. To activate it,
source venv/bin/actiavte
Now, you can install all your required packages inside this virtualenv by pip3 install <package-name>. This will keep it separated from your system environment.
I think I broke my pip after desperately trying to uninstall Python3.7. (I am on a Mac)
I removed the python folder from the application, removed /Library/Frameworks/Python.framework and removed the folders from /usr/local/bin according to https://osxuninstaller.com/uninstall-guides/properly-uninstall-python-mac/. I then installed python3.6.
src$ pip3.6 --version pip 9.0.1 from
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages
(python 3.6)
but pip itself does no longer work.
src$ which pip
/usr/local/bin/pip
src$ pip --version
-bash: /Library/Frameworks/Python.framework/Versions/3.7/bin/pip: No such file or directory $
It still references the python 3.7 folder. How can I unlink that reference? THe same is btw. true for the package virtualenv that I installed. I can install it with pip3.6 but calling virtualenv still references to the framework folder of 3.7
You need to install pip in that environment. i.e 3.7.
Reason is when you deleted the default install you deleted the dependencies/ libraries that came with it.
Easiest way is to install python afresh via homebrew if you have it.
brew install python
Alternatively you can follow the instructions here to download a file to install pip securely. Homebrew can become messy when managing python environments. I'd recommend looking at something like anaconda if you are going to using different python versions with their own dependencies etc.
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py
https://pip.pypa.io/en/stable/installing/#installing-with-get-pip-py
Other options for fresh install are described in answer here. Hope that helps.
How do I install pip on macOS or OS X?
I have pyenv installed on an up to date debian testing distribution, and have python 2.7.1 in it.
I create a virtualenv for a project with this version, but when activated it gives me the python system version (3.7).
Here is what I did:
$ pyenv virtualenv 2.7.16 my_project-2.7
Requirement already satisfied: virtualenv in /home/user/.local/lib/python2.7/site-packages (15.1.0)
You are using pip version 18.1, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Using base prefix '/usr'
New python executable in /home/user/.pyenv/versions/2.7.16/envs/my_project-2.7/bin/python
Installing setuptools, pip, wheel...
done.
Installing pip from https://bootstrap.pypa.io/get-pip.py...
Collecting pip
Using cached https://files.pythonhosted.org/packages/5c/e0/be401c003291b56efc55aeba6a80ab790d3d4cece2778288d65323009420/pip-19.1.1-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 19.1.1
Uninstalling pip-19.1.1:
Successfully uninstalled pip-19.1.1
Successfully installed pip-19.1.1
$ pyenv activate my_project-2.7
pyenv-virtualenv: prompt changing will be removed from future release. configure `export PYENV_VIRTUALENV_DISABLE_PROMPT=1' to simulate the behavior.
$ python --version
Python 3.7.3
$ pip --version
pip 19.1.1 from /home/user/.pyenv/versions/2.7.16/envs/my_project-2.7/lib/python3.7/site-packages/pip (python 3.7)
It is strange. The virtualenv is correctly set into ~/.pyenv/version/2.7.16, but I can see that pip come from /home/user/.pyenv/versions/2.7.16/envs/my_project-2.7/lib/python3.7/
I think I missed something, or I might have some mess in my python installation, but I can't find out what this python3.7 lib is doing here.
How can I have this virtualenv created for python2.7 ?
Edit
I tried to completely remove pyenv (removing the $(pyenv root) directory), then reinstall it with pyenv-installer but it did not change anything.
I created two virtualenvs for two projects I have (the above 2.7 and one in 3.5).
$ pyenv virtualenvs
2.7.16/envs/my_project-2.7 (created from /usr)
3.6.8/envs/other_project-3.6 (created from /home/user/.pyenv/versions/3.6.8)
my_project-2.7 (created from /usr)
other_project-3.6 (created from /home/user/.pyenv/versions/3.6.8)
I think the problem could be found from this created from /usr but I don't understand what it means. Why was this virtualenv not created from the pyenv python version ? Is it at all relevant ?
It seems that when I create a 3.+ virtual env, venv is used. But on 2.7, virtualenv is used and fails.
WFM as written if I install pyenv and pyenv-virtualenv from scratch according to the linked instructions.
I can see that pip come from /home/user/.pyenv/versions/2.7.16/envs/my_project-2.7/lib/python3.7/
This shows that most probably, that virtual environment already exists and has a wrong Python version.
I can only conjecture how it ended up like this. Most probably, you were messing with that virtualenv or with pyenv envvars by hand; or ran scripts/have something in your bash stratup scripts that do that. E.g. if you are using pyenv-virtualenv commands to manage your environments, you are not supposed to mix them with regular virtualenv commands.
Deleting this virtualenv with pyenv virtualenv-delete my_project-2.7 then recreating it should help.
If not, delete ~/.pyenv and everything relevant from your startup scripts, restart the shell and reinstall pyenv and pyenv-virtualenv.
For a less destructive fixing, you need to carefully inspect your shell environment (envvars, aliases, defined shell functions) and startup scripts for anything that might conflict with pyenv's work; if that doesn't help, debug pyenv machinery (export PS4='+(${BASH_SOURCE}:${LINENO}): ${FUNCNAME[0]:+${FUNCNAME[0]}(): }' helps here very much) to find out what exactly is happening.
The convention to record package requirements for future installation in a python virtual environment is to use pip freeze > requirements.txt and then install using pip install -r requirements.txt
The python interpreter version, however, is not recorded in the requirements.txt file.
Is there a similar convention to record the python version so that the entire virtual env, including the python interpreter used, can be easily rebuilt?
You can use pipenv for virtualenv creation and package management needs, e.g.:
pipenv --python 3.6
pipenv --python 3
See the documentation:
https://pipenv.kennethreitz.org/basics/#specifying-versions-of-python
That way, when you'll recreate the env with pipenv, it'll use the specified Python version and install the required packages.
Such requirements cannot be written in requirements.txt or setup.py: when one runs
pip install -r requirements.txt
pip is already being run with some version of Python.
Python version requirement could be implemented in a script that creates and populates a virtual env.
But best of all it must be written in the docs!
I use python's pip to install packages. Now I want to install scipy, which is already installed on the system, but an old version and on a part of the system where I don't have access to. If I try
pip install scipy
pip rightfully tells me that the package is already installed. If I do
pip install scipy --upgrade
pip tries to upgrade the package but I don't have the access rights to do that.
How can I tell pip to install the package local to my user and to ignore the other scipy package?
I think the best way for avoid override packages it's using a virtual environment. Python has it's own virtual environment and you could install it by:
Python 2.7
> sudo apt-get install python-virtualenv
Python 3
> sudo apt-get install virtualenv
With modern python versions, virtualenv is usually included. Once installed, you could generate a virtual enviroment typing:
> virtualenv venv
This would create a folder in the current directory named venv (you could name it whatever you want). In this package the libraries will be installed.
So, it's time to activate the virtual environment
> source venv/bin/activate
You could verify the environment has been activated by checking the prompt changes. If it happens, all the packages installed using pip will be installed locally.
(venv)> pip install scipy
You could check this website for more info.
Don't forget that you eventually have to clear your $PYTHONPATH variable, in order for it to not pick up other packages.