I have been having trouble using the requests package in Python 3.6.5. If I run Python 3.5, it works, but not if I run 3.6.5.
I have been trying to remedy this problem by using a virtual environment, but have not had any luck. I have created the virtualenv using this:
mkvirtualenvwrapper -p /usr/local/bin/python3.5 env_test
When I activate that environment and type python -V, it tells me it is using 3.6.5 still. When I type python3.5 -V though, it tells me it is using Python 3.5.5, and which python3.5 says it is sourced in my virtual environment folder.
I guess that virtualenv is installing other versions of Python into the environment as well, and using 3.6.5 as the default, which I don't understand why, but I need it to access python3.5 when I type python. What can I do here?
As you mentioned in your comment, you have an alias set up, where when you type python, it will automatically run python3.6.
To get past this, you have to avoid the alias. You'll have to look up alias in the man of your shell. If it's bash, you could remove the alias by typing unalias python, but chance are, this will only work in the shell you're currently in. Next time you open up a terminal or otherwise have a need to execute your .bashrc and/or .bash_profile, this will likely execute again. You could look for where your alias is set in those files, and remove that line. Or, if the alias gets set in a file outside your control, you could just add unalias ptyhon into your .bashrc.
Or, if you want to just temporarily avoid it,enclose your command in ' or ". 'python' will avoid the alias.
Once the alias is out of your way, what python will likely point you to an area in your virtual env, which should then be a symbolic link to the right version of python.
I am using ZSH. I was having trouble running the Anaconda package manager commands in my terminal. I found that I could add the bin to my PATH using this code:
export PATH="$HOME/anaconda3/bin:$PATH"
That worked. All of the Anaconda and associated commands work and it left me with this PATH:
/Users/USER/anaconda3/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin
But the PATH variable would reset each time I closed the terminal window. After some research, I figured out how modify my .zshrc to make it permanent. I simply pasted this into my .zshrc:
export PATH=/Users/USER/anaconda3/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin
All the commands work and are permanent, but aren't I setting myself up for failure down the road? Everything in the PATH, minus the "/Users/USER/anaconda3/bin" were set automatically. What happens when I update Python, for example? How can I include the "USER/anaconda3/bin" portion of my PATH in my .zshrc without being so explicit?
Your PATH concern is not zsh specific. You could organize things differently.
For example, you could decide to add a directory $HOME/bin/ early in your PATH and put inside that $HOME/bin/ symbolic links to the executables (or scripts) that you want to use. BTW, I recommend to have a short PATH (containing some $HOME/bin/ etc...) since it is more efficient and less messy to understand.
What happens when I update Python, for example?
Let's suppose that Python is installed in your system in /usr/bin/python (and that you use some Linux distribution -or some other Unix- with a good package manager dealing with that;on MacOSX consider homebrew). When that file /usr/bin/python is updated, any future exec of it (e.g. by some shell) will use the new version. Read carefully execve(2).
If ou have several Python-s and the one you want to use is under /Library/Frameworks/Python.framework/Versions/3.6/bin and you upgraded it to some Python 3.7 installed under /Library/Frameworks/Python.framework/Versions/3.7/bin you would need to change your PATH.
If you followed my suggestion, you would (for example) just have a symlink from $HOME/bin/python to /Library/Frameworks/Python.framework/Versions/3.6/bin/python and you would upgrade that symlink when installing Python3.7 using for example:
# remove the old symlink
rm -v $HOME/bin/python
# add the new one
ln -sv /Library/Frameworks/Python.framework/Versions/3.7/bin/python \
$HOME/bin/python
In all cases, you are responsible of having a good enough PATH (and preferably a short one).
This question is not a duplicate.
It pertains not just to renaming a virtual environment, but to actually moving it to a different directory, including, potentially, a different user's directory.
This is not the same as merely renaming a virtual environment, especially to people unfamiliar with virtualenvs.
If I create a virtualenv, and I move it to a different folder, will it still work?
$ virtualenv -p /usr/bin/python3 /home/me/Env/my-python-venv
$ source Env/my-python-venv/bin/activate
(my-python-venv) $
...later that day, the virtual environment MOVED...
(my-python-venv) $ deactivate
$ mkdir -p /home/me/PeskyPartyPEnvs
$ mv /home/me/Env/my-python-venv /home/me/PeskyPartyPEnvs/
Question:
Will this work?
$ source /home/me/PeskyPartyPEnvs/my-python-venv/bin/activate
(my-python-venv) $ /home/me/PeskyPartyPEnvs/my-python-venv/bin/pip3 install foaas
I mean this as less of a question about the wisdom of trying this (unless that wisdom is humorous, of course), and more about whether it's possible. I really want to know whether it's possible to do in Python 3, or whether I just have to suck it up and clone it.
Can I just mv a virtualenv like that without sadness? I do want to avoid sadness.
Yes. It is possible to move it on the same platform. You can use --relocatable on an existing environment.
From --help:
--relocatable -- Make an EXISTING virtualenv environment relocatable.
This fixes up scripts and makes all .pth files relative.
HOWEVER, this does NOT seem to change the activate script, and rather only changes the pip* and easy_install* scripts. In the activate script, the $VIRTUAL_ENV environment variable hardcoded as the original /path/to/original/venv. The $VIRTUAL_ENV variable is used to set the PATH of your active environment too, so it must be changed based on the new location in order to call python and pip etc. without absolute path.
To fix this issue, you can change the $VIRTUAL_ENV environment variable in the activate script (for example using sed), and everything should be good to go.
An example of usage:
$ cd ~/first
$ virtualenv my-venv
$ grep 'VIRTUAL_ENV=' my-venv/bin/activate
VIRTUAL_ENV="/home/username/first/my-venv"
$ virtualenv --relocatable my-venv
Making script my-venv/bin/easy_install relative
Making script my-venv/bin/easy_install-2.7 relative
Making script my-venv/bin/pip relative
Making script my-venv/bin/pip2 relative
Making script my-venv/bin/pip2.7 relative
### Note that `activate` has not been touched
$ mkdir ~/second
$ mv my-venv ~/second
$ cd ~/second
$ grep 'VIRTUAL_ENV=' my-venv/bin/activate
VIRTUAL_ENV=/home/username/first/my-venv
### (This variable hasn't been changed, it still refers to the old, now non-existent directory!)
$ sed -i -e 's|username/first|username/second|' my-venv/bin/activate
## sed can be used to change the path.
## Note that the `-i` (in place) flag won't work on all machines.
$ source my-venv/bin/activate
(my-venv) $ pip install foass
...
(my-venv) $ python
[...]
> import foass
Hooray, now you can install things and load them into your newly located virtual environment.
For Python 3.3+ (with new venv built-in module)
Short Answer (regardless of version):
There's no clean, direct way to move a virtual environment
Just recreate, it's easy!!
Long Answer:
As of Python v3.3, the virtualenv package has become a built-in module named venv.
The --relocatable option mentioned in other answers has not been included in venv, and currently there is no good, safe way that I'm aware of to either rename or relocate a Python virtual environment.
However, it is fairly simple to recreate a virtual environment, with all its currently installed packages. See this answer, or see the section below. During the process you can recreate the new environment in whatever location and with whatever name you desire.
In the answer linked above, he mentions some 3rd party packages which may support direct renames or moves. If you are settled on pursuing a way to move a virtual environment, you could look into if those work with venv as well.
Note: In that answer, it is focused on virtualenv, rather than venv. See next section for how to translate.
venv vs. older virtualenv command syntax
The command to use venv is:
python -m venv
rather than just virtualenv, which installs as a command in the original package. Where "python" refers to however you run your python executable, which could be a variety of things, such as:
python
py or py -3.7 or similar (the Python Launcher for Windows for Python 3.3+ and bundled with Python for Windows, or the py package that can be installed separately for Linux [and MacOS?])
python3 (convention for linux environments that dual install python 2 and 3)
If you are having issues, use the absolute path to the python executable you want to run: e.g. c:\program files\python37\python.exe
If you are unsure which version is being run, you can always python --version to find out.
How to recreate a virtual environment
Creating/recreating a virtual environment is easy and should become second nature after you work with them for a bit. This process mirrors what you would do to distribute your script as a package (with it's dependencies) in the first half, and then what someone would do to install your script/package for further development.
First, get an updated list of what is in the virtual environment. With it active, get the Python version it uses and save out the list of dependencies to a file.
Use python --version with the virtual environment activated to see what version of Python it is using.
This is for clarity - you may want to update the Python version for various reasons - at least to the latest patch version
For example, if the existing venv is using Python v3.7.4, but now v3.7.6 is out - use v3.7.6 instead, which should including only non-breaking security and bug fixes.
Use python -m pip freeze > requirements.txt to create the list of current package dependencies and put them into the requirements.txt file. This command works in Linux or the Git Bash for sure - not 100% sure about Powershell or Command Line in Windows.
Now create a new virtual environment and then add the dependencies from the old one.
Make your new venv.
Make sure you are using the correct version of python that you want to install to the venv.
If you want it to be exactly the same Python version:
While in the old venv, type "python --version", then make sure you create the new venv with that version of the python command.
For the new venv folder entry in the command:
Either add an absolute or relative path to the desired final folder location.
Use python -m venv my_new_venv to create a new virtual environment in the current working directory in a new my_new_venv folder.
The name of the venv folder will be the name of the venv (what shows up in the prompt when it is activated).
Install your dependencies from the requirements.txt file.
python -m pip install -r requirements.txt
You might need to reinstall local packages that are in development mode.
Note, if you ever need to see the specific location a package is installed to, use:
python -m pip list -v
The -v or "verbose" option will add some extra information about each package that is installed, including the path it is installed in. This is useful to make sure you are keeping virtual, user, and system installed packages straight.
At this point you can just delete the old venv folder and all contents. I recommend using a GUI for that - file deletions are often permanent from the linux command line, and a small typo can be bad news.
BUT ALAS:
No, you can't simply mv. There are workarounds, but it might be easier to reinstall.
(my-python-venv)$ /home/me/PeskyPartyPEnvs/pip3 install foaas
zsh: /home/me/PeskyPartyPEnvs/pip3: bad interpreter: /home/me/Env/my-python-venv/bin/python3: no such file or directory
(my-python-venv)$ deactivate
$
... presses enter a lot in frustration, and the following works
$
$
$ pip3 search foaas
Except it is not from my-python-venv, ergo sadness.
Want to mv your virtualenv and use it, otherwise unmodified?
Short Answer:
Well, ya can't.
The --relocatable argument to virtualenv appears to allow you to do this.
YES, YOU CAN! (In windows)
The workaround is easy, just move your virtual environment anywhere then edit activate.bat inside scripts\:
Move to the virtual environment to the desired directory
Right-click and edit activate.bat located at venv_folder\scripts.
Change VIRTUAL_ENV variable from:
set VIRTUAL_ENV=C:\old_directory\venv_name
into
set VIRTUAL_ENV=C:\new_directory\venv_name
Save the edited batch file, and thats it!
NOTE: My solution should work and save windows users setting up new virtual environments, I doubt this will work in other operating system since .bat is from MS-DOS
Yes, this should be possible if you haven't done anything that depends on the current directory of the virtualenv.
However, if you have the choice, the best thing to do is to create new virtualenv and start using the new virtualenv instead. This is the safest choice and least likely to cause issues later.
The documentation does mention that:
Each virtualenv has path information hard-coded into it,
For example, if you have run setvirtualenvproject then it won't be able to switch to the right directory after you run workon ... so in that case you'd need to fix that manually.
In general a virtualenv is little more than a directory with the necessary Python interpreter files plus packages that you need.
Using answers of this and other threads about similar topic, I've made a bash script that, located and executed within the virtualenv directory itself, will help with your virtualenv moves.
After doing virtualenv --relocatable yourenv you'll need to change your VIRTUAL_ENV variable every time you move the directory, so if you don't wan't to change it manually, use this.
#!/bin/bash \n
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
EXISTING=$(grep 'VIRTUAL_ENV=' bin/activate)
NEWDIR=VIRTUAL_ENV=\"$DIR\"
sed -i -e "s|$EXISTING|$NEWDIR|" bin/activate
source bin/activate
I hope it helps.
I wrote a venv-move script.
The first argument is the path to the venv. It deletes any __pycache__ under that path.
It detects the old path, and replaces it with the current path, after confirming. It seems to work okay, even when moving to a different machine of the same type.
It would make sense to re-write this in Python, but the program would be longer.
#!/bin/bash -eu
venv=$1
old=`perl -ne '/VIRTUAL_ENV="(.*?)"/ && print "$1\n"' "$venv/bin/activate"`
new=$PWD/$venv
find "$venv" -name __pycache__ | xargs rm -rf --
files=`fgrep -r "$old" "$venv" -l`
echo "replace $old with $new in:"
echo "$files"
read -p "[yn] ? " YN
[ "$YN" = y ]
sed -i "s:$old:$new:g" $files
TL;DR
virtualenv-clone is included part of virtualenvwrapper
virtualenv-clone /path/to/old/venv /path/to/new/venv
Alternatively
You could also try cpvirtualenv
cpvirtualenv /path/to/old/venv /path/to/new/venv
But cpvirtualenv expects the /path/to/old/venv to be existing inside $WORKON_HOME and if it isn't it fails. Since this calls virtualenv-clone you may as well use that instead; to avoid errors like
mark#Desktop:~/venvs$ cpvirtualenv ./random/ $WORKON_HOME/random
Copying random as /home/mark/.virtualenvs/venvs/random...
Usage: virtualenv-clone [options] /path/to/existing/venv /path/to/cloned/venv
virtualenv-clone: error: src dir '/home/mark/.virtualenvs/venvs/random' does not exist
Warning as per virtualenvwrapper documentation
Copying virtual environments is not well supported. Each virtualenv
has path information hard-coded into it, and there may be cases where
the copy code does not know it needs to update a particular file. Use
with caution.
What does it actually do ?
As per virtualenv-clone PyPi page
A script for cloning a non-relocatable virtualenv.
Virtualenv provides a way to make virtualenv's relocatable which could
then be copied as we wanted. However making a virtualenv relocatable
this way breaks the no-site-packages isolation of the virtualenv as
well as other aspects that come with relative paths and /usr/bin/env
shebangs that may be undesirable.
Also, the .pth and .egg-link rewriting doesn't seem to work as
intended. This attempts to overcome these issues and provide a way to
easily clone an existing virtualenv.
It performs the following:
copies sys.argv[1] dir to sys.argv[2]
updates the hardcoded VIRTUAL_ENV variable in the activate script to
the new repo location. (--relocatable doesn't touch this)
updates the shebangs of the various scripts in bin to the new Python
if they pointed to the old Python. (version numbering is retained.)
it can also change /usr/bin/env python shebangs to be absolute too,
though this functionality is not exposed at present.
checks sys.path of the cloned virtualenv and if any of the paths are
from the old environment it finds any .pth or .egg link files within
sys.path located in the new environment and makes sure any absolute
paths to the old environment are updated to the new environment.
finally it double checks sys.path again and will fail if there are
still paths from the old environment present.
NOTE: This script requires Python 2.7 or 3.4+
I want to install python using homebrew and I noticed there are 2 different formulas for it, one for python 2.x and another for 3.x. The first symlinks "python" and the other uses "python3". so I ran brew install python3.
I really only care about using python 3 so I would like the default command to be "python" instead of having to type "python3" every time. Is there a way to do this? I tried brew switch python 3.3 but I get a "python is not found in the Cellar" error.
You definitely do not want to do this! You may only care about Python 3, but many people write code that expects python to symlink to Python 2. Changing this can seriously mess your system up.
If you're doing this for personal use, don't change the symlink for python. Many of your system programs depend on python pointing to Python 2.6, and you'll break them if you change the symlink.
Instead, pick a shorter name like py and write an alias for it in ~/.bashrc, like alias py=python3.
For example, with testing:
$ echo "alias py=python3" >> ~/.bashrc
$ bash
$ py
>>> 3+3
6
This will give you the convenience without effecting the system or other users.
If you are absolutely sure that you will never want to install / use Python 2, I think you can just create additional symlinks in /usr/local/bin.
Check for everything that links to something in
../Cellar/python3/3.3.0/
and create a link without the 3 at the end, like
python -> ../Cellar/python3/3.3.0/bin/python3
Think twice though, why give up the advantages of having two Pythons side-by-side? Maybe just use the homebrew Python as intended, and create your Python 3 environments with virtualenv.
Yes, far better to use [virtual environments] (https://docs.python.org/3/library/venv.html) for python 3 than mess with the system default
pyvenv /path/to/new/virtual/environment
which will setup python 3 as the default python and also isolate pip installs to that environment which is what you want to do on any project.
As mentioned this is not the best idea. However, the simplest thing to do when necessary is run python3 in terminal. If you need to run something for python3 then run python3
this might be a very simple question but I need your help. I work in a network and I cannot install the programs I want. Anyway, I need to use another version of python, which is installed in the directory /new_version/.
Now, when I type "python" in a shell (I use bash) the command point to the version of python installed in the machine I'm working with. I'd love that when I type "python" this command point to the /new_version/ which I've installed. It would be also better if I can call this "new version" with another command, i.e. python2.
I tried changing the PYTHONPATH in the .bashrc but it didn't work.
alias newpython="/path/to/your/new_version/python"
Add this to your .bashrc, you can then start the new python with newpython and the standard one with python.
Add the line
export PATH=/new_version/:$PATH
to your ~/.bashrc (or ~/.bash_profile) file. Then, whenever you run python, it will find the new version first in your PATH. Note this is PATH, not PYTHONPATH. See the comment by #Aaron.
Edit: Only do it this way if you want python to point to the new version. Use an alias as #cularis suggested if you want to call it something different, or make a symlink with:
ln -s /new_version/python /path/to/a/dir/you/add/to/your/path/newpython
Install virtualenv. With this you can easily set up different Python versions like that:
virtualenv -p /new_version/bin/python
Also, virtualenv enables you to easily install other Python packages via pip install.
And finally, there's a package called tox which can automate testing with different Python versions...