Develop Reference

FileExistsError: [Errno 17] File exists: '/usr/bin/python' -> '/home/had2000/.virtualenvs/cv/bin/python'

When i have python 3.5 and python 3.6 on ubuntu .I entered some alternate commands to use python 3.5 only (when I type python -V and python3 -V the same output is 3.5.2)
And then i install virtualenv and virtualenvwrapper — these packages allow me to create and manage Python virtual environments:
$ sudo pip install virtualenv virtualenvwrapper
$ sudo rm -rf ~/get-pip.py ~/.cache/pip
To finish the install of these tools, I updated our ~/.bashrc file.I added the following lines to your ~/.bashrc :
export WORKON_HOME=$HOME/.virtualenvs
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
source /usr/local/bin/virtualenvwrapper.sh
Next, source the ~/.bashrc file:
$ source ~/.bashrc
And final I created your OpenCV 4 + Python 3 virtual environment:
$ mkvirtualenv cv -p python3
i have created the virtual environment but had some problems in the back end and i guess it was due to the presence of python3.6. In the end i decided to uninstall python 3.6 and rerun the steps above from scratch and had a problem at the last step that I mentioned above.When i enter command "mkvirtualenv cv -p python3" i get an ERROR:
FileExistsError: [Errno 17] File exists: '/usr/bin/python' -> '/home/had2000/.virtualenvs/cv/bin/python'
At the same time when i enter the command "update-alternatives --config python" python3.6 is no longer there,but i get a warning:
update-alternatives: warning: alternative /usr/bin/python3.6 (part of link group python) doesn't exist; removing from list of alternatives
There is 1 choice for the alternative python (providing /usr/bin/python).
Looking forward to your help, thank you

From the commands you've shared, the error arises from the mkvirtualenv cv being run twice - i.e. the environment already exists. To remove the environment you created, do: rmvirtualenv env-name-here which in this case will become rmvirtualenv cv. This shouldn't be done with that environment active, BTW. An alternate route is to delete $WORKON_HOME/env-name-here. By default, $WORKON_HOME is usually .virtualenvs.

Python virtualenv: Why is pip still looking in a non-$path path?

I've been keeping all my python2.7 installs in my ~/.local/ directory that way I don't have to sudo every time I want to do a pip install. I also have $HOME/.local/lib/python2.7/site-packages on my $PYTHONPATH. This has worked well for years but now I find myself needing to run python3 programs more frequently. After much research it seems like virtualenv is the most recommended way to deal with python 2 and 3 on the same system. But I am running into troubles. I can spin up a python3 virtual environment but when I try to install new libs with pip, my old global path (i.e. ~/.local/) is still being searched by pip, which makes sense. However, this is even the case if I remove my ~/.local/bin/ directory from my $PATH and unset my $PYTHONPATH.
Here is are the steps I took:
First check the preliminaries before activating virtualenv. (I'm on Ubuntu 16.04 btw)
maddoxw#firefly:~$ echo $PATH
/usr/local/cuda-8.0/bin:/home/maddoxw/.node_modules_global/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/maddoxw/bin:/home/maddoxw/scripts
maddoxw#firefly:~$ echo $PYTHONPATH
maddoxw#firefly:~$ python --version
Python 2.7.12
maddoxw#firefly:~$ python3 --version
Python 3.5.2
maddoxw#firefly:~$ which pip
Since I removed my ~/.local/bin directory from my path, then I can be certain pip will not be found. Also, $PYTHONPATH is still empty. Now I create my virtualenv:
maddoxw#firefly:~$ mkdir test && cd test/
mkdir: created directory 'test'
maddoxw#firefly:~/test$ python3 -m venv .env
maddoxw#firefly:~/test$ source .env/bin/activate
(.env) maddoxw#firefly:~/test$ echo $PATH
/home/maddoxw/test/.env/bin:/usr/local/cuda-8.0/bin:/home/maddoxw/.node_modules_global/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/maddoxw/bin:/home/maddoxw/scripts
(.env) maddoxw#firefly:~/test$ echo $PYTHONPATH
(.env) maddoxw#firefly:~/test$ which python
/home/maddoxw/test/.env/bin/python
(.env) maddoxw#firefly:~/test$ python --version
Python 3.5.2
(.env) maddoxw#firefly:~/test$ which pip
/home/maddoxw/test/.env/bin/pip
Good. My ~/.local/ is still NOT on my $PATH, $PYTHONPATH is still empty, python points to the correct path and version, and pip is pointing to the correct location. Now lets try to pip install a fresh lib.
(.env) maddoxw#firefly:~/test$ pip install Cython
Requirement already satisfied: Cython in /home/maddoxw/.local/lib/python2.7/site-packages
Why is pip still looking in a non-$PATH path?

First, install pip3 to use with python3. You can install it with the following command and then use pip3 to install your packages.
sudo apt-get install python3-pip

[Solved]
When I initially set up my python2.7 environment way back when, I created for myself a handy little function wrapper around pip so that I wouldn't have to type out the --user every time I wanted to pip install
pip() {
if [ "$1" = "install" -o "$1" = "bundle" ]; then
cmd="$1"
shift
$HOME/.local/bin/pip $cmd --user $#
else
$HOME/.local/bin/pip $#
fi
}
I put this function in ~/.bash.d/bash_functions
and in my ~/.bashrc i added the line,
[ -f ~/.bash.d/bash_functions ] && source ~/.bash.d/bash_functions
So, although I removed $HOME/.local/ from my path. This wrapper function was still being called everytime I fired up a new terminal. Weather or not I was or was not in a virtualenv was irrelevant.
Solution?
Commented out (or delete completely) the function wrapper fixed it.

pyvenv returns non-zero exit status 1 (during the installation of pip stage)

If you should ever encounter the following error when creating a Python virtual environment using the pyvenv command:
user$ pyvenv my_venv_dir
Error: Command '['/home/user/my_venv_dir/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1
... then the answer (below) provides a simple way to work around it, without resorting to setuptools and it's related acrobatics.

Here's an approach that is fairly O/S agnostic...
Both the pyvenv and python commands themselves include a --without-pip option that enable you to work around this issue; without resorting to setuptool or other headaches. Taking note of my inline comments below, here's how to do it, and is very easy to understand:
user$ pyvenv --without-pip ./pyvenv.d # Create virtual environment this way;
user$ python -m venv --without-pip ./pyvenv.d # --OR-- this newer way. Both work.
user$ source ./pyvenv.d/bin/activate # Now activate this new virtual environment.
(pyvenv.d) user$
# Within it, invoke this well-known script to manually install pip(1) into /pyvenv.d:
(pyvenv.d) user$ curl https://bootstrap.pypa.io/get-pip.py | python
(pyvenv.d) user$ deactivate # Next, reactivate this virtual environment,
user$ source ./pyvenv.d/bin/activate # which will now include the pip(1) command.
(pyvenv.d) user$
(pyvenv.d) user$ which pip # Verify that pip(1) is indeed present.
/path/to/pyvenv.d/bin/pip
(pyvenv.d) user$ pip install --upgrade pip # And finally, upgrade pip(1) itself;
(pyvenv.d) user$ # although it will likely be the
# latest version already.
# And that's it!
I hope this helps. \(◠﹏◠)/

2020, on python3.8 on WSL2 (Ubuntu) the following solved this for me:
sudo apt install python3.8-venv

If you have a module whose file is named after a python standard library, in the folder on which you invoke python -m venv venv, then this command will fail without a hint about that.
For instance, you name a file email.py.
What I did to find this was coding a bash script which moves the .py files off the current directory one by one (to a holdspace/ subdir), and try at each move to invoke the venv dir creation. If the python -m venv venv command exits with 0 code, so it's successful and the last file moved was the culprit.
#!/bin/bash
if [ ! -d ./holdspace ]
then
mkdir holdspace/
fi
for file in *.py
do
mv "$file" holdspace/
python -m venv venv >/dev/null 2>&1
if [ $? -eq 0 ]
then
echo "$file was the culprit."
rm -rf venv/
break
else
echo "$file isn't the culprit."
fi
rm -rf venv/
done
mv holdspace/*.py .

The following should fix it
brew update
brew upgrade
brew install zlib

Installing pip on Bluehost

I have successfully installed python 2.7.11 on a shared Bluehost server.
In the home directory I installed get-pip.py When I run that now,
# python get-pip.py
Requirement already up-to-date: pip in ./python/lib/python2.7/site-packages
But when I try to run pip I get,
usernmame#example.com [~]# pip
-bash: pip: command not found
Why is pip not running? How can I check what python packages are installed?
My ~/.bashrc looks like this,
# .bashrc
export PATH=$HOME/python/Python-2.7.11/:$PATH
# User specific aliases and functions
alias mv='mv -i'
alias rm='rm -i'
alias cp='cp -i'
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
Additionally,
# echo $PATH
/usr/local/jdk/bin:/home2/username/python/Python-2.7.11/:/usr/lib64/qt-3.3/bin:/home2/username/perl5/bin:/ramdisk/php/54/bin:/usr/php/54/usr/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11R6/bin:/home2/username/bin
EDIT:
My attempt to add the path to $PATH in ~/.bashrc
export PATH=$HOME/python/Python-2.7.11/:$HOME/python/lib/python2.7/site-packages/:$PATH
But it doesn't work, same error.
Thanks

You can run pip by invoking the -m option when you use python to run pip. Suppose you want to use pip to install a Python package called "package-of-interest" from PyPI. Then, you can install it by running:
python -m pip install package-of-interest

How should a Python virtual environment be created so that it can be archived in a repository for use by multiple people?

I've got a Python virtual environment that is intended to provide an environment for many people.
It is to be stored in SVN in order to ensure that the environment is "frozen" for various releases.
I have attempted to make it "dynamic" in the sense that the environment activation script is made to not contain a hardcoded path and I have attempted to make it relocatable because it is to be used by many people at many different directories.
I have archived the environment because storing it as a directory in the repository would mean that updates to the environment would result in SVN change summaries that would be almost unreadable.
In have split the archive across multiple files in order to respect SVN file size restrictions.
Is this approach robust? Are there likely to be problems when multiple people use it at multiple directories? Is archiving in the way I am doing it a good idea? Are there better approaches to any of the ideas I've expressed?
The specific procedure I'm using is something like the following:
ssh "${USER}"#sern.ch
setupSALTAEnvironment
echo "create Python virtual environment"
virtual_environment_name="virtual_environment"
virtualenv "${virtual_environment_name}"
echo "make activation dynamic"
activate_filename="virtual_environment/bin/activate"
temporary_filename="/tmp/"$(date "+%Y-%m-%dT%H%MZ" --utc)""
cat > "${temporary_filename}" << "EOF"
directory_bin="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
directory_env="$(dirname "${directory_bin}")"
VIRTUAL_ENV="${directory_env}"
EOF
sed -i "/^VIRTUAL_ENV.*/ {
r ${temporary_filename}
d
}" "${activate_filename}"
rm "${temporary_filename}"
echo "activate Python virtual environment"
source "${virtual_environment_name}"/bin/activate
IFS= read -d '' text << "EOF"
import sys
reload(sys)
sys.setdefaultencoding("utf8")
EOF
echo "${text}" > "${virtual_environment_name}"/lib/python2.7/site-packages/sitecustomize.py
echo "install software in Python virtual environment"
pip install docopt
pip install dataset
pip install numpy
pip install matplotlib
pip install pyfiglet
pip install datavision
pip install nodemaster
pip install propyte
pip install git+https://github.com/wdbm/pyprel.git
pip install git+https://github.com/wdbm/pyrecon.git
pip install shijian
pip install technicolor
#pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
#pip install git+git://github.com/google/skflow.git
echo "make Python virtual environment relocatable"
virtualenv --relocatable virtual_environment
echo "archive Python virtual environment with respect to SVN limitations"
tar -czf - virtual_environment/ | split --bytes=50MB - virtual_environment.tgz.
rm -rf virtual_environment