I recently stumbled upon some issue with running coverage measurements within virtual environment. I do not remember similar issues in the past, nor I was able to find solution on the web.
Basically, when I am trying to run test suite in virtualenv, it works fine. But as soon, as I try to do it using coverage, it fails because of lack of modules it requires. Based on some answer on StackOverflow I checked my script and found out that coverage uses different interpreter, even if running from inside the same virtualenv.
Here is how to reproduce it:
$ virtualenv --no-site-packages venv
New python executable in venv/bin/python
Installing Setuptools................................................done.
Installing Pip.......................................................done.
$ source venv/bin/activate
(venv)$ echo 'import sys; print(sys.executable)' > test.py
(venv)$ python test.py
/home/tadeck/testground/venv/bin/python
(venv)$ coverage run test.py
/usr/bin/python
The question is: how to make coverage work with virtual environment seamlessly? I could alter sys.path or install required modules system-wide, but there has to be a cleaner way.
I had to leave my virtualenv after installing coverage and reactivate it to get coverage to work.
[alex#gesa ~]$ virtualenv --no-site-packages venv
[alex#gesa ~]$ source venv/bin/activate
(venv)[alex#gesa ~]$ pip install coverage
(venv)[alex#gesa ~]$ deactivate
[alex#gesa ~]$ source venv/bin/activate
pip install coverage in your new venv
[alex#gesa ~]$ virtualenv venv
[alex#gesa ~]$ source venv/bin/activate
(venv)[alex#gesa ~]$ pip install coverage
(venv)[alex#gesa ~]$ echo 'import sys; print(sys.executable)' > test.py
(venv)[alex#gesa ~]$ python test.py
/home/alex/venv/bin/python
(venv)[alex#gesa ~]$ coverage run test.py
/home/alex/venv/bin/python
(venv)[alex#gesa ~]$
Related
I setup Python 3.6 using pyenv so I could manage multiple Python versions (e.g. 3.7 and 3.8) in the future. I didn't use Homebrew to install Python since it changes the system version. It's my first time to use zsh shell since it's the default shell in Catalina OS. Currently, I use 3.6.8 version for my existing project.
So here's my current setup:
% pyenv versions
result:
system
* 3.6.8 (set by /Users/macbook/.python-version)
3.7.3
% python -V results to Python 3.6.8
% which python results to /Users/macbook/.pyenv/shims/python
% echo $PATH results to /Users/macbook/.pyenv/shims:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
The content of my ~/.zshrc is PATH=$(pyenv root)/shims:$PATH
I created a virtual env using % python -m venv venv, installed all the necessary packages, and when I activate it and get the python path,
(venv) % python -V
Python 3.6.8
(venv) % which python
/Users/macbook/python-project/venv/bin/python
(venv) % echo $PATH
/Users/macbook/python-project/venv/bin:/Users/macbook/.pyenv/shims:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
Finally, when I try to run the app, I always get zsh: abort error:
(venv) % python app.py
zsh: abort python app.py
(venv) % export FLASK_APP=app.py
(venv) % flask run
zsh: abort flask run
I don't know what else is still missing or are there anything wrong with my python path?
Thanks!
After searching through the web, I think this is a common issue with the latest MacOS or Homebrew. This thread fixed the issue.
brew update && brew upgrade && brew install openssl
copy the two files from /usr/local/Cellar/openssl#1.1/1.1.1g to /usr/local/lib/
cd /usr/local/Cellar/openssl#1.1/1.1.1g/
sudo cp libssl.1.1.1.dylib libcrypto.1.1.1.dylib /usr/local/lib/
add symlink to missing openssl libs
cd /usr/local/lib
sudo ln -s libssl.1.1.1.dylib libssl.dylib
sudo ln -s libcrypto.1.1.1.dylib libcrypto.dylib
For me, the below worked:
Python 3.6.9
MacOs Catalina 10.15.7
cd /usr/local/Cellar/openssl#1.1/1.1.1h/
cp lib/libssl.1.1.dylib lib/libcrypto.1.1.dylib /usr/local/lib
cd /usr/local/lib
sudo ln -s libssl.1.1.dylib libssl.dylib
sudo ln -s libcrypto.1.1.dylib libcrypto.dylib
Thanks to the answer by Zhanrah
Cloud9 (an online ide) doesn't seem to support my virtual environment:
me:~/workspace/dir (master) $ source venv/bin/activate
(venv) me:~/workspace/dir (master) $ which python
/usr/bin/python
This same virtual directory worked fine on my local machine:
(venv) me$ which python
/Users/me/dir2/dir/venv/bin/python
How can I fix this?
The following works for me.
sudo apt-get install python3.5-venv
python3.5 -m venv --clear ./mypy3.5/
source ./mypy3.5/bin/activate
It uses the
(mypy3.5) $ which python
/home/ubuntu/mypy3.5/bin/python
But there is a gotcha which might have been your problem. The python3 -m venv uses soft links to how your python resolves in your environment. I had Python 3.3, 3.4 and 3.5 installed in /usr/local so the /usr/local/bin/python3 would change and break my Python3 venv. Note that "python3" is evaluated for the environment not for an absolute path. To be careful, when there are more than one Python 3 on you system, create your virtual environment with an explicit path like the following.
/usr/bin/python3.5 -m venv --clear ./mypy3.5/
source ./mypy3.5/bin/activate
ls -l $(which python3.5)
/home/ubuntu/mypy3.5/bin/python3.5 -> /usr/bin/python3.5*
If you should ever encounter the following error when creating a Python virtual environment using the pyvenv command:
user$ pyvenv my_venv_dir
Error: Command '['/home/user/my_venv_dir/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1
... then the answer (below) provides a simple way to work around it, without resorting to setuptools and it's related acrobatics.
Here's an approach that is fairly O/S agnostic...
Both the pyvenv and python commands themselves include a --without-pip option that enable you to work around this issue; without resorting to setuptool or other headaches. Taking note of my inline comments below, here's how to do it, and is very easy to understand:
user$ pyvenv --without-pip ./pyvenv.d # Create virtual environment this way;
user$ python -m venv --without-pip ./pyvenv.d # --OR-- this newer way. Both work.
user$ source ./pyvenv.d/bin/activate # Now activate this new virtual environment.
(pyvenv.d) user$
# Within it, invoke this well-known script to manually install pip(1) into /pyvenv.d:
(pyvenv.d) user$ curl https://bootstrap.pypa.io/get-pip.py | python
(pyvenv.d) user$ deactivate # Next, reactivate this virtual environment,
user$ source ./pyvenv.d/bin/activate # which will now include the pip(1) command.
(pyvenv.d) user$
(pyvenv.d) user$ which pip # Verify that pip(1) is indeed present.
/path/to/pyvenv.d/bin/pip
(pyvenv.d) user$ pip install --upgrade pip # And finally, upgrade pip(1) itself;
(pyvenv.d) user$ # although it will likely be the
# latest version already.
# And that's it!
I hope this helps. \(◠﹏◠)/
2020, on python3.8 on WSL2 (Ubuntu) the following solved this for me:
sudo apt install python3.8-venv
If you have a module whose file is named after a python standard library, in the folder on which you invoke python -m venv venv, then this command will fail without a hint about that.
For instance, you name a file email.py.
What I did to find this was coding a bash script which moves the .py files off the current directory one by one (to a holdspace/ subdir), and try at each move to invoke the venv dir creation. If the python -m venv venv command exits with 0 code, so it's successful and the last file moved was the culprit.
#!/bin/bash
if [ ! -d ./holdspace ]
then
mkdir holdspace/
fi
for file in *.py
do
mv "$file" holdspace/
python -m venv venv >/dev/null 2>&1
if [ $? -eq 0 ]
then
echo "$file was the culprit."
rm -rf venv/
break
else
echo "$file isn't the culprit."
fi
rm -rf venv/
done
mv holdspace/*.py .
The following should fix it
brew update
brew upgrade
brew install zlib
I'm trying to create a virtual environment with 3.4. I have followed instructions found through searching stackoverflow, as well as posting this issue on reddit, and continue to have the same problem: even when I specify python3.4, it installs both 3.4 AND 2.7. This then leads to all of the problems I was trying to avoid by using a virtual environment in the first place.
$ which python3
/Library/Frameworks/Python.framework/Versions/3.4/bin/python3
$ mkdir test
$ cd test
$ virtualenv -p /Library/Frameworks/Python.framework/Versions/3.4/bin/python3.4 venv
(various output saying it's creating venv/bin/python with 3.4)
$ source venv/bin/activate
(venv)$ python --version
Python 2.7.9
when I look in the venv/bin folder it has binaries for both 2.7.9 and 3.4.
after activating the environment:
$ echo $PATH
/Users/theinevitable/Documents/python/test/venv/bin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/Library/Frameworks/Python.framework/Versions/2.7/bin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Postgres.app/Contents/Versions/9.4/bin
$ which python
/Users/theinevitable/Documents/python/test/venv/bin/python
$ which python3
/Library/Frameworks/Python.framework/Versions/3.4/bin/python3
one person on reddit suggested trying to use venv instead:
$ python3.4 -m venv my_venv
/Library/Frameworks/Python.framework/Versions/3.4/bin/python3.4: No module named venv
$ pyvenv tester
File "/Library/Frameworks/Python.framework/Versions/3.4/bin/pyvenv", line 10
print('Error: %s' % e, file=sys.stderr)
^
SyntaxError: invalid syntax
I've been trying to get up and running with the built-in "venv" module of Python 3.3 on my OS X machine. I've installed Python 3.3 using Homebrew.
As per the docs, creating and switching virtual environment works as you'd expect:
$ python3 -m venv myvenv
$ source myvenv/bin/activate
And I've tested something like this:
$ echo "YEAH = 'YEAH!'" > myvenv/lib/python3.3/site-packages/thingy.py
$ python
>>> import thingy
>>> print(thingy.YEAH)
'YEAH!'
But when I try to install distribute, it simply won't go in the proper place. For some reason, it insists on trying to install into /usr/local/lib/python3.3/site-packages/, which fails with the following messages:
No setuptools distribution found
running install
Checking .pth file support in /usr/local/lib/python3.3/site-packages/
/Users/victor/myvenv/bin/python -E -c pass
TEST FAILED: /usr/local/lib/python3.3/site-packages/ does NOT support .pth files
error: bad install directory or PYTHONPATH
You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from. The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/usr/local/lib/python3.3/site-packages/
and your PYTHONPATH environment variable currently contains:
''
This happens regardless if I try to install using distribute_setup.py or using the source distribution directly. I've even tried using --prefix=/Users/victor/myenv but it still tries to put everything in my "global" site-packages.
I can't figure out why this happens, but it's consistent on two of my machines. Note that sys.prefix reports the correct path (the virtual environment).
Is this a problem with Homebrew? OS X? Python 3.3? venv? Me?
This has been an issue with Homebrew, yes, but it is working now since https://github.com/mxcl/homebrew/commit/0b50110107ea2998e65011ec31ce45931b446dab.
$ brew update
$ brew rm python3 #if you have installed it before
$ brew install python3
$ cd /tmp
$ which python3
/usr/local/bin/python3
$ python3 -m venv myvenv
$ source myvenv/bin/activate
$ wget http://python-distribute.org/distribute_setup.py # may need brew install wget
$ python3 distribute_setup.py
...
Finished processing dependencies for distribute==0.6.45
After install bootstrap.
Creating /private/tmp/myvenv/lib/python3.3/site-packages/setuptools-0.6c11-py3.3.egg-info
Creating /private/tmp/myvenv/lib/python3.3/site-packages/setuptools.pth
You see that distribute install successfully into the /tmp dir.
This happens because homebrew installs distutils config file:
$ brew cat python3 | grep "Tell distutils" -A5
# Tell distutils-based installers where to put scripts
(prefix/"Frameworks/Python.framework/Versions/#{VER}/lib/python#{VER}/distutils/distutils.cfg").write <<-EOF.undent
[install]
install-scripts=#{scripts_folder}
install-lib=#{site_packages}
EOF
$ mv ~/.local/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/distutils.cfg ~/tmp/
$ cat ~/tmp/distutils.cfg
[install]
install-scripts=/Users/gatto/.local/share/python3
install-lib=/Users/gatto/.local/lib/python3.3/site-packages
$ . venv/bin/activate
(venv) $ python distribute-0.6.36/distribute_setup.py
(venv) $ ls venv/lib/python3.3/site-packages/
distribute-0.6.36-py3.3.egg easy-install.pth setuptools-0.6c11-py3.3.egg-info setuptools.pth
See "distutils.cfg Can Break venv" issue at bugs.python.org.