How to configure a virtual environment that doesn't require sudo? - python

In my Ubuntu 12.04 machine, the installation of pip requirements is asking me for sudo permission every time it attempts to install. How would I override this, as this is terrible for my working environment to install things globally instead of inside the venv?
Note: I did not setup the venv using sudo.

Have you activated your virtual environment? Run:
. bin/activate
in your shell. Then the local pip installation will take over the system one.

Thanks to #MartijnPieters, I found a workaround:
Running
~/.virtualenvs/myapp/bin/pip install -r requirements.txt
Instead of just
pip install -r requirements.txt

Make sure you use a recent version of virtualenv itself, the latest at the time of writing is 1.7.2. Old versions required the use of -E switch, to install into the virtual environment.

Related

bash: virtualenv: command not found "ON Linux"

I am using a form of Lubuntu called GalliumOS (optimized for Chromebooks). I installed pip using $ sudo apt-get install python-pip. I then used pip install --user virtualenv and pip install virtualenv, and then when I tried to subsequently use virtualenv venv I experienced the message bash: virtualenv: command not found.
Between the pip installs above, I used pip uninstall virtualenv to get back to square one. The error remained after a reinstall.
I read several other posts, but all of them seemed to deal with similar problems on MacOS. One that came close was installing python pip and virtualenv simultaneously. Since I had already installed pip, I didn't think that these quite applied to my issue. Why is pip install virtualenv not working this way on LUbuntu / GalliumOS?
Are you sure pip install is "failing"? To me, it sounds like the directory to which pip is installing modules on your machine is not in your PATH environment variable, so when virtualenv is installed, your computer has no idea where to find it when you just type in virtualenv.
Find where pip is installing things on your computer, and then check if the directory where the pyenv executable is placed is in your PATH variable (e.g. by doing echo $PATH to print your PATH variable). If it's not, you need to update your PATH variable by adding the following to your .bashrc or .bash_profile or etc.:
export PATH="PATH_TO_WHERE_PIP_PUTS_EXECUTABLES:$PATH"
What finally worked for me was this. I used
$ sudo apt-get install python-virtualenv.
I was then able to create a virtual environment using $ virtualenv venv.
I was seeking to avoid using $ sudo pip install virtualenv, because of admonitions in other posts to not do this, and agreed, because of experiences I'd had with subsequent difficulties when doing this.
pip install virtualenv
This command worked for me for. This problem that raised to me on Kali Linux.

Python - Virtual Environment uses System Directories

I've created a Python virtual environment, and activate it by doing:
joe#joe-mint $ source ./venvs/deep-learning/bin/activate
Which turns the prompt into:
(deep-learning) joe#joe-mint $
Now whenever I run a python package or try to install one, the system seems to ignore the fact that it's in a virtual environment and does things system-wide:
(deep-learning) joe#joe-mint $ which pip
/usr/local/bin/pip
The same happens when I try to install new packages that aren't on my system; it installs them to the system files (i.e. /usr/bin) instead of the virtual environment.
What's wrong with my virtual environment? How do I get it to ignore system files and do everything inside the environment?
I've looked at this question which says to use an explicit flag when creating the virtual environment to make it use the local environment packages, but I used python-3.5 -m venv to create the virtual environment, and this flag is removed in this version as it's now a default option.
I've also looked at this question and can confirm that the VIRTUAL_ENV variable is set correctly in the activate file of the virtual environment.
Here was the problem:
It seems that if you run pip on a venv without a local pip installation, then it will default to the system's pip outside the venv. Even if you've activated a virtual environment, this seems to want to install packages on the system rather than in the venv.
Here was the solution:
First, I had to install the virtual environment without pip due to a bug that has long remained unresolved.
Second, I installed pip in the virtual environment as per the instruction here. However, doing so required using some temporary folders that for some reason my user didn't have access to. So this failed, and the only way I could get it to work was to become root.
sudo su
activate ..../venvs/deep-learning/bin/activate to activate the virtual environment.
curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | python as per the answer linked above.
Although which pip now indicated the correct pip (inside the venv) was being used, running pip would use the system one! Deactivating (deactivate) and reactivating the venv solved this.
Now it took me a while to realise that having installed this as root caused me permission errors when trying to install more packages using pip inside the virtual environment.
chown <user>:<group> -R ..../venvs/deep-learning/*
And that was it. After these steps, I could activate the venv and run pip correctly. It would use the pip inside the venv, and install packages inside the venv.

Using Python 3 in virtualenv

Using virtualenv, I run my projects with the default version of Python (2.7). On one project, I need to use Python 3.4.
I used brew install python3 to install it on my Mac. Now, how do I create a virtualenv that uses the new version?
e.g. sudo virtualenv envPython3
If I try:
virtualenv -p python3 test
I get:
Running virtualenv with interpreter /usr/local/bin/python3
Using base prefix '/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4'
New python executable in test/bin/python3.4
Also creating executable in test/bin/python
Failed to import the site module
Traceback (most recent call last):
File "/Users/user/Documents/workspace/test/test/bin/../lib/python3.4/site.py", line 67, in <module>
import os
File "/Users/user/Documents/workspace/test/test/bin/../lib/python3.4/os.py", line 634, in <module>
from _collections_abc import MutableMapping
ImportError: No module named '_collections_abc'
ERROR: The executable test/bin/python3.4 is not functioning
ERROR: It thinks sys.prefix is '/Users/user/Documents/workspace/test' (should be '/Users/user/Documents/workspace/test/test')
ERROR: virtualenv is not compatible with this system or executable
simply run
virtualenv -p python3 envname
Update after OP's edit:
There was a bug in the OP's version of virtualenv, as described here. The problem was fixed by running:
pip install --upgrade virtualenv
Python 3 has a built-in support for virtual environments - venv. It might be better to use that instead. Referring to the docs:
Creation of virtual environments is done by executing the pyvenv
script:
pyvenv /path/to/new/virtual/environment
Update for Python 3.6 and newer:
As pawciobiel correctly comments, pyvenv is deprecated as of Python 3.6 and the new way is:
python3 -m venv /path/to/new/virtual/environment
I'v tried pyenv and it's very handy for switching python versions (global, local in folder or in the virtualenv):
brew install pyenv
then install Python version you want:
pyenv install 3.5.0
and simply create virtualenv with path to needed interpreter version:
virtualenv -p /Users/johnny/.pyenv/versions/3.5.0/bin/python3.5 myenv
That's it, check the version:
. ./myenv/bin/activate && python -V
There are also plugin for pyenv pyenv-virtualenv but it didn't work for me somehow.
Install prerequisites.
sudo apt-get install python3 python3-pip virtualenvwrapper
Create a Python3 based virtual environment. Optionally enable --system-site-packages flag.
mkvirtualenv -p /usr/bin/python3 <venv-name>
Set into the virtual environment.
workon <venv-name>
Install other requirements using pip package manager.
pip install -r requirements.txt
pip install <package_name>
When working on multiple python projects simultaneously it is usually recommended to install common packages like pdbpp globally and then reuse them in virtualenvs.
Using this technique saves a lot of time spent on fetching packages and installing them, apart from consuming minimal disk space and network bandwidth.
sudo -H pip3 -v install pdbpp
mkvirtualenv -p $(which python3) --system-site-packages <venv-name>
Django specific instructions
If there are a lot of system wide python packages then it is recommended to not use --system-site-packages flag especially during development since I have noticed that it slows down Django startup a lot. I presume Django environment initialisation is manually scanning and appending all site packages from the system path which might be the reason. Even python manage.py shell becomes very slow.
Having said that experiment which option works better. Might be safe to just skip --system-site-packages flag for Django projects.
virtualenv --python=/usr/bin/python3 <name of env>
worked for me.
This is all you need, in order to run a virtual environment in python / python3
First if virtualenv not installed, run
pip3 install virtualenv
Now Run:
virtualenv -p python3 <env name> # you can specify full path instead <env_name> to install the files in a different location other than the current location
Sometime the cmd virtualenv fails, if so use this:
python3 -m virtualenv <env_name> # you can specify full path instead <env_name> to install the files in a different location other than the current location
Now activate the virtual env:
source <env_name>/bin/activate
Or:
source `pwd`/<env_name>/bin/activate
Now run
which python
You should see the full path to your dir and <env_name>/bin/python suffix
To exit the virtualenv, run:
deactivate
To troubleshoot Python location got to here
You can specify specific Version of Python while creating environment.
It's mentioned in virtualenv.py
virtualenv --python=python3.5 envname
In some cases this has to be the full path to the executable:
virtualenv --python=/Users/username/.pyenv/versions/3.6.0/bin/python3.6 envname
How -p works
parser.add_option(
'-p', '--python',
dest='python',
metavar='PYTHON_EXE',
help='The Python interpreter to use, e.g., --python=python3.5 will use the python3.5 '
'interpreter to create the new environment. The default is the interpreter that '
'virtualenv was installed with (%s)' % sys.executable)
I had the same ERROR message. tbrisker's solution did not work in my case. Instead this solved the issue:
$ python3 -m venv .env
In addition to the other answers, I recommend checking what instance of virtualenv you are executing:
which virtualenv
If this turns up something in /usr/local/bin, then it is possible - even likely - that you installed virtualenv (possibly using an instance of easy_tools or pip) without using your system's package manager (brew in OP's case). This was my problem.
Years ago - when I was even more ignorant - I had installed virtualenv and it was masking my system's package-provided virtualenv.
After removing this old, broken virtualenv, my problems went away.
The below simple commands can create a virtual env with version 3.5
apt-get install python3-venv
python3.5 -m venv <your env name>
if you want virtual env version as 3.6
python3.6 -m venv <your env name>
Python now comes with its own implementation of virtual environment, by the name of "venv". I would suggest using that, instead of virtualenv.
Quoting from venv - docs,
Deprecated since version 3.6: pyvenv was the recommended tool for
creating virtual environments for Python 3.3 and 3.4, and is
deprecated in Python 3.6.
Changed in version 3.5: The use of venv is now recommended for
creating virtual environments.
For windows, to initiate venv on some project, open cmd:
python -m venv "c:\path\to\myenv"
(Would suggest using double quote around directory path if it contains any spaces. Ex: "C:/My Dox/Spaced Directory/Something")
Once venv is set up, you will see some new folders inside your project directory. One of them would be "Scripts".
To activate or invoke venv you need:
C:\> <venv>\Scripts\activate.bat
You can deactivate a virtual environment by typing “deactivate” in your shell. With this, you are now ready to install your project specific libraries, which will reside under the folder "Lib".
================================ Edit 1 ====================================
The scenario which will be discussed below is not what originally asked, just adding this in case someone use vscode with python extension
In case, you use vs code with its python extension, you might face an issue with its pylint which points to the global installation. In this case, pylint won't be able to see the modules that are installed in your virtual environment and hence will show errors while importing.
Here is a simple method to get past this.
cd Workspace\Scripts
.\Activate.ps1
code .
We are basically activating the environment first and then invoking vs-code so that pylint starts within the environment and can see all local packages.
In python3.6 I tried
python3 -m venv myenv,
as per the documentation, but it was taking so long. So the very simple and quick command is
python -m venv yourenv
It worked for me on python3.6.
On Mac I had to do the following to get it to work.
mkvirtualenv --python=/usr/bin/python3 YourEnvNameHere
If you install python3 (brew install python3) along with virtualenv burrito, you can then do mkvirtualenv -p $(which python3) env_name
Of course, I know virtualenv burrito is just a wrapper, but it has served me well over the years, reducing some learning curves.
virtualenv --python=/usr/local/bin/python3 <VIRTUAL ENV NAME>
this will add python3
path for your virtual enviroment.
It worked for me
virtualenv --no-site-packages --distribute -p /usr/bin/python3 ~/.virtualenvs/py3
For those having troubles while working with Anaconda3 (Python 3).
You could use
conda create -n name_of_your_virtualenv python=python_version
To activate the environment ( Linux, MacOS)
source activate name_of_your_virtualenv
For Windows
activate name_of_your_virtualenv
I tried all the above stuff, it still didn't work. So as a brute force, I just re-installed the anaconda, re-installed the virtualenv... and it worked.
Amans-MacBook-Pro:~ amanmadan$ pip install virtualenv
You are using pip version 6.1.1, however version 8.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting virtualenv
Downloading virtualenv-15.0.3-py2.py3-none-any.whl (3.5MB)
100% |████████████████████████████████| 3.5MB 114kB/s
Installing collected packages: virtualenv
Successfully installed virtualenv-15.0.3
Amans-MacBook-Pro:python amanmadan$ virtualenv my_env
New python executable in /Users/amanmadan/Documents/HadoopStuff/python/my_env/bin/python
Installing setuptools, pip, wheel...done.
Amans-MacBook-Pro:python amanmadan$
I wanted to keep python 2.7.5 as default version on Centos 7 but have python 3.6.1 in a virtual environment running alongside other virtual environments in python 2.x
I found the below link the best solution for the newest python version ( python 3.6.1)
https://www.digitalocean.com/community/tutorial_series/how-to-install-and-set-up-a-local-programming-environment-for-python-3.
It shows the steps for different platforms but the basic steps are
Install python3.x (if not present) for your platform
Install python3.x-devel for your platform
Create virtual environment in python 3.x
(for example $ python3.6 -m venv virenv_test_p3/ )
Activate the testenvironment for python 3.x
(for example source virenv_test_p3/bin/activate)
Install the packages which you want to use in your new python 3 virtual environment and which are supported ( for example pip install Django==1.11.2)
On Windows command line, the following worked for me. First find out where your python executables are located:
where python
This will output the paths to the different python.exe on your system. Here were mine:
C:\Users\carandangc\Anaconda3\python.exe
C:\Python27\python.exe
So for Python3, this was located in the first path for me, so I cd to the root folder of the application where I want to create a virtual environment folder. Then I run the following which includes the path to my Python3 executable, naming my virtual environment 'venv':
virtualenv --python=/Users/carandangc/Anaconda3/python.exe venv
Next, activate the virtual environment:
call venv\Scripts\activate.bat
Finally, install the dependencies for this virtual environment:
pip install -r requirements.txt
This requirements.txt could be populated manually if you know the libraries/modules needed for your application in the virtual environment. If you had the application running in another environment, then you can automatically produce the dependencies by running the following (cd to the application folder in the environment where it is working):
pip freeze > requirements.txt
Then once you have the requirements.txt that you have 'frozen', then you can install the requirements on another machine or clean environment with the following (after cd to the application folder):
pip install -r requirements.txt
To see your python version in the virtual environment, run:
python --version
Then voila...you have your Python3 running in your virtual environment. Output for me:
Python 3.7.2
For those of you who are using pipenv and want to install specific version:
pipenv install --python 3.6
I got the same error due to it being a conflict with miniconda3 install so when you type "which virtualenv" and if you've installed miniconda and it's pointing to that install you can either remove it (if your like me and haven't moved to it yet) or change your environment variable to point to the install you want.

pip freeze does not show all installed packages

I am using a virtualenv. I have fabric installed, with pip. But a pip freeze does not give any hint about that. The package is there, in my virtualenv, but pip is silent about it. Why could that be? Any way to debug this?
I just tried this myself:
create a virtualenv in to the "env" directory:
$virtualenv2.7 --distribute env
New python executable in env/bin/python
Installing distribute....done.
Installing pip................done.
next, activate the virtual environment:
$source env/bin/activate
the prompt changed. now install fabric:
(env)$pip install fabric
Downloading/unpacking fabric
Downloading Fabric-1.6.1.tar.gz (216Kb): 216Kb downloaded
Running setup.py egg_info for package fabric
...
Successfully installed fabric paramiko pycrypto
Cleaning up...
And pip freeze shows the correct result:
(env)$pip freeze
Fabric==1.6.1
distribute==0.6.27
paramiko==1.10.1
pycrypto==2.6
wsgiref==0.1.2
Maybe you forgot to activate the virtual environment? On a *nix console type which pip to find out.
You can try using the --all flag, like this:
pip freeze --all > requirements.txt
Although your problem was specifically due to a typo, to help other users:
pip freeze doesn't show the dependencies that pip depends on. If you want to obtain all packages you can use pip freeze --all or pip list.
If you have redirected all the pre-installed packages in a file named pip-requirements.txt then it is pretty simple to fix the above issue.
1) Delete your virtualenv folder or create new one (I am giving it a name as venv)
rm -rf venv && virtualenv venv
2) Install all the requirements/dependencies from the pip-requirements.txt
pip install -r pip-requirements.txt
3) Now you can check the installed packages for your Django application
pip freeze
4) If you had forgotten to update your requirements file(pip-requirements.txt), then install fabric again (Optional Step)
Note: After installing any dependency for your Django app, always update the requirements in any file as follows (make sure your virtualenv is activated)
pip freeze > pip requirements.txt
That's it.
Adding my fix in addition of above fix also ,
I was also facing the same issue on windows,even after activating the virtualenv too pip freeze was not giving me all list of installed packages. So i upgraded my pip with python -m pip install --upgrade pip command and then used pip freeze.
This time it worked and gave me all list of installed packages.
This might be stupid but I have got the same problem. I solved it by refreshing vs code file directory (inside vscode there is a reload button). :)
If none of the above answers are working for you.
As with me you might have problem in you venv and pip configuration.
Go inside your venv/bin and open pip and see the 2nd line as:
'''exec' "path/to/yourvenv/bin/python3" "$0" "$#"
See if this line is correctly pointing inside your venv or not
For example in my case.
I initially named my virtual environment as venv1
and later just renamed it to venv2.
In doing so my pip file 2nd line had: '''exec' "venv1/bin/python3" "$0" "$#"
which to work properly should have: '''exec' "venv2/bin/python3" "$0" "$#" notice "venv2" not "venv1" since venv1 in now renamed to venv2.
Due to this python was looking inside pip of venv2 and throwing error or not working as desired.
For those who added Python modules via PyCharm IDE, after generating a virtual environment from the command prompt, good luck! You will need to rebuild the requirements.txt file manually with the ones missing by first running pip3 freeze and adding what is missing from PyCharm.
I highly suggest switching to Visual Studio Code.

Why would pip install python packages outside of my current virtualenv?

I just installed virtualenv earlier today, and I've been trying to get django-nonrel to work using a virtual environment. After many hours of trying and failing to do so, I chatted with someone who suggested that the problem is with pip itself.
I create a virtual environment with virtualenv env_name. I enter it with:
source env_name/bin/activate
I then install something with pip. For example,
sudo pip install git+https://github.com/django-nonrel/django-nonrel;
I then perform a pip freeze, and the only output is:
wsgiref=0.1.2
Can anyone see why this might happen?
Thanks,
ParagonRG
Notes:
pip is version 1.1.
virtualenv is version 1..7.1.2
You forgot the git extension
$> mkvirtualenv test
(test)> pip install git+https://github.com/django-nonrel/django-nonrel.git
(test)> pip freeze
Django==1.3.1
wsgiref==0.1.2
By the way, you dont need to use sudo if you work with virtualenv. The idea is having your virtualen somewhere you have permission like ~/.virtualenvs/

Categories

Resources