I can install pysvn site-wide using the binary package. For example, in Ubuntu:
$ sudo apt-get install python-svn
Or, on Windows, I can install site-wide using the .exe installer.
Outside of a virtualenv, I can do this
$ python -c "import pysvn; print 'ok'"
ok
Now I make a virtualenv (I use the mkvirtualenv command from the virtualenvwrapper package)
$ mkvirtualenv test1
But since virtualenv defaults to not importing global site packages, I can not use pysvn inside this virtualenv.
(test1)$ python -c "import pysvn; print 'ok'"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named pysvn
How do I access pysvn in a virtualenv without enabling global site packages?
There are a number of ways to handle this.
Option 0
Allow access to the global site packages from within the virtualenv. Pass the --system-site-packages option to virtualenv when creating the virtual environment.
Or, use the toggleglobalsitepackages command (from virtualenvwrapper) to allow access to global site packages.
(test1)$ toggleglobalsitepackages
Enabled global site-packages
(test1)$ python -c "import pysvn; print 'ok'"
ok
(test1)$ toggleglobalsitepackages
Disabled global site-packages
(test1)$ python -c "import pysvn; print 'ok'"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named pysvn
Option 1
Use easy_install to install the package in to the virtualenv using a binary installer. For example, on Windows the process might look like this:
Download the binary installer file. In this example, let's call it example_installer.msi (or example_installer.exe)
Activate the virtualenv (I use virtualenvwrapper-win on Windows)
easy_install example_installer.msi
Verify that you can install the installer site-wide, by double-clicking and running the installer in gui mode (then uninstal using the Windows Add/Remove Programs control panel). If you can install it site-wide, then easy_install can probably install it in to a virtualenv.
However, the pysvn binary installer is not structured properly for easy_install. If you try this with the Windows pysvn binary installer you get this error:
error: py27-pysvn-svn185-1.7.9-1572.exe is not a valid distutils Windows .exe
Option 2
Use the add2virtualenv command from virtualenvwrapper. This adds a .pth file to the virtualenv's site-packages directory, which gives the virtualenv access to the specified directories.
Note that you must specify the parent directory, instead of the specific package. That is, instead of
add2virtualenv /usr/lib/python2.7/dist-packages/pysvn
It should be
add2virtualenv /usr/lib/python2.7/dist-packages
See this question: add2virtualenv (virtualenv wrapper) does not work with scipy
To find the directory where a package is installed, do this:
$ python
>>> import pysvn
>>> pysvn.__file__
'/usr/lib/python2.7/dist-packages/pysvn/__init__.pyc'
The problem is, this includes all the packages in the specified directory, not just pysvn. So, it has the same drawback as toggleglobalsitepackages.
Option 3
Symlink the install directory in to the virtualenv's site-packages.
A convenient way to get to the virtualenv's site-packages directory is to use virtualenvwrapper's cdsitepackages command
cdsitepackages
ln -s /usr/lib/python2.7/dist-packages/pysvn pysvn
Summary
On Windows, try Option 1 (easy_install from binary installer). If that fails, install globally and allow the virtualenv to access it by using virtualenvwrapper-win's toggleglobalsitepackages command, or by passing the --system-site-packages option to virtualenv.
On systems that support symlinking, such as Linux and OS X, use Option 3. It allows you to access the specific packages you need without allowing access to the whole global site packages.
Related
I am trying to install a Python package with this command
pip install <name of package>
I'm getting permission errors and I'm not sure why. I could run it with sudo, but someone told me that was a bad idea, and I should use a virtualenv instead.
What is a virtualenv? What does it do for me?
Running with the system Python and libraries limits you to one specific Python version, chosen by your OS provider. Trying to run all Python applications on one Python installation makes it likely that version conflicts will occur among the collection of libraries. It's also possible that changes to the system Python will break other OS features that depend on it.
Virtual environments, or "virtualenvs" are lightweight, self-contained Python installations, designed to be set up with a minimum of fuss, and to "just work" without requiring extensive configuration or specialized knowledge.
virtualenv avoids the need to install Python packages globally. When a virtualenv is active, pip will install packages within the environment, which does not affect the base Python installation in any way.
In Python 3.3 or later, you can create a virtualenv as follows:
$ python3 -m venv ENV_DIR
For Windows, you should replace python3 with the full path to python.exe:
>C:\Python34\python.exe -m venv ENV_DIR
(This is a typical Python installation; your system may vary.)
In older versions of Python, including Python 2, one of the following commands should work in most cases:
$ virtualenv ENV_DIR
$ venv ENV_DIR
$ pyvenv ENV_DIR
$ pyvenv3 ENV_DIR
ENV_DIR should be a non-existent directory. The directory can have any name, but to keep these instructions simple, I will assume you have created your virtualenv in a directory called venv (e.g. with python3 -m venv ./venv).
To work in your virtualenv, you activate it:
$ . ./venv/bin/activate
(venv)$
Or use this if you have a windows system:
$ venv\Scripts\activate
The (venv) in the shell prompt lets you know which virtualenv you have activated, but you can turn this feature off if you do not like it. You can run all the usual Python commands, and they will be local to your virtualenv:
(venv)$ pip install requests numpy
[...]
(venv)$ python
[...]
>>> import requests
>>> import numpy as np
>>>
python will run the version of Python that you installed into your virtualenv, so (for example) you don't have to type python3 to get Python 3. The Python that it runs will have access to all the standard library modules and all the packages you installed into the virtualenv, but (by default) none of the packages installed in the system-wide site-packages directory.
This last rule is important: by restricting your virtualenv to only use locally-installed packages, you can ensure that you control exactly which dependencies your project is using, even if some new system-wide package gets installed or updated next week. If you like, you can get a listing of your installed packages:
(venv)$ pip freeze
requests==2.13.0
numpy==1.12.0
(venv)$
pip can also parse this format and install from it, and it will install the same versions, even if updates have been released in the meantime:
(venv)$ pip freeze >requirements.txt
(some-other-venv)$ pip install -r requirements.txt
[...]
(some-other-venv)$ python
>>> import requests
>>> import numpy as np
>>>
You can get out of the virtualenv by deactivating it:
(venv)$ deactivate
$ python
[...]
>>> import requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'requests'
You can create as many virtualenvs as you like, and they won't interfere with each other, nor with your system packages. A virtualenv is "just" a directory with a bunch of binaries and scripts under it, so you can remove a virtualenv the same way you remove any directory (rm -r venv on Unix). If the virtualenv is activated when you remove it, you may confuse your shell, so it's probably a good idea to deactivate first in that case.
Some times you are not given root privileges and you might end up not being able to use sudo. Many other times, it's not advisable to use sudo to install packages as it might overwrite some package which might be in use by some other applications.
Virtualenv can help you create a separate environment where you don't need root privileges as well as be able to tailor the environment according to your need. It consists of self-contained python installation which only interacts with your specific created environment.
So basically, it gives you a bit of freedom as well as avoid damaging (or modifying) the root environment which might be hosting many old functionalities of old applications.
Installation is pretty easy too.
Installing packages with sudo pip will install packages globally, which may break some system tools.
By install globally it means you will install your packages in place like /usr/lib/python2.7/site-package so if some packages need a previous version of your python packages, this action may break it.
virtualenv allows you to avoid installing Python packages globally by making an isolated python environment. That means it will install packages just in your desire project folder.
On mac and linux
Install
python3 -m pip install --user virtualenv
Creating a Virtual Env: Go to your desired project folder
python3 -m virtualenv env
Activating a virtualenv: In your desired project folder
source env/bin/activate
After activating you can install your packages using pip.
For more information about using it in Windows:
How to use virtualenv in Windows
I am going to break your question into two parts.
What is a virtualenv?
Python has its own way of downloading, storing, and resolving site packages. But Python can not differentiate between different versions in the site-package directory. Packages will be installed in one of the directories, whose name can be found by running the site.getsitepackages() commands.
>>> import site
>>> site.getsitepackages()
This means package_v2.0.1 and package_v3.0.1 have to be in the same directory with the same name i.e. package, which is obviously not possible. Now, you may ask why we would need the same package with different versions on our system. This is because multiple projects may require different versions of Python packages or even different Python versions themselves. So there was a need to have something which will resolve these conflicts and Virtualenv came into the picture to solve this issue.
What does it do for me?
It isolates the environment for Python projects so that each project can have its own dependencies.
I am currently using Fabric3 in a python project and I want to use the azure-cli package. However when I install the package, it appears to break my Fabric3 install:
root ~ $ fab --version
Traceback (most recent call last):
File "/opt/rh/rh-python36/root/usr/bin/fab", line 7, in
from Fabric3.main import program
ModuleNotFoundError: No module named 'Fabric3'
I noticed that the pip install of azure-cli appears to install fabric (2.5.0), and this appears to overwrite some of the fabric files within the "/opt/rh/rh-python36/root/usrbin/fab" script and the "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/fabric/" directory.
So my question is, is there a way to run different versions of Fabric within the same project?
Thanks in advance
I found a way to do this by using python virtual environments.
I created a virtual env, activated the virtual env, installed the pip package, deactivated the virtual env, then used Fabric 3 to call the function in the venv.
python -m venv azure_cli_venv
. azure_cli_venv/bin/activate
pip install azure-cli
deactivate
Fabric3 local call:
local(f'./azure_cli_venv/bin/az --help
I am trying to install a Python package with this command
pip install <name of package>
I'm getting permission errors and I'm not sure why. I could run it with sudo, but someone told me that was a bad idea, and I should use a virtualenv instead.
What is a virtualenv? What does it do for me?
Running with the system Python and libraries limits you to one specific Python version, chosen by your OS provider. Trying to run all Python applications on one Python installation makes it likely that version conflicts will occur among the collection of libraries. It's also possible that changes to the system Python will break other OS features that depend on it.
Virtual environments, or "virtualenvs" are lightweight, self-contained Python installations, designed to be set up with a minimum of fuss, and to "just work" without requiring extensive configuration or specialized knowledge.
virtualenv avoids the need to install Python packages globally. When a virtualenv is active, pip will install packages within the environment, which does not affect the base Python installation in any way.
In Python 3.3 or later, you can create a virtualenv as follows:
$ python3 -m venv ENV_DIR
For Windows, you should replace python3 with the full path to python.exe:
>C:\Python34\python.exe -m venv ENV_DIR
(This is a typical Python installation; your system may vary.)
In older versions of Python, including Python 2, one of the following commands should work in most cases:
$ virtualenv ENV_DIR
$ venv ENV_DIR
$ pyvenv ENV_DIR
$ pyvenv3 ENV_DIR
ENV_DIR should be a non-existent directory. The directory can have any name, but to keep these instructions simple, I will assume you have created your virtualenv in a directory called venv (e.g. with python3 -m venv ./venv).
To work in your virtualenv, you activate it:
$ . ./venv/bin/activate
(venv)$
Or use this if you have a windows system:
$ venv\Scripts\activate
The (venv) in the shell prompt lets you know which virtualenv you have activated, but you can turn this feature off if you do not like it. You can run all the usual Python commands, and they will be local to your virtualenv:
(venv)$ pip install requests numpy
[...]
(venv)$ python
[...]
>>> import requests
>>> import numpy as np
>>>
python will run the version of Python that you installed into your virtualenv, so (for example) you don't have to type python3 to get Python 3. The Python that it runs will have access to all the standard library modules and all the packages you installed into the virtualenv, but (by default) none of the packages installed in the system-wide site-packages directory.
This last rule is important: by restricting your virtualenv to only use locally-installed packages, you can ensure that you control exactly which dependencies your project is using, even if some new system-wide package gets installed or updated next week. If you like, you can get a listing of your installed packages:
(venv)$ pip freeze
requests==2.13.0
numpy==1.12.0
(venv)$
pip can also parse this format and install from it, and it will install the same versions, even if updates have been released in the meantime:
(venv)$ pip freeze >requirements.txt
(some-other-venv)$ pip install -r requirements.txt
[...]
(some-other-venv)$ python
>>> import requests
>>> import numpy as np
>>>
You can get out of the virtualenv by deactivating it:
(venv)$ deactivate
$ python
[...]
>>> import requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'requests'
You can create as many virtualenvs as you like, and they won't interfere with each other, nor with your system packages. A virtualenv is "just" a directory with a bunch of binaries and scripts under it, so you can remove a virtualenv the same way you remove any directory (rm -r venv on Unix). If the virtualenv is activated when you remove it, you may confuse your shell, so it's probably a good idea to deactivate first in that case.
Some times you are not given root privileges and you might end up not being able to use sudo. Many other times, it's not advisable to use sudo to install packages as it might overwrite some package which might be in use by some other applications.
Virtualenv can help you create a separate environment where you don't need root privileges as well as be able to tailor the environment according to your need. It consists of self-contained python installation which only interacts with your specific created environment.
So basically, it gives you a bit of freedom as well as avoid damaging (or modifying) the root environment which might be hosting many old functionalities of old applications.
Installation is pretty easy too.
Installing packages with sudo pip will install packages globally, which may break some system tools.
By install globally it means you will install your packages in place like /usr/lib/python2.7/site-package so if some packages need a previous version of your python packages, this action may break it.
virtualenv allows you to avoid installing Python packages globally by making an isolated python environment. That means it will install packages just in your desire project folder.
On mac and linux
Install
python3 -m pip install --user virtualenv
Creating a Virtual Env: Go to your desired project folder
python3 -m virtualenv env
Activating a virtualenv: In your desired project folder
source env/bin/activate
After activating you can install your packages using pip.
For more information about using it in Windows:
How to use virtualenv in Windows
I am going to break your question into two parts.
What is a virtualenv?
Python has its own way of downloading, storing, and resolving site packages. But Python can not differentiate between different versions in the site-package directory. Packages will be installed in one of the directories, whose name can be found by running the site.getsitepackages() commands.
>>> import site
>>> site.getsitepackages()
This means package_v2.0.1 and package_v3.0.1 have to be in the same directory with the same name i.e. package, which is obviously not possible. Now, you may ask why we would need the same package with different versions on our system. This is because multiple projects may require different versions of Python packages or even different Python versions themselves. So there was a need to have something which will resolve these conflicts and Virtualenv came into the picture to solve this issue.
What does it do for me?
It isolates the environment for Python projects so that each project can have its own dependencies.
I am working in a virtual environment in Python 3.I need to use a 3 party module "mglearn" and I copy it to my virtual environment's lib/:
/home/abigail/environments/my_env/lib/python3.5/site-packages/mglearn
However, in ipython command line, it can't find the module name:
In [1]: import mglearn
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-1-e19748f92cd9> in <module>()
----> 1 import mglearn
ImportError: No module named 'mglearn'
It should find it. Right?
Then I checked my sys.path:
In [4]: print(sys.path)
['', '/usr/bin', '/usr/lib64/python35.zip', '/usr/lib64/python3.5', '/usr/lib64/python3.5/plat-linux', '/usr/lib64/python3.5/lib-dynload', '/usr/lib64/python3.5/site-packages', '/usr/lib/python3.5/site-packages', '/usr/lib/python3.5/site-packages/IPython/extensions', '/home/abigail/.ipython']
Why does "sys.path" only contain directories starting from the root /, not my virtual environment? How can I get that module to be searched by Python?
Edited:
[abigail#localhost bin]$ ll activate
activate activate.csh activate.fish
[abigail#localhost bin]$ ./activate
bash: ./activate: Permission denied
[abigail#localhost bin]$ sudo ./activate
sudo: ./activate: command not found
Strange! why is that?
VirtualEnv creates a clone of a Python installation and bakes an additional path into sys.path that point to the site-packages directory of a given virtualenv.
When you launch your IPython, it is likely installed in your main Python installation and does not know about any additional virtual environments you have created.
If you install IPython into a virtual environment, it will know about the site-packages location for that virtualenv. Try and run:
which ipython
Then look at your ipython script and you will see it begin with either:
#!/usr/bin/python
or:
#!/home/abigail/environments/my_env/bin/python3
The first indicates a globally installed ipython and the second is an ipython that has been installed into a specific virtualenv.
FYI, you can add paths to a Python interpreter by exporting the PYTHONPATH environment variable:
```export PYTHONPATH=/home/abigail/environments/my_env/lib/python3.5/site-packages```
This would let you use a globally installed IPython with your virtualenv. However, the typical way to do this would be to install a second copy of IPython in your virtualenv and use that copy.
```/home/abigail/environments/my_env/bin/ipython```
The activate shell commands for a virtualenv only do two things:
Add the virtualenv Python interpreter to your PATH. So when you type python3 you run /home/abigail/environments/my_env/bin/python3 instead of /usr/bin/python3. It is this binary at /home/abigail/environments/my_env/bin/python3 which will automatically include the /home/abigail/environments/my_env/lib/python3.5/site-packages location on the sys.path.
Change your PS1 environment variable so your terminal has a prompt to remind you which virtualenv you are working in.
It is up to you to use the activate shell script or not (as it's just very simple helper script, you can adjust environment in whatever way makes sense for yo). If you are only using one virtualenv, you can add exports to your ~/.bashrc file instead, e.g.:
```export PATH=/home/abigail/environments/my_env/bin/:$PATH```
Would automatically make python3 run your virtualenv Python the same as running source activate within your virtualenv.
Generally speaking for a virtual environment you will want to do an install to get the module you are looking to import to pre-pend correctly in your path variable at virtual environment activation time. Consider trying this:
Since it looks like you already have a virtual environment set up, and it looks like you are using some form of Unix/Linux:
/home/abigail/environments/ $ source my_env/bin/activate
You should then see your terminal look something like:
(my_env) /home/abigail/environments
that means you have an active virtual environment.
Next you should install the module you want. I am assuming that module is available via pip install.
(my_env) /home/abigail/environments $ pip install mglearn
This should get you all set up. When you check your sys path you should now see at the front of it your virtual environement python stuffs. And your import error should go away.
You may need to delete out the copy of mglearn you dropped into the directories manually if things get stuck.
I had some problem on installing python + virtualenv + django and need help.
System: Windows 7, 64b
What i do?
1) Installed Python 2.7.2 (32bits)
2) Installed SetupTools (32 bits)
3) Installed VirtualEnv
E:\APPZ\Console2>C:\Python27\Scripts\easy_install.exe virtualenv
4) Created virtualenv:
E:\APPZ\Console2>virtualenv E:\CODE\wamp\www\AMBIENTES\env
5) Fine, now I created a ".bat" to use my env and put then in C:\Windows.
C:\Windows\python.bat
cmd.exe /k E:\CODE\wamp\www\AMBIENTES\env\Scripts\activate.bat
So far so good
Now I executed the python.bat and installed django:
E:\APPZ\Console2>python
E:\APPZ\Console2>cmd.exe /k E:\CODE\wamp\www\AMBIENTES\env\Scripts\activate.bat
(env) E:\APPZ\Console2>cd E:\CODE\wamp\www\AMBIENTES\Django-1.2.7
(env) E:\CODE\wamp\www\AMBIENTES\Django-1.2.7>python setup.py install
django installed (1.2.7) successfully.
And now, the problem:
(env) E:\CODE\wamp\www\AMBIENTES\Django-1.2.7>E:\CODE\wamp\www\AMBIENTES\env\Scripts\django-admin.py --version
Traceback (most recent call last):
File "E:\CODE\wamp\www\AMBIENTES\env\Scripts\django-admin.py", line 2, in <module>
from django.core import management
ImportError: No module named django.core
(env) E:\CODE\wamp\www\AMBIENTES\Django-1.2.7>
-
Does anyone know what I can do about it?
I know this question is old and maybe not actual anymore for author. But as far as it appears at Google's top, I would leave the answer that helped me.
Basically the correct answer is posted for the similar question.
Strictly speaking the wrong Python installation is called when you execute django-admin.py --version. in order to check which Python you use in the case, type ftype Python.File in "command line". If it's not the virtualenv's one, then you could reassociate the default Python:
ftype Python.File="E:\CODE\wamp\www\AMBIENTES\env\Scripts\python.exe" "%1" %*
Or unset the file association (from cmd.exe):
assoc .py=
ftype Python.File=
After you reassociate the .py extension program, you should specify full path to execute Python files:
E:\CODE\wamp\www\AMBIENTES\env\Scripts\python.exe E:\CODE\wamp\www\AMBIENTES\env\Scripts\django-admin.py --version
Or if you want, you could edit virtualenv's activate.bat to put specific .py association, using assoc and ftype command line utils, mentioned above.
I believe your problem is that using python setup.py install with the Django source is installing Django in your primary site-packages/dist-packages path instead of that of your virtual environment.
Instead, use pip or easy_install:
$ pip install Django==1.2.7 --OR -- $ easy_install Django==1.2.7
If you can't directly download from PyPi (corporate firewall, etc.) you can use the source you already have by modifying the command slightly:
$ pip install -f file:///E/CODE/wamp/www/AMBIENTES/ Django==1.2.7
(Converted Windows path may need some tweaking. I think that's right, but it's been awhile)