Here's my problem. I can't install MatPlotLib both via pip and from source (Matplotlib installation on Mavericks). I tried brew install matplotlib and the installation successfully ended. However, it installed MatPlotLib globally and not inside the currently activated VirtualEnv.
Is it possible to tell brew to install a package inside the current VirtualEnv?
I help maintain the homebrew-python tap. This is not and will not be supported and there probably isn't an easy way to do it.
This is due to the fact Homebrew installs packages to Homebrew's prefix, which is necessarily global. Homebrew doesn't know about virtualenvs or how to install packages into them. Changing that would violate a core assumption of Homebrew's design, i.e. that packages are always installed to their private Homebrew prefix.
Sorry!
All that a virtual environment does is change what Python interpreter will be invoked by modifying the PATH variable. Homebrew doesn't look at that because it doesn't use Python to install packages (like pip does). By design, as Tim said, Homebrew installs inside its own prefix every time.
One could conceive of a modified Python virtual environment that also installs its own copy of Homebrew, but that would be almost entirely a bad thing.
What you can do, if you want Homebrew packages in a virtual environment, is to enable system site packages in the venv (in Python3, this would be python3 -m venv --system-site-packages ~/path/to/venv). This can be done to update existing virtual environments, as well.
Enabling system site packages allows Python to use globally installed packages, but it will still search inside the venv first, and you can override global packages by installing something with pip. As far as I can tell, this is as close as you'll get to The Right Way in that situation.
Related
Running command line:
virtualenv --system-site-packages venv
I'm expecting venv folder venv\Lib\site-packages to contain all the necessary library from the projects that are located in:
C:\Users\XXX\AppData\Local\Programs\Python\Python36\Lib\site-packages\
But it's not the case, only a few are installed.
Example, my program currently use pdfminer which is in
C:\Users\XXXX\AppData\Local\Programs\Python\Python36\Lib\site-packages\
I want it to be included in venv\Lib\site-packages but it is not copied.
Any advice?
--system-site-packages doesn't copy packages, it just allows python from the virtualenv to access packages in C:\Users\XXX\AppData\Local\Programs\Python\Python36\Lib\site-packages\.
There is no way to copy packages because they could depend on their installation directory. If you want these packages in the virtualenv don't use --system-site-packages and install all packages in the virtualenv.
A virtualenv environment is the same as if you have just installed a new version of Python. It has no packages other than the standard packages provided with Python. If you want other packages, you have to install them with 'pip' or however you'd do it with the native Python version that you are using.
So in general, just do pip install <packagename>.
If you find yourself often wanting to create virtualenvs with a standard set of base packages, then put together a requirements.txt file listing all of the packages you want to install as a base, and do pip install -r requirements.txt inside a new virtualenv, right after you create it.
One nice thing about a virtualenv is that it's all yours. Your user owns 100% of it, unlike the base Python version that is owned by the system. To install new packages into the base Python version, you often have to have root access (sudo privileges). With virtualenvs, you don't need special permissions (in fact, you'll get all screwed up if you use sudo in a virtualenv) to install all the packages you want. Everything you do lives within your own home directory. Another neat thing is that when you are done with a virtualenv, you just throw away the root directory that contains it.
If its not mandatory to use virtualenv, I would suggest to go with Anaconda. That'll pretty much help your concern.
Conda as a package manager helps you find and install packages. By default quite a few packages are already installed, so as to set you up quickly for your project. To check the list of packages installed in terminal, type: conda list to obtain the packages installed using conda.
If you need a package that requires a different version of Python, you do not need to switch to a different environment manager, because conda is also an environment manager.
With just a few commands, you can set up a totally separate environment to run that different version of Python, while continuing to run your usual version of Python in your normal environment
I'm new to python, so please be gentle.
In learning python and writing my first few scripts, I quickly glossed over any tutorial sections on virtualenv, figuring it wouldn't provide me any benefit in my nascent stage.
I proceeded to hack away, installing packages as I went with pip3 install package
Now I've built something that is potentially useful to my organization, and I'd like to share it. In this case, I want to distribute it as a windows executable.
Before building this distribution, I figure it's now time to take the next leap from individual scripts to proper python projects. It seems like virtualenv should be part of that.
Given that I've installed a number of packages to my "base" python environment: in order to do development in a "clean" virtual environment, do I need to somehow "revert" my base python environment (i.e. uninstall all non-standard packages), or will virtualenv shield a project within a virtual environment from non-standard packages installed to my "base" environment?
If you are using the venv module there is --system-site-packages flag that will grant the created virtual environment access to the system-wide site-packages directory:
--system-site-packages
Give the virtual environment access to the system
site-packages dir.
Go install VirtualEnvWrapper first. After that, create a new virtualenv, activate it, and run pip freeze. You should see nothing in there because nothing is installed. Deactivate the env to go back to your 'Base' environment and pip freeze again. You will see all the installs you have.
A best practice is to create a requirements.txt file and version control it so everyone can use the same versions of the same packages. If you don't want to do this, simply activate your new virtual env and pip install everything you want.
You can specify separately the required libraries and check if they are installed and if not then you can install them automatically.
Have a look at:
https://packaging.python.org/discussions/install-requires-vs-requirements/
We have an application which does some of its work in Python in a python virtual environment setup using virtualenv.
We've hit a problem where the version of a system library does not match the version installed in the virtual environment. That is we have NetCDF4 installed into the virtual environment and and previously had libnetcdf.so.7 installed through yum. The python package appears to be dependent on having libnetcdf.so.7 available.
Due to a system update libnetcdf.so.7 no longer exists and has been replaced by libnetcdf.so.11.
So the question is this: Does setting up the virtual environment detect the system library version or is there some other mechanism? Also do we need to re-build the environment to fix this or is there another option?
When you use virtualenv to create a virtual environment you have the option of whether or not to include the standard site packages as part of the environment. Since this is now default behaviour (though it can be asserted by using --no-site-packages in the command line) it's possible that you are using an older version of virtualenv that doesn't insist on this.
In that case you should be able to re-create the environment fairly easily. First of all capture the currently-installed packages in the existing environment with the commmand
pip freeze > /tmp/requirements.txt
Then delete the virtual environment, and re-create it with the following commands:
virtualenv --no-site-packages envname
source envname/bin/activate
pip install -r /tmp/requirements.txt
However none of this addresses the tricky issue of not having the required support libraries installed. You might try creating a symbolic link to the new library from the old library's position - it may be thatNetCDF4 can work with multiple versions of libnetCDF and is simply badly configured to use a specific version. If not then solving thsi issue might turn out to be long and painful.
I am a ruby programmer trying to learn python. I am pretty familiar with pyenv since it is like a copy and paste from rbenv. Pyenv helps allow to have more than one version of python in a system and also to isolate the python without touching sensitive parts of system.
I suppose every python installation comes with pip package. What I still don't understand is, there are many good python libs out there that suggest to use this virtualenv and anaconda. I can even find a virtualenv plugin for pyenv.
Now I am getting confused with the purpose of these two pyenv and virtualenv.
worse inside pyenv there is a virtualenv plugin.
My questions are:
what is the difference between pyenv and virtualenv?
Is there any difference in using pip command inside both pyenv and virtualenv?
what does this pyenv virutalenv do?
Your explanation with example will be highly appreciated.
Edit: It's worth mentioning pip here as well, as conda and pip have similarities and differences that are relevant to this topic.
pip: the Python Package Manager.
You might think of pip as the python equivalent of the ruby gem command
pip is not included with python by default.
You may install Python using homebrew, which will install pip automatically: brew install python
The final version of OSX did not include pip by default. To add pip to your mac system's version of python, you can sudo easy_install pip
You can find and publish python packages using PyPI: The Python Package Index
The requirements.txt file is comparable to the ruby gemfile
To create a requirements text file, pip freeze > requirements.txt
Note, at this point, we have python installed on our system, and we have created a requirements.txt file that outlines all of the python packages that have been installed on your system.
pyenv: Python Version Manager
From the docs: pyenv lets you easily switch between multiple versions of Python. It's simple, unobtrusive, and follows the UNIX tradition of single-purpose tools that do one thing well. This project was forked from rbenv and ruby-build, and modified for Python.
Many folks hesitate to use python3.
If you need to use different versions of python, pyenv lets you manage this easily.
virtualenv: Python Environment Manager.
From the docs: The basic problem being addressed is one of dependencies and versions, and indirectly permissions. Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into /usr/lib/python2.7/site-packages (or whatever your platform’s standard location is), it’s easy to end up in a situation where you unintentionally upgrade an application that shouldn’t be upgraded.
To create a virtualenv, simply invoke virtualenv ENV, where ENV is is a directory to place the new virtual environment.
To initialize the virtualenv, you need to source ENV/bin/activate. To stop using, simply call deactivate.
Once you activate the virtualenv, you might install all of a workspace's package requirements by running pip install -r against the project's requirements.txt file.
Anaconda: Package Manager + Environment Manager + Additional Scientific Libraries.
**Anaconda is a commercial distribution of Python with the most popular python libraries, you are not permitted to use Anaconda in an organisation with more than 200 employees.
From the docs: Anaconda 4.2.0 includes an easy installation of Python (2.7.12, 3.4.5, and/or 3.5.2) and updates of over 100 pre-built and tested scientific and analytic Python packages that include NumPy, Pandas, SciPy, Matplotlib, and IPython, with over 620 more packages available via a simple conda install <packagename>
As a web developer, I haven't used Anaconda. It's ~3GB including all the packages.
There is a slimmed down miniconda version, which seems like it could be a more simple option than using pip + virtualenv, although I don't have experience using it personally.
While conda allows you to install packages, these packages are separate than PyPI packages, so you may still need to use pip additionally depending on the types of packages you need to install.
See also:
conda vs pip vs virtualenv (section in documentation from anaconda)
the difference between pip and conda (stackoverflow)
the relationship between virtualenv and pyenv (stackoverflow)
Simple analogy:
pyenv ~ rbenv
pip ~ bundler
virtual env ~ gemset in rvm. This can be managed by bundler directly without gemset.
Since I use python3 I prefer the python3 built-in virtual environment named venv. venv is simple and easy to use. I would recommend you to read its official docs. The doc is short and concise.
In ruby, we don't really need a virtual environment because the bundler takes care of it. Both virtual env and bundler are great, however, they have different solutions to solve the same problem.
Simple explanation: https://docs.conda.io/projects/conda/en/latest/commands.html#conda-vs-pip-vs-virtualenv-commands
If you have used pip and virtualenv in the past, you can use conda to perform all of the same operations.
Pip is a package manager
virtualenv is an environment manager
Conda is both
Virtualenvwrapper is a user-friendly shell around Python's virtualenv.
Python 3.3 ships with pyvenv built into the standard library, which aims to supercede virtualenv.
But if I install Virtualenvwrapper on Python3.3, it still installs virtualenv, leading me to believe it doesn't use 'pyvenv' under the covers.
Presumably this doesn't really matter - if I wish to use virtualenvwrapper on Python3.3 I should happily let it use virtualenv instead of pyvenv, and will (for the moment) suffer no ill effects?
Sorry this answer is a bit delayed. pyvenv does not aim to supersede virtualenv, in fact virtualenv in Python 3 depends on the standard library venv module.
The pyvenv command creates an absolutely minimal virtual environment into which other packages can be installed.
The Python 3 version of virtualenv actually subclasses the standard library's implementation and provides hooks to automatically install setuptools and pip into the environment which pyvenv doesn't do on it's own.
As far as I know virtualenvwrapper depends on virtualenv only because the mkvirtualenv or mkproject commands allow you to specify packages to be installed into the new environment, this only works because virtualenv will have already installed setuptools and pip.
So to answer your question I believe you should be able to use virtualenvwrapper on environments created by pyvenv as long as you follow virtualenvwrapper's conventions for where to put things and you either manually install setuptools and pip into the environment or don't use any package management features of virtualenvwrapper.