I have a system with certain python version and packages installed suing the distribution repositories. For some project (calculation) I need newer version the the packages. I am thinking of installing anaconda and use conda virtual environments. Will this broke programs that must use the system packages?
(note: I tried virtual enviroment, but I couldn't install a newver version of matplotlib, because of problems with pygtk)
No this will not break your system's python. As long as you don't tick the option "register miniconda as the default system python" (or whatever that option is called depending on your OS).
One of the key benefits of conda is that you can create isolated python environments, fully independent of each other.
Related
I had first installed python using the standard python distribution available on their official website and I would be using pip to install all necessary packages.
However, now I wish to use miniconda, since it is a better choice for data science.
But, it installs python along with It and I don't want to disturb my earlier setup of pip+Python.
Will installing miniconda affect my python installation.
Is there a way of installing it without disturbing the python installation?
I am on a Windows operating system.
You can safely install Anaconada (or Miniconda) on top of other Python installations. It goes into a completely different folder on your local disk. But leave the default installation options on default, especially don't add Python to your path.
The important thing is that you activate your environment before you use it via
conda activate
and then start Python from there (or let your IDE do that for you).
(base)> python
Without activatation conda doesn't work and calling python from the command prompt will start your 'standard installation' again.
The advantage of Anaconda is that it guarantees maximumum consistency for the 'scientific stack' and in case you are still missing some 3rd party packages you can always install them aditionally via `pip install' into an activated conda environment.
When installing packages with sudo apt-get install or building libraries from source inside a python virtual environment (I am not talking about pip install), does doing it inside a python virtual environment isolate the applications being installed? I mean do they exist only inside the python virtual environment?
Things that a virtual environment gives you an isolated version of:
You get a separate PATH entry, so unqualified command-line references to python, pip, etc., will refer to the selected Python distribution. This can be convenient if you have many copies of Python installed on the system (common on developer workstations). This means that a shebang line like #!/usr/bin/env python will "do the right thing" inside of a virtualenv (on a Unix or Unix-like system, at least).
You get a separate site-packages directory, so Python packages (installed using pip or built locally inside this environment using e.g. setup.py build) are installed locally to the virtualenv and not in a system-wide location. This is especially useful on systems where the core Python interpreter is installed in a place where unprivileged users are not allowed to write files, as it allows each user to have their own private virtualenvs with third-party packages installed, without needing to use sudo or equivalent to install those third-party packages system-wide.
... and that's about it.
A virtual environment will not isolate you from:
Your operating system (Linux, Windows) or machine architecture (x86).
Scripts that reference a particular Python interpreter directly (e.g. #!/usr/bin/python).
Non-Python things on your system PATH (e.g. third party programs or utilities installed via your operating system's package manager).
Non-Python libraries or headers that are installed into a operating system specific location (e.g. /usr/lib, /usr/include, /usr/local/lib, /usr/local/include).
Python packages that are installed using the operating system's package manager (e.g. apt) rather than a Python package manager (pip) might not be visible from the the virtualenv's site-packages folder, but the "native" parts of such packages (in e.g. /usr/lib) will (probably) still be visible.
As per the comment by #deceze, virtual environments have no influence over apt operations.
When building from source, any compiled binaries will be linked to the python binaries of that environment. So if your virtualenv python version varies from the system version, and you use the system python (path problems usually), you can encounter runtime linking errors.
As for isolation, this same property (binary compatibility) isolates you from system upgrades which might change your system python binaries. Generally we're stable in the 2.x and 3.x, so it isn't likely to happen. But has, and can.
And of course, when building from source inside a virtualenv, installed packages are stashed in that virtualenv; no other python binary will have access to those packages, unless you are manipulating your path or PYTHONPATH in strange ways.
I already have Python 2.7 installed in Windows. I have normally used pip to install packages. However, Pandas recommends using Anaconda and it appears that it has many benefits so I wanted to try it.
I installed miniconda and it just reinstalled Python under its own directory. Does Anaconda always duplicate the python libraries or can it be used to manage the system's python.
I use python to develop and also wanted to use Pandas to analyse data. However, I would like to avoid have two copies of Python. I want to have one python environment that is constant with all the packages that I intend to have. Otherwise, I feel that I will have to install the same packages multiple times.
I know that Anaconda is to separate different environments. Does this mean that I am trying to do something it is not its purpose or have I installed it incorrectly?
Anaconda has a root environment that includes a bit more than 100 of the most popular Python packages.
Yes, you can use the root Python as your system's Python executable.
The anaconda installation comes with Conda, which is a robust environment manager. If you want to keep your root environment stable, you can use Conda to create new environments for each project, and Conda handles the dependencies of each environment as well.
You can create a new environment named "analysis" that has Python, IPython, and Pandas using:
conda create --name analysis python ipython pandas
After installing all of the packages, you can use the environment by running (from the CMD prompt):
conda activate analysis
We have an application which does some of its work in Python in a python virtual environment setup using virtualenv.
We've hit a problem where the version of a system library does not match the version installed in the virtual environment. That is we have NetCDF4 installed into the virtual environment and and previously had libnetcdf.so.7 installed through yum. The python package appears to be dependent on having libnetcdf.so.7 available.
Due to a system update libnetcdf.so.7 no longer exists and has been replaced by libnetcdf.so.11.
So the question is this: Does setting up the virtual environment detect the system library version or is there some other mechanism? Also do we need to re-build the environment to fix this or is there another option?
When you use virtualenv to create a virtual environment you have the option of whether or not to include the standard site packages as part of the environment. Since this is now default behaviour (though it can be asserted by using --no-site-packages in the command line) it's possible that you are using an older version of virtualenv that doesn't insist on this.
In that case you should be able to re-create the environment fairly easily. First of all capture the currently-installed packages in the existing environment with the commmand
pip freeze > /tmp/requirements.txt
Then delete the virtual environment, and re-create it with the following commands:
virtualenv --no-site-packages envname
source envname/bin/activate
pip install -r /tmp/requirements.txt
However none of this addresses the tricky issue of not having the required support libraries installed. You might try creating a symbolic link to the new library from the old library's position - it may be thatNetCDF4 can work with multiple versions of libnetCDF and is simply badly configured to use a specific version. If not then solving thsi issue might turn out to be long and painful.
I need to use the SciPy libraries inside the PyCharm IDE (on a Mac OSX Lion machine). The SciPy website writes that the simplest installation method for Mac users is to install Anaconda (or an equivalent distro). I used the Anaconda installer, and it created an anaconda directory in my home folder, where I find a lib/python2.7/site-packages directory with the required packages. However, PyCharm is not aware of all this and the SciPy import statements remain unresolved.
My question is then how to make PyCharm work with Anaconda?
I'm still coming to terms with the Python ecosystem and PyCharm, so take the following with a grain of salt, but after reading up a bit, I thought I'd write a detailed explanation.
During installation, Anaconda changes the default Python interpreter to ~/anaconda/bin/python. This interpreter is configured with a sys.path that defaults to the libraries in ~/anaconda/lib . Package managers like python's pip use the interpreter that's running them to determine the path in which to install packages, so after Anaconda is installed, all packages installed via pip or other methods will be placed somewhere inside ~/anaconda/lib. i.e. even without using something like virtualenv, every Python interpreter has its own ecosystem and running pip with different interpreters will install packages into different directories.
PyCharm handles all this in the Project Settings-->Project Interpreter-->Python Interpreters screen. To make PyCharm aware of the Anaconda distribution, you need to add the Anaconda python interpreter to the Project Interpreter-->Python Interpreters list and make it the default for the project. PyCharm will then locate all packages installed in Anaconda's interpreter ecosystem (~/anaconda/lib) and list them under packages in the lower pane. It will also prompt you to install setup_tools and pip for that interpreter, and once you do that you'll be able to use the install button in the lower pane to add more packages to the Anaconda ecosystem.
Once you've added the Anaconda interpreter, you can also use the virtualenv button from the toolbar on the upper pane to create a virtualenv that inherits from the Anaconda interpreter's environment. That way you can install new packages in a way that would not affect the global Anaconda distribution.