I am new to Python. I am a very old R programmer. One of my PhD courses is performed via Python.
So, I installed Jupyter Notebook; I wrote simple "Hello World" in Jupyter Notebook. I wonder whether I must install Anaconda or not?
I ask this because: I looked at Anaconda's IDE in youtube, and it shows RStudio, Jupyter Notebook, etc. in a bundled manner. In RStudio, one can perform all the package handlings within RStudio.
So, I wondered is there a way to install python packages within Jupyter Notebook, or should I really install Anaconda? What is the benefit of installing Anaconda (besides Jupyter Notebook)?
The packages in python are generally installed with "pip" (or "conda") in the terminal and are then available regardless where you run your script from. Assuming you don't have multiple python versions set up on your PC, they should then all be available in your jupyter-notebook also.
If you don't want to open another window to do this, you can also run BASH code from Jupyter itself, just start the line with an exclamation mark (!)
i.e.
!pip install pandas
The benefits of Anaconda are that it bundles everything you need to at least start your more basic projects (python release, basic packages, IDE) and that you can set-up project-specific environments that do not interfere with your system-wide package installations.
Python packages are not installed with python functions, like it would be the case in R with install.packages("package name"). Instead, an external package manager usually is used to install and possibly compile the package files to a directory where python can import it from.
Anaconda offers (among others) the package manager conda. Most popular is pip. Some Linux distributions also offer python packages via their package manager (e.g. apt on Debian/Ubuntu). All these package managers download packages from their own repositories, so conda install numpy, pip install numpy and apt install python3-numpy all install a package numpy, but from different sources and in possibly different versions.
Jupyter Notebook is a programming environment, where you can execute shell commands with !command, so depending on the system where the Jupyter server is running, you can use !pip install numpy, !conda install numpy or other commands as cells in the Jupyter Notebook you are working in. This will run the command in a shell.
That graphical menu with Jupyter, RStudio etc. you describe is the program "Anaconda Navigator", which is installed with Anaconda. Jupyter is just a Python library, which is pre-installed with Anaconda, but can also be installed via pip, apt and other package managers.
Like Virtualenv, Anaconda also uses the concept of creating environments so as to isolate different libraries and versions. Anaconda also introduces its own package manager, called conda, from where you can install libraries.
Additionally, Anaconda still has the useful interaction with pip that allows you to install any additional libraries which are not available in the Anaconda package manager.
It is a good option for setting up of a better environment for working with jupyter.
I want to install PyPy on my Ubuntu and I want it to be installed system-wide so I could call PyPy in terminal everywhere like: pypy main.py. I also want to have standard python and pip available.
I'm not a very experienced Linux user yet so I'm lost.
As #keith-thompson says, you can do sudo apt install pypy pypy-dev. If you want a more up-to-date version you can do snap install pypy pypy-dev. Another way to get a working environment is to use conda, as per this link. The advantage of conda is that you can easily install pre-built packages such as numpy, sciy (coming soon), and more. If you use the first recipes, pip install will currently have to build many packages from source.
Note that pip is never "available anywhere", it is tied to the particular python instance, and installs packages into a path for that specific instance. Thus the pip you use for python2 is different than the pip you use for python3, likewise for pypy.
I want to upgrade my base environment to Python 3.8. According to the official documentation, I should just run conda install -c anaconda python. This, however, only upgrades Python 3.7, so, in the end:
(base) C:\Users\bob>conda install -c anaconda python
Collecting package metadata (current_repodata.json): done
Solving environment: done
# All requested packages already installed.
(base) C:\Users\bob>python --version
Python 3.7.6
conda update -c anaconda python only updated a number of packages, but not Python itself.
And if I specify a version, like explained here, after two hours analyzing dependencies, the upgrade fails because of conflicts.
I managed to set up a 3.8 environment, but I'd rather upgrade my base environment as it's the one my system path for use outside of Anaconda.
Any suggestion?
It doesn't appear to be possible to update the base version of anaconda to 3.8 yet because of the conflicts you mention. I assume once all the standard packages are updated to support 3.8+ (and updated versions of all dependencies), then the anaconda release will include 3.8.x as the default. This might take a while.
The problem here is version conflicts in the packages that anaconda installs by default. The reason you can install 3.8 in a clean environment is because no other packages are installed in that environment, so no conflicts.
This does now present a solution, although it requires reinstalling everything from scratch...
disclaimer: I don't use windows, so adapt my instructions accordingly to your OS.
Remove your existing anaconda install from the path and delete
anything in your bashrc (or windows equivalent) that points to it.
You can just delete/uninstall anaconda, but it's good to have a backup just incase, so leaving it there and removing links to it is a good option.
Download and install miniconda, and put it in your path as you had before.
Install python 3.8 in the base environment conda install python=3.8
Install all the packages that you need to do whatever you do.
Depending on what you install, at some point you may run into the same conflicts that prevent a standard anaconda install from updating at the moment, but that hasn't happened for me.
The latest stable version of anaconda is version 3.7
Please see here:
https://www.anaconda.com/distribution/
There isn't an option to get python 3.8, this is due to anaconda sometimes running a version behind the release of python.
I had the same issue and found there is a 2020-07 release of Anaconda with Python 3.8 https://www.anaconda.com/products/individual
I installed Anaconda 3 2.3.0 on Windows xp. It is supposed to be the last version of anaconda to support windows xp as it contains python 3.4.3, and python 3.4.x is the last version of python to support windows xp.
When installing a package with
conda install
,one of its dependencies was updating conda to conda 4.x. conda 4.x crashed in the commandline when running conda install . This made anaconda unusable that i uninstalled anaconda.
My question, is it possible to lock versions of packages ? For the two use cases:
lock and forget : for packages I never want them to update, i need to run a command to lock them once so they will never update as a depency
ignore updating : install a package while ignoring to update certain package passed by name in this update.
If only one of the 2 use cases is possible or is known or is easier, please write it as answer.
There are two bits to this. First, you can prevent conda from auto-updating by changing the configuration option auto_update_conda to False:
conda config --set auto_update_conda False
The other bit is to pin packages to certain versions. For those packages you don't want to update, you can pin the version by adding a line to a file called pinned (you might need to create it) in the environment's conda-meta directory. The syntax is
[The code] below [placed in conda-meta/pinned] forces NumPy to stay on the 1.7 series, which is any version that starts with 1.7, and forces SciPy to stay at exactly version 0.14.2:
numpy 1.7.*
scipy ==0.14.2
See the documentation for more information.
Pin for conda install (=): package=1.0
vs.
Pin for pip install (==): package==1.0
Try
pip install <package name==version no*>
for instance
pip install musdb==0.2.*
That would pin your package
I'm trying to install the basic scipy stack (numpy, scipy, matplotlib, pandas, sympy, ipython, nose) into a virtualenv; currently, I'm using OSX Mountain Lion. From the installation pages for these packages, as well as various threads here and around the web, it seems that pandas, sympy, and nose can be installed easily with just pip (although some list numpy/scipy/etc. as dependencies).
However, there seems to be conflicting and kind of convoluted instructions for properly installing numpy, scipy, matplotlib, and, to a certain extent, ipython*. Installing solely with pip doesn't seem to be the proper way to install these packages; it seems that some dependencies need to be installed with homebrew, but various places list different packages to brew install before pip installing numpy/scipy/etc. Is there a comprehensive and current list of package dependencies that need to be installed with homebrew before pip installing numpy, scipy, and matplotlib?
Just as a note, I've looked at tapping homebrew/python for installing numpy, scipy, and matplotlib properly with homebrew, but I want to install into a virtualenv and I don't think I can use homebrew to do that.
Any help would be greatly appreciated; thanks in advance!
*Also, for ipython, the installation page says that pip install ipython[all] should be sufficient, but some other sources (http://www.coderstart.com/setup/python-setup.html; http://www.lowindata.com/2013/installing-scientific-python-on-mac-os-x/) seem to suggest that the qt, pyqt, and zmq packages need to be brew installed first before pip installing ipython; are the homebrew installation necessary or is it fine to just pip install according to the ipython installation page?
Answering my own question here, but hopefully this is helpful; feel free to correct if there are any mistakes. The original version is a gigantic wall of text, so I've added a tl;dr to the top with just the steps to hopefully make the process more clear.
tl;dr: In terminal/bash, go into a virtualenv (if you want to install into one) and enter these commands in order. This is tested for OSX Mountain Lion.
pip install numpy
brew install gcc
pip install scipy
brew install freetype
pip install matplotlib
pip install nose
pip install pandas
pip install sympy
pip install ipython[all]
brew install pyqt
brew install qt
brew install sip
echo "export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH" >> ~/.bash_profile
source ~/.bash_profile
*Notes: brew installing pyqt might already install qt and sip; if it does, no need to install qt and sip after installing pyqt. For the second to last line, might be be more reliable to directly c/p that line into ~/.bash_profile, since it might need to be at the top of the ~/.bash_profile contents. Also, brew install pandoc is optional, but necessary for ipython notebook's nbconvert command work properly.
edit 10/13/14 [see edit at bottom]: editing PYTHONPATH in ~/.bash_profile forces virtual environments to inherit global packages; if you want to be able to make isolated environments, don't do the last two steps. Instead, assuming virtualenvwrapper is installed, edit the local postactivate and predeactivate scripts in the bin directory under the virtualenv that contains the scipy stuff.
In postactivate, enter:
export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH
In predeactivate, enter:
unset PYTHONPATH
This should edit PYTHONPATH when when the virtualenv with scipy stuff is activated so that ipython qtconsole works, but should reset PYTHONPATH when the virtualenv is deactivated, so that other virtualenvs are not affected by PYTHONPATH changes.
Below is the long narrative version.
Anyway, from some trial and error after I originally posted the question, I found that the following steps work; I used the two sets of instructions that I linked above as general guides, and this was tested on OSX Mountain Lion.
After activating the virtualenv in which the packages are going to be installed, first pip install numpy; this should work as expected and numpy should be installed (note that numpy should be installed first since a lot of the other packages in the scipy stack depend on numpy).
Now, before installing scipy, several sources note that gfortran (this seems to be the most common, but I think any fortran compiler should work) needs to be installed; brew install gfortran returns an error that says that gcc now includes gfortran so the gfortran formula is deprecated. Hence, we brew install gcc (note that even though xcode command line tools, which is generally installed before homebrew, already includes gcc, its version of gcc somehow doesn't work or doesn't include gfortran). After installing gcc, pip install scipy works as expected, and scipy should be installed. Quick aside: brew install gcc installs gcc as well as a bunch of dependencies, namely cloog, gcc, gmp, isl, libmpc, mpfr. [These should all be installed into /usr/local/Cellar, which is the default install location for homebrew.]
For installing matplotlib, freetype needs to be installed first, so we brew install freetype; this should install freetype and libpng, which seems to be a freetype dependency. Afterwards, pip install matplotlib works as expected, installing matplotlib successfully. Note that matplotlib installs with mock (needed to run matplotlib test suite), pyparsing (required for mathtext support), python-dateutil (required for date axis support), six (no reason given) along with it. [These should all be installed, along with any other pip install within a virtualenv, into the site-packages directory within the virtualenv.]
Installing nose, sympy, and pandas simply require pip installing, as they don't have any dependencies that need to be brew installed. However, of these, note that at least pandas depends on numpy, so installing pandas (not sure about the others) after installing numpy is probably preferred. Also, note that pandas installs with pytz (for time zone calculations).
Installing ipython is pretty simple, but setting it up is a little more convoluted. As a heads up, ipython can be used with both a qt console and something called ipython notebook, which have various benefits. You can choose to just install ipython with pip install ipython and install optional dependencies later on as needed, but I installed all the main optional dependencies with pip install ipython[all]. This installs ipython, along with a lot of other packages dependencies (installs with backports.ssl-match-hostname (from tornado), certifi (from tornado), docutils (from sphinx), gnureadline, ipython, jinja2, markupsafe (from jinja2), numpydoc (from ipython[all]), pygments, pyzmq, sphinx, tornado). This should be a good foundation for ipython to use both the standard ipython shell, the qt console, and the ipython notebook. However, it's not fully set up if you want to use either the qt console or the notebook.
To use the qt console, the pyqt, qt, and sip packages must be brew installed, as these are dependencies that cannot be installed with pip; from experience, brew install pyqt seems to install all three packages, but individually installing the three might be a safer bet. After doing this, go into ~/.bash_profile and add the line 'export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH' to it; then 'source ~/.bash_profile' in terminal to reload the shell. This should successfully allow the qt console to be launched. [I'm not totally sure why this line needs to be added, since I had already edited the PATH variable to put /usr/local/bin before /usr/bin, but perhaps qt/pyqt/sip were still trying to link themselves with the system default python instead of homebrew installed python.]
The notebook seems to work fine out of the box as far as I've seen, but there's one thing of note: in order to use nbconvert (converting the notebook to different file formats), the pandoc package must be installed, presumably with homebrew. Like qt/pyqt/sip, it can't be installed with pip which is why it wasn't installed with pip install ipython[all].
edit 10/13/14: So apparently, editing PYTHONPATH will nullify empty virtual environments, making global packages also available in a virtualenv (how to isolate virtualenv from local dist-packages?); this, for the most part, defeats the purpose of a virtualenv, assuming that you want to a fresh environment, but is necessary for ipython qtconsole to work correctly.
The fix is to edit (assuming virtualenvwrapper is installed) the local postactivate and predeactivate scripts in the bin file of your virtualenv. In postactivate, enter the line "export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH; in predeactivate, enter the line "reset PYTHONPATH". Don't do the last two steps of the original sequence, or delete the line that was added to ~/bash_profile. This should make it so that the changes to PYTHONPATH are made only when the virtualenv with our installed packages is activated, so that qtconsole works, but are reset before the virtualenv deactivates, so that other environments are not affected.
I'm on OSX 10.9, but I had good luck following these instructions and get roughly the same environment that you outlined above
http://hackercodex.com/guide/python-development-environment-on-mac-osx/
Hope this helps