I have some experience with C++ and Fortran, and I want to start using python for my post-processing as I am starting to realise how inefficient MATLAB is for what I need to do (mostly involves plots with millions of points).
I already had a few versions of python installed, from every time I wanted to start using. It has now become a mess. In /usr/local/bin/, here is what the command ls python* returns:
python python2.7 python3 python3.5 python3.5m pythonw-32
python-32 python2.7-32 python3-32 python3.5-32 python3.5m-config pythonw2.7
python-config python2.7-config python3-config python3.5-config pythonw pythonw2.7-32
I now want a clean slate. I want a safe way to remove all the previous versions of python, including all of their packages, so I can just install the latest version and import all the libraries I want like numpy and matplotlib smoothly (I had some issues with that).
EDIT:
I am running on OSX Yosemite 10.10.
Do not uninstall your system's Python interpreter (Python 2.7 most probably). You might consider uninstalling the other version (Python 3.5 most probably), but I do not think you really need to do that (it may not be a bad idea to keep a system-wide Python 3 interpreter... who knows!).
If you want a clean state I would recommend you to use virtual environments for now on. You have two options:
Use virtualenv and pip to setup your virtual environments and packages. However, using pip means you will have to compile the packages that need compilation (numpy, matplotlib and many other scientific Python packages that you may use for your "post-processing").
Use Conda (or Miniconda). This way you will be able to handle virtual
environments but without having to compile Python packages yourself.
Conda also allows you to handle different Python interpreters without
the need of having them installed in your system (it will download
them for you).
Also, you say you are feeling MATLAB is inefficient for plotting millions of points. I do not know your actual needs/constraints, but I find Matplotlib to be very inefficient for plotting large data and/or real-time data.
Just as a suggestion, consider using PyQtGraph. If you still feel that is not fast enough, consider using VisPy (probably less functional/convenient at the moment, but more efficient).
Related
I have installed on my MacBook Air (v10.10.5) a number of different Python Versions (2.7, 2.6 + 3). I have tried changing python PATH settings, however, it has become too awkward to maintain. Because of this, I wish to uninstall all versions (without damaging my OS) and then clean install Python 3. How can I do this?
First of, you cannot (and should not!) uninstall a version of python that is shipped with macOS. They are shipped for a reason and might break your system as soon as you remove them (as a lot of essential processes might rely on them).
Python2.7 for instance has been shipped with macOS for a long time. However, you are also able to download and install the same version of python (e.g. 2.7) yourself, which usually means you end up with conflicting versions and controlling your PATH becomes a fulltime job.
You are always safe to remove those versions that you installed yourself (but only those!).
How to do so depends greatly on how you installed them. If you've done so via downloaded packages of python.org look here for uninstall instructions. If you used another application like brew or macports you need to follow the guidelines of those programs which could be easily fetched via a Google search.
Good luck!
I wish to install python numpy, scipy, and matplotlib packages. I used the following:
sudo port install py27-numpy py27-scipy py27-matplotlib py27-ipython +notebook py27-pandas py27-sympy py27-nose
I was able to install and run numpy, but not scipy or matplotlib. I get an error, "No module named scipy", even though installation appeared to proceed normally. I suspect that this has something to do with the version of scipy and bumpy. I am running OS X Version 10.8.4, python 2.7.2 and numpy 1.6.1. I suspect the problem is either the version or the directory which scipy is installed; scipy is in /Users/myname/src/scipy. I couldn't find out where numpy is located (how can this be done? Spotlight search does not indicate file directory, and grep hangs). How can I install scipy and other packages?
It's hard to be 100% sure without a little more information, but I'm 99% sure…
Your problem is that you're installing all this stuff for the MacPorts Python, but then trying to run Apple's Python, which doesn't have any of it installed. (Part of the reason I suspect this is that Apple's Python is version 2.7.2, while MacPorts has been on 2.7.5 for a long time now. The other part of the reason is that this is exactly the same kind of problem people have when they install two different Python X.Y versions without realizing that's what they're doing.)
If you want to use the MacPorts Python, run /opt/local/bin/python instead of just python.
If you know what you're doing, you can change your PATH, or create a symlink, etc., but I suspect you will just confuse yourself further.
I personally think you'd be a lot happier just sticking to Apple's Python. You need to install a Fortran compiler with MacPorts (or Homebrew, or manually, or …), but to install Python packages, just use pip.
(At this point, that last bit of advice probably won't help you, as the first pip on your PATH is probably the MacPorts one, and doesn't match the first python on your PATH. But if you're willing to start over, just port uninstall python, and then look at these blog posts or the dozens of related questions here on SO.)
I've installed Python 2.7 using macports and because of my $PATH variable, that's the one I get when I type $ python. However, virtualenv defaults to using Python 2.6 unless I remember to force it to do otherwise with the -p flag.
On a related note, globally running yolk -l shows the following:
Python - 2.6.1 - active development (/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload)
So my question is, should I do something to ensure the system is always using MacPorts' Python 2.7 or is it not worth worrying that Apple's Python 2.6.1 is apparently in the mix?
If you force python 2.7 globally on your system, things will break. You don't want to do that.
You should use virtualenvwrapper. It should choose your preferred python.
http://www.doughellmann.com/docs/virtualenvwrapper/install.html#python-interpreter-virtualenv-and-path
I have had similar problems, although not related specifically to virtualenv. I found that, after installing a particular version of python with macports, and having that version set as default, installing 3rd party modules that are build on or use python using macports solved all problems. You essentially end up with parallel builds, on top of different versions of python. I have a few, and switch between them using python_select. Also, see here on SO for how to get easy_install to work with your new python.
A colleague of mine wants to use my python 2 code on his OS X (10.6) machine. My code imports several built-in python packages, including Tkinter and shelve, and also uses third-party packages, including numpy, scipy, matplotlib, and ipython.
I've encountered a few problems with OS X's built-in python. (IDLE doesn't work, for example*). I suspect I should install a more recent version of python, and a different version of Tk.
My questions:
Will having two different versions of python/Tk on the same machine cause problems?
I would like to associate the terminal commands 'python', 'ipython', and 'easy_install' with the more recent version of python. How should I do this?
When I install third-party packages like numpy using a .dmg file, how do I control which version of python numpy installs into?
Is there a better way to do this?
If this process goes well, I'd consider adding OS X instructions to my code's documentation, so I'd like to boil down this process to the simplest, most general approach.
*EDIT: Also, this
EDIT: Thank you everyone for the useful answers. My colleague tried MacPorts, which seems to work well, but has a few speedbumps. First we had to install Xcode from the system install disk. This is not a fast or lightweight install (several GB). Luckily we still had the disk! Once Xcode was installed, MacPorts was easy to install. Python and the python subpackages we needed were also easy to install, but he told me this installation took several hours. Presumably this delay is due to compilation? He had an easy time setting the MacPorts python as default. However, I think we have to change the 'Python Launcher' application by hand, this seems to still default to the system python.
Even though he has a working system now, I'm tempted to ask him to try one of the other solutions. I'm not sure all of my code's potential users will tolerate a multi-hour, multi-gigabyte installation.
I use brew to install all my libraries/compilers/interpreters.
To install python try this:
brew install python
Then add Python's binaries directory to your $PATH in your ~/.profile:
export PATH=`brew --prefix python`/bin:$PATH
I'd recommend you to install pip, virtualenv and virtualenvwrapper to have better control over your environment too.
Have you tried ActivePython?
It includes a package manager (PyPM) that, by default, installs into your home directory (eg: ~/Library/Python/2.7). Main scripts get symlinked in /usr/local/bin; use the included pythonselect to set the active Python version.
You don't have to bother installing .dmg packages, as PyPM is a binary package manager ... therefore you can install non-pure Python packages like NumPy without having to compile things yourself.
ActivePython can use Apple's Tcl/Tk or, if installed, ActiveTcl.
A "simplest, most general approach" in your documentation could be:
Install ActivePython 2.7
Open Terminal and type pypm-2.7 install matplotlib ipython
Using MacPorts, you can install python 2.6, 2.7, 3.1 and 3.2 at the same time, with their own packages, without ever touching the built-in python.
numpy, scipy, matplotlib, and ipython are also available as ports for most of those python versions.
Moreover, if you install the python_select port, you'll be able:
to choose which one of those (plus the built-in python) is the "default" python;
to install python packages through easy_install/pip for the "selected" python, if they're not available as ports.
Add virtualenv to the mix, and you'll have a very, very flexible Python development environment.
As for your questions:
Q1: with MacPorts, no. while not a frequent user, I've installed and used matplotlib in 2.6 and 2.7, switching between the two using python_select.
Q2: easy_install, pip, ipython will be "linked" to the python they were installed by. (but see tip 1)
Q3: it's easier to install one of the py{26,27,xx}-numpy ports, or pip install numpy under your python_select'ed python.
Q4: well, MacPorts is the best thing I know after APT on Debian/Ubuntu... :-)
Now, two tips if you try MacPorts:
MacPorts cleanly installs ports separately from the OS X installation, in an /opt/local directory, and each python version is installed in a /opt/local/Library/Frameworks/Python.framework/Versions/{2.5,2.6,2.7,...} directory. Using python_select cleanly switch the "python" command using links. BUT... the Versions/{2.5,2.6,2.7,...}/bin directory, where python scripts are installed, is not added to the PATH. Just adding: export PATH=/opt/local/Library/Frameworks/Python.framework/Versions/Current/bin:$PATH to your ~/.profile will always give you direct access to the scripts installed for the selected python.
to avoid bad surprises, I've added a echo Selected python is \"$(python_select -s)\" line to my ~/.profile, so I always know which is my currently selected python when opening a session... :-)
Regards,
Georges
In almost all cases, the best python to use is the one from http://python.org/. It sets up the paths correctly and doesn't overwrite anything. DMG package installs usually work automatically, as does python setup.py install, and it's not too hard to get setuptools to work. If you want per-user installs, it is easy to set up .pydistutils.cfg and python automatically recognizes the path install_lib = ~/Library/Python/$py_version_short/site-packages
An addendum regarding the usage of brew:
Since some time, brew install python will install python3.
If you intend to install python2, you want to use
brew install python#2
It is perfectly fine to install both python and python3 using brew!
Here is an old post that answers your questions too.
In general it is not a problem at all to have more than one python installation on your machine. You just have to watch out which one you are calling on the command line.
>> which python
... helps to identify where your python binary is located. The original Mac OS X python is usually at "/usr/bin/python"
I personally use the MacPorts python installation. It also supports you with the installation of modules. (see link above)
I have 4 versions of python on my MacBook Pro. 2 from the original install of OS X 10.6 and a subsequent update, then self installed copies of python 2.7 and 3.2. You can update the python command to point at any of the versions. They all install in separate directories and cause no problems with each other.
I'm not sure what will happen when you install from a .dmg file. I believe it will simply use whatever version python points to.
This post on superuser.com answers your questions on changing default paths.
Fedora Core 9 includes Python 2.5.1. I can use YUM to get latest and greatest releases.
To get ready for 2.6 official testing, I wanted to start with 2.5.4. It appears that there's no Fedora 9 YUM package, because 2.5.4 isn't an official part of FC9.
I downloaded 2.5.4, did ./configure; make; make install and wound up with two Pythons. The official 2.5.1 (in /usr/bin) and the new 2.5.4. (in /usr/local/bin).
None of my technology stack is installed in /usr/local/lib/python2.5.
It appears that I have several choices for going forward. Anyone have any preferences?
Copy /usr/lib/python2.5/* to /usr/local/lib/python2.5 to replicate my environment. This should work, unless some part of the Python libraries have /usr/bin/python wired in during installation. This is sure simple, but is there a down side?
Reinstall everything by running easy_install. Except, easy_install is (currently) hard-wired to /usr/bin/python. So, I'd have to fix easy_install first, then reinstall everything.
This takes some time, but it gives me a clean, new latest-and-greatest environment. But is there a down-side? [And why does easy_install hard-wire itself?]
Relink /usr/bin/python to be /usr/local/bin/python. I'd still have to copy or reinstall the library, so I don't think this does me any good. [It would make easy_install work; but so would editing /usr/bin/easy_install.]
Has anyone copied their library? Is it that simple?
Or should I fix easy_install and simply step through the installation guide and build a new, clean, latest-and-greatest?
Edit
Or, should I
Skip trying to resolve the 2.5.1 and 2.5.4 issues and just jump straight to 2.6?
Normally, you would only have one version of a python release installed. Since 2.5.1 and 2.5.4 are from the same release, copying your libraries should work fine. What you would need to watch out for, is that you now have /usr/bin/python, and /usr/local/bin/python in your path, and some utilities may get confused.
If you need to have both micro-releases installed at once, I would keep 2.5.4 out of your path altogether, or allow it to completely clobber the other (do so at your own risk though ;)
If you go with the former, you can also point 2.5.4 to your site-packages by using the PYTHONPATH environment variable.
Ubuntu takes a different route, and this is how you can handle different major releases. The python binary is given with the version appended:
/usr/bin/python -> python2.6
/usr/bin/python2.5
/usr/bin/python2.6
Each has their own /usr/lib/python2.X directory with versions of all the modules.
And lastly, you can further customize your setup by modifying your site.py
I suggest you create a virtualenv (or several) for installing packages into.
I've had similar experiences and issues when installing Python 2.5 on an older release of ubuntu that supplied 2.4 out of the box.
I first tried to patch easy_install, but this led to problems with anything that wanted to use the os-supplied version of python. I was often fiddling with the tool chain to fix different errors that might crop up with every install. Installing any python software via apt, or installing any software from apt that had a python easy_install script as part of the install, was often amusing. I'm sure I could probably have been more vigilant in patching easy_install, but I gave up.
Instead, I copied the library, and everything worked. As you say, there may be issues depending on what you have installed, but I didn't run into issues. Double-checking Python's site.py module, I did see that it operates entirely on relative paths, building absolute paths dynamically; this gave me some confidence to try the "copy everything" approach. I double-checked any .pth files, then went for it.