I am not a total newbie but I am trying to install modules for quite a long time and at this point i would like to have a fresh start and install python and all the modules I need so i really understand them. My problem is that some of them import, but most of them install either to the wrong site-packages or dont import maybe because i messed up my system/python. Also I tried the PYTHONPATH and PATH to set this up right, but it never worked.
So my questions are:
Is there a way to ensure I can clean everything up and start from zero ?
Ideally this would be without having to set up Mac OSX new.
Is there a way to install all the modules in the correct place (whatever the directory is I dont care, it should just work)?
Is there a good step-by-step description on how installing modules works. And I dont mean just the info to use easy_install, pip install etc, but a way to fully understand what I need to consider, where I need to put them, why these modules are recognized in certain directories, how the system finds them and most important what are the most common pitfalls and how to avoid them.
I also tried Macports and various other similiar ways to install but even if some of them worked and while I am sure that these are really great tools, most I had to hack to work.
So if someone can recommend a good and stable way to install a lot of modules at once, this would be incredibly useful.
Thanks a lot !!!!
And sorry for the long questions.
Buildout and virtualenv should be what you are looking for.
Buildout helps you configure a python installation and virtualenv allows you to isolate multiple different configurations from each other.
Here's a nice blog post explaining how to use them together.
Also, see this other question: Buildout and Virtualenv
You can safely install an up-to-date Python 2 and/or Python 3 on OS X using the python.org installers here. They will coexist with any other Pythons you have installed or that were shipped by Apple with OS X. To install packages, for each Python instance first install Distribute which will install a version-specific easy_install command, i.e. easy_install-2.7 or easy_install-3.2. Many people prefer to use pip to manage packages; you can use easy_install to install a version-specific copy of it. If you want something fancier, you could also install virtualenv but, with the isolation provided by Python framework builds on OS X, that isn't as necessary as on most other platforms.
Is there a way to install all the modules in the correct place?
Download and untar/gunzip/etc the module source (Most of the modules ares available in gzip form at http://pypi.python.org/pypi), then run configure with --prefix set to the same thing for every install:
[ 11:06 jon#hozbox.com ~ ]$ ./configure --prefix=/usr/local
/usr/local is usually the default, but it doesn't hurt to specify it and will ensure that every module you install will be placed in /usr/local/lib/python/...
Is there a good step-by-step description on how installing modules works?
The Python website has a great page called: Installing Python Modules
http://pypi.python.org/pypi
http://docs.python.org/install/index.html
Related
I'm installing a bunch of python modules on my system that are specific to this code I am going to be working with. Specifically, I used pip install pyda to get the pyda module. To make sure I had gotten all the modules, I went through and ran some of the code snippets, and came across the following error:
ModuleNotFoundError: No module named 'pyda.utilities'
I tried using pip install pyda.utilities, but that honestly doesn't make sense, it should have just come with the pyda module. According to this website https://pypi.org/project/pyda/ it seems like it should come with the package. I tried determining if I just installed it in the wrong python version, but I'm having a difficult time forcing it to use the specific python version that I installed the module in (specifically, I tried to create an alias for /usr/bin/python3.7 or something like this as I have seen on other websites, but it just fusses at me that this is simply a directory, incredibly unhelpful because I can't find the corresponding executable, so I'm a bit confused here).
This is a very long question likely with a very simple answer, any thoughts or help on what the issue might be would be appreciated.
Edit: I have determined that it's a package problem, not a python problem. The command 'pip install pyda' is not actually installing everything, oddly enough, which is why it cannot find the pyda.utilities module. Unfortunately, I think this means I will have to install the package manually. I will keep this question posted because of the useful answer on virtual environments, so thanks everyone.
The answer is indeed straightforward. As #Chris indicated in the comments, start using virtual environments.
It's not as complicated as it sounds and there's plenty of tutorials on getting started with virtualenv for Python, like https://uoa-eresearch.github.io/eresearch-cookbook/recipe/2014/11/26/python-virtual-env/
The basic steps:
check you're using the version of Python you want in your virtual environment
if you don't, change directories to where it lives
ensure you have pip and it works
check if you have virtualenv and if you don't pip install virtualenv
create a virtual environment virtualenv /your/env/folder/here
activate the virtual environment with /your/env/folder/here/Scripts/activate
After that, just install the packages you need with pip and they will end up in your virtual environment, with no interference from other Python versions or packages.
Check your python version, if it does not work restart your computer and try run setup.py install on the python command line
I'm a front-end developer with zero server-side experience. I'm trying to learn a little python for an app I'm building, and I literally cannot find anywhere that makes it easy to install easy_install. Everything I read is full of lingo I don't get, like using homebrew or PATHs. And pypi.python.org is TERRIBLY written and designed. So far it's only sent me in circles.
I'm looking for a simple set of instructions in the following format:
Go to x website and click x link
Type x into the terminal and run
Type x to be sure it's installed properly
Please for the benefit of myself and the whole internet, can someone provide this? I'm definitely up for learning how to do some things the hard way, but Python isn't something I need to know very thoroughly right now, and I imagine that's the case for others just looking to get their feet wet.
Assume I have Python installed, but not that I know anything else about using it.
(If there really is a resource out there like this that exists, let me know... I just haven't seen anything like it yet and I've been googling around for about an hour.)
Thank you!
easy_install is part of setuptools. For a while, distribute was a separate fork of setuptools that included its own easy_install, and you had to put a bit of thought into which one you wanted, but that's no longer an issue; the two have merged back into one project. So, the question is, how do you get setuptools?
If you're using Apple's pre-installed versions of Python, you already have setuptools, with a working easy_install, for all versions. You can also use easy_install-2.5, etc., for the earlier versions of Python that Apple also pre-installs. So there's nothing to do.
If you're using a Homebrew Python, it automatically installs setuptools by default, so again, nothing to do.
If you're using a third-party package manager that doesn't automatically install setuptools (as at least was true for MacPorts last time I checked), it probably has its own package for python32-setuptools or python32-easy-install or something like that.
Most of the "extra-batteries" distributions (ActiveState, Enthought, etc.) of Python similarly include setuptools, so again, nothing to do.
If you're using a Python.org binary, a custom-built Python with default configuration, etc., the standard setuptools instructions for "Unix-based Systems including Mac OS X" work fine.
If you installed Python somewhere you don't have write access, you will need to sudo the installation step, as in the second variant.
So:
wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py -O - | sudo python
And that's all there is to it.
All that being said, you almost never want to use easy_install, except to install two things:
readline, to replace the fake libedit-based support with real libreadline-based support. (And any other packages that don't work with pip, but this is the only really popular one.)
pip, so you never need to use easy_install again.
Most of the cases above that include setuptools also include pip; the one exception is Apple pre-installed Python, where sudo easy_install pip is the simplest way to get pip.
For the cases that don't come with either, the standard documentation for installing pip also works:
Install or upgrade setuptools (as above) if you have a version older than 0.7.
Download get-pip.py.
python get-pip.py
Again, sudo if you need to, and replace the python with python3.3 or python2.6 or /opt/local/mypython/bin/python or whatever to install for any non-default versions.
Finally, for upgrading, once you have any version of pip, and setuptools 0.7 or later, it's as easy as you could imagine:
pip install --upgrade setuptools pip
Again, sudo if necessary, and pip-3.3 or whatever if necessary.
Note that to make easy_install or pip actually useful, you may need a compiler toolchain. The easy way to do this is to install Xcode from the Mac App Store (free as of late 2013), launch Xcode, open Preferences, go to Downloads, and install the "Command Line Tools". This is sufficient for almost everything you'd ever want to do.
The one major exception is SciPy and related packages, for which you will also need a Fortran compiler. See this blog post for Python 2.x and this one for 3.x for details. If all of that looks scary to you, you may be happier using Enthought or one of the other pre-packaged scientific distributions.
One last thing: If you installed a non-Apple Python, and then upgraded your OS X (or Homebrew or MacPorts or Enthought or …) to a newer major version (e.g., from OS X 10.8 to 10.9), you may need some extra steps. For every case but OS X itself, they should be documented on the home page for whichever package manager or Python distribution you're using. For an OS X upgrade, reinstalling your third-party Python is usually the simplest solution if you run into a problem (after first verifying that you actually need a third-party Python in the first place), even if it means you have to reinstall your site packages.
The short correct answer is don't use EasyInstall.
Use pip!
I want to use the default (no site packages) of virtualenv.
But some modules are difficult to install in a virtualenv (for example gtk). With "difficult" I mean that you need to have a lot of c-header files installed and a lot of stuff needs to be compiled.
I know that I can solve this by not installing these packages with pip, but to create symlinks to make some modules available from the global site-packages directory.
But is this the right direction?
Is there a way to create the symlinks with pip or virtualenv?
Update
In 2013 I wanted some modules like psycopg2, gtk, python-ldap and other which are installed on my linux server via rpm/dpkg in the virtualenv.
The symlinking or other work-arounds did make things more complicated, not simpler. We use this option today (2017)
--system-site-packages
Give the virtual environment access to the global
site-packages.
I'd say yeah, that's the right direction.
Your questions sounds similar to something I dealt with: installing OpenCV into virtualenv. My problem was that OpenCV wasn't available via pip (Python Package Index). What I ended up doing was querying the system-wide global Python installation for the module in question, and then copy-ing the .so into my virtualenv.
The whole process, including the boilerplate Makefile I used, are captured here: https://stackoverflow.com/a/19213369/1510289
You could do something similar by sym-linking instead of copying. The reason I ended up copying the library was because I use Make, and Make doesn't handle dependencies for symbolic links in a way I needed (as explained in the URL above.)
Hope this helps...
How are you compiling each of these 'hard' packages from scratch?
Are you doing something like:
python setup.py install
If so, replace that with:
python setup.py bdist_wheel
Then look in the ./dist directory for a .whl file. Then take whatever that file is, and do (after activating the environment)
pip install `./dist/whateverTheFileIsCalled.whl`
I am looking for the best option to keep my Python upgraded and to install new packages. I know there was previous discussion like this: Why use pip over easy_install? but they often quite old.
The problem is: I have ubuntu 12.04 at home and Windows 7 at work, would like to keep both of them on the same (newest) version level.
In ubuntu I found that repository contains often old version so I use different procedures to install new packages. I also found that pip installs new version, but did not check other options. In windows I try to install from provided .exe files which sometimes is difficult, and to have ipython working I had to install pythontools. Over the internet I see a lot of opinion about the subject, and advocates to different tools.
Is there a way to keep to different computers (ubuntu/windows) upgraded on the same level and what is the best choice.
I am looking for a constructive opinions like: if you use ... then you don't have to add paths manually but you need to clean downloaded files, or this ... will work perfectly on linux but for the reason ... not on windows.
I would like to use it in ubuntu 12.04 and Windows 7
I'm not sure about Windows. I think people like pip because it does dependencies better than easy_install. But I don't know enough about that to say more. Using virtualenv is a great choice in that it keeps your projects versions. You can also to pip freeze > requirements.txt to get a list of packages you installed within virtual env. This file can then be used to download those versions in another virtualenv. virtualenv installs its own local pip
If someone has problem with installation basic packages under windows this http://viziblr.com/news/2012/4/21/step-by-step-installing-pandas-on-windows-7-from-pypi-with-e.html gives good help. It works well in windows, although does it does not resolve the issue.
A colleague of mine wants to use my python 2 code on his OS X (10.6) machine. My code imports several built-in python packages, including Tkinter and shelve, and also uses third-party packages, including numpy, scipy, matplotlib, and ipython.
I've encountered a few problems with OS X's built-in python. (IDLE doesn't work, for example*). I suspect I should install a more recent version of python, and a different version of Tk.
My questions:
Will having two different versions of python/Tk on the same machine cause problems?
I would like to associate the terminal commands 'python', 'ipython', and 'easy_install' with the more recent version of python. How should I do this?
When I install third-party packages like numpy using a .dmg file, how do I control which version of python numpy installs into?
Is there a better way to do this?
If this process goes well, I'd consider adding OS X instructions to my code's documentation, so I'd like to boil down this process to the simplest, most general approach.
*EDIT: Also, this
EDIT: Thank you everyone for the useful answers. My colleague tried MacPorts, which seems to work well, but has a few speedbumps. First we had to install Xcode from the system install disk. This is not a fast or lightweight install (several GB). Luckily we still had the disk! Once Xcode was installed, MacPorts was easy to install. Python and the python subpackages we needed were also easy to install, but he told me this installation took several hours. Presumably this delay is due to compilation? He had an easy time setting the MacPorts python as default. However, I think we have to change the 'Python Launcher' application by hand, this seems to still default to the system python.
Even though he has a working system now, I'm tempted to ask him to try one of the other solutions. I'm not sure all of my code's potential users will tolerate a multi-hour, multi-gigabyte installation.
I use brew to install all my libraries/compilers/interpreters.
To install python try this:
brew install python
Then add Python's binaries directory to your $PATH in your ~/.profile:
export PATH=`brew --prefix python`/bin:$PATH
I'd recommend you to install pip, virtualenv and virtualenvwrapper to have better control over your environment too.
Have you tried ActivePython?
It includes a package manager (PyPM) that, by default, installs into your home directory (eg: ~/Library/Python/2.7). Main scripts get symlinked in /usr/local/bin; use the included pythonselect to set the active Python version.
You don't have to bother installing .dmg packages, as PyPM is a binary package manager ... therefore you can install non-pure Python packages like NumPy without having to compile things yourself.
ActivePython can use Apple's Tcl/Tk or, if installed, ActiveTcl.
A "simplest, most general approach" in your documentation could be:
Install ActivePython 2.7
Open Terminal and type pypm-2.7 install matplotlib ipython
Using MacPorts, you can install python 2.6, 2.7, 3.1 and 3.2 at the same time, with their own packages, without ever touching the built-in python.
numpy, scipy, matplotlib, and ipython are also available as ports for most of those python versions.
Moreover, if you install the python_select port, you'll be able:
to choose which one of those (plus the built-in python) is the "default" python;
to install python packages through easy_install/pip for the "selected" python, if they're not available as ports.
Add virtualenv to the mix, and you'll have a very, very flexible Python development environment.
As for your questions:
Q1: with MacPorts, no. while not a frequent user, I've installed and used matplotlib in 2.6 and 2.7, switching between the two using python_select.
Q2: easy_install, pip, ipython will be "linked" to the python they were installed by. (but see tip 1)
Q3: it's easier to install one of the py{26,27,xx}-numpy ports, or pip install numpy under your python_select'ed python.
Q4: well, MacPorts is the best thing I know after APT on Debian/Ubuntu... :-)
Now, two tips if you try MacPorts:
MacPorts cleanly installs ports separately from the OS X installation, in an /opt/local directory, and each python version is installed in a /opt/local/Library/Frameworks/Python.framework/Versions/{2.5,2.6,2.7,...} directory. Using python_select cleanly switch the "python" command using links. BUT... the Versions/{2.5,2.6,2.7,...}/bin directory, where python scripts are installed, is not added to the PATH. Just adding: export PATH=/opt/local/Library/Frameworks/Python.framework/Versions/Current/bin:$PATH to your ~/.profile will always give you direct access to the scripts installed for the selected python.
to avoid bad surprises, I've added a echo Selected python is \"$(python_select -s)\" line to my ~/.profile, so I always know which is my currently selected python when opening a session... :-)
Regards,
Georges
In almost all cases, the best python to use is the one from http://python.org/. It sets up the paths correctly and doesn't overwrite anything. DMG package installs usually work automatically, as does python setup.py install, and it's not too hard to get setuptools to work. If you want per-user installs, it is easy to set up .pydistutils.cfg and python automatically recognizes the path install_lib = ~/Library/Python/$py_version_short/site-packages
An addendum regarding the usage of brew:
Since some time, brew install python will install python3.
If you intend to install python2, you want to use
brew install python#2
It is perfectly fine to install both python and python3 using brew!
Here is an old post that answers your questions too.
In general it is not a problem at all to have more than one python installation on your machine. You just have to watch out which one you are calling on the command line.
>> which python
... helps to identify where your python binary is located. The original Mac OS X python is usually at "/usr/bin/python"
I personally use the MacPorts python installation. It also supports you with the installation of modules. (see link above)
I have 4 versions of python on my MacBook Pro. 2 from the original install of OS X 10.6 and a subsequent update, then self installed copies of python 2.7 and 3.2. You can update the python command to point at any of the versions. They all install in separate directories and cause no problems with each other.
I'm not sure what will happen when you install from a .dmg file. I believe it will simply use whatever version python points to.
This post on superuser.com answers your questions on changing default paths.