Develop Reference

Should Which Pip and Which Python Return the Same Directory? Zeppelin Configuration On Unix RHEL

This is probably a really dumb question but I am stuck and wasting too much time on this so I would SO appreciate any help.
I am using a RHEL 7 box and installed Apache Zeppelin on it. Everything works except for the life of me I can't import Python packages such as Pandas.
I realized I didn't have PIP so I installed it with these steps: https://pip.pypa.io/en/stable/installing/ (notice I had to use the "--user" argument for the command "python get-pip.py").
Finally, I did "pip install pandas --user" which worked perfectly. I then go into my Zeppelin notebook and I cannot import pandas, even after restarting the Python interpreter.
I did some research and I think the problem is that "which python" and "which pip" are installed in different directories as the former results in "/usr/bin/python" while the latter in "~/.local/bin/pip".
So I suspect the packages installed with pip are basically getting loaded into a different version of python? If it helps, when I do "whereis python" I get 5 different results such as "/usr/bin/python" and "/usr/bin/python2.7" etc.

First thing to understand is: Python packages aren't installed globally, every installed Python has its own set of packages. BTW, pip being a Python package with a script is also not global. If you have a few different pythons you need different pips for them. I don't know Apache Zeppelin so I cannot guess if it uses the system Python (/usr/bin/python) or has its own Python; in the latter case you need to install pip specifically for Zeppelin so its pip install packages available for Zeppelin.
To investigate to what Python pip installs packages you need to find out under what python it runs. Start with shebang:
head -1 `which pip`
The command will prints something like ~/.local/bin/python. If it's not the version of Python you need to install packages for you need to install a different pip using that Python.
The most complex case would be if the shebang is PATH-dependent, something like #!/usr/bin/env python. In that case pip runs Python that you can find with which python.
PS. AFAIK the simplest way to install pip at RedHat is dnf install python-pip.

phd's answer was very helpful but I found that it was just a matter of using the root account to install the python packages. Then my Zeppelin was able to see any packages.

Cannot uninstall 'numpy'

I've been trying to install astropy and at the end of the installation I get this message:
Cannot uninstall 'numpy'. It is a distutils installed project and thus
we cannot accurately determine which files belong to it which would
lead to only a partial uninstall.
I have tried: pip uninstall numpy and then I get the same message.
I have Python 2.7.10 in a macOS High Sierra version 13.10.5

This doesn't directly answer your question, but that's because you're asking the wrong question.
Astropy requires Python 3.5 or 3.6. Trying to get it working with Apple's pre-installed Python 2.7 is a waste of time. You might be able to get an old version working this way, but not by using the installation instructions on astropy.org, and it won't be supported even if you do.
The easy solution is to just Install the latest Anaconda 5.x with Python 3.6, because it comes with Astropy built in.
The almost-as-easy solution is to install Python 3.6 from either a python.org binary installer, or Homebrew, and then use pip3 or, better, python3 -m pip to install everything, as explained on the Astropy install page.
Either way, before doing anything else, you want to get back to a clean slate. In particular, you do not want pip, or any other scripts, attached to Apple's Python 2.7; they will only cause confusion. Assuming you can't reinstall macOS from scratch, the best way to do this is:
Look in /Library/Python/2.7/site-packages and delete everything there except for README and Extras.pth.
Look in /usr/local/bin for symlinks to anything in that site-packages. (If you don't know much about using Unix, try this command: ls -l /usr/local/bin | grep 2.7.) You'll probably have pip and pip2.7 here, and probably nothing else. But whatever you have here, delete it.
Now, when you install Python 3.6, the only thing named pip anywhere will be that Python 3.6's pip. You still want to use pip3 or python3 -m pip, but if you screw up and type pip by accident, it won't break anything.
Also, you should strongly consider using a virtual environment. See the Python Packaging Authority's User Guide (or the Anaconda docs, if you went that way) for more on this.

One simple solution I found:
sudo -H pip install astropy --ignore-installed numpy

I also had this issue and couldn't get around it in a clean way, but this is what I did:
Inside the Lib folder search "numpy" for an egg_info file (eg. numpy-1.11.0.dev0_2329eae.egg-info).
In my case, this is what Pip was looking at to determine if a current version already exists. I deleted it, then ran normal
pip install numpy
and installed the newest version.
I don't recommend this because I don't understand what it's doing under the hood and it doesn't properly uninstall the old version which could be a recipe for trouble down the line, but if you're desperate like I was then maybe this is a solution for you.

Does building pure python modules w/ conda require setuptools?

This weekend I've been reading up on conda and the python packaging user guide because I have a simple pure python project that depends on numpy. It seemed to me that distributing/installing this project via conda was better than pip due to this dependency.
One thing on which I'm still not clear: conda will install a python package from a recipe in build.sh, but it seems like build.sh just ends up calling python setup.py install for most python packages.
So even if I want to distribute/install my python package with conda, I still end up depending on setuptools (or distutils) for the actual installation, correct? I was unable to find a conda utility analogous to setuptools; am I missing something?
FWIW, I posted this question on the conda issue tracker.
Thanks!

Typically you will still be using distutils (or setuptools if the library requires it) to install things, yes. It is not technically required. The build.sh can be anything. If you wanted to, you could just copy the code into site-packages. Using setup.py install is recommended, though, as libraries will already have setup.py working, it will install metadata that can be read by pip, and it will compile any extension modules and install any data files.

Installing Python modules with Anaconda or Canopy

I have played around with Python a little but never have I had to install my own packages. I am currently trying to write a program that reads in 'tiff' files so I'm trying to install the 'libtiff' package and I'm having a nightmare!
First, I was using the Anaconda distribution and the phrase 'conda install libtiff' which would tell me the install was successful. However then I was never able to find libtiff in the Spyder IDE or ipython console.
Having used Canopy in the past, I uninstalled Anaconda and gave Canopy another shot using 'pip install libtiff', however I receive an error saying it failed with error code 1 and I don't know what this is.
I must be missing something fairly crucial but installing this module is proving almost impossible!

Conda's libtiff package is the C library, not the Python library. The easiest way to get it would be to use conda to install the dependencies (like libtiff, numpy), and then use pip to install libtiff (pip install libtiff).

Using ! on the IPython console within spyder allows you to use pip. So, in the example, you could do:
In [1]: !pip install libtiff
Note, this is also available (though perhaps unreliably) on the Python console for Spyder versions before ~2.3.3.

Python - pip install pandas, not working

I am trying to install pandas via pip install pandas but when I do, I get the error:
Command python setup.py egg_info failed with error code 1 in
c:\users[username]\appdata\local\temp\pip_build_[username]\pandas
I followed the answer given here and installed ez_setup.py without incident, but still get the error when doing pip install pandas.
Thanks for any help, if I can provide more information please let me know.

The easiest way to install pandas and its dependencies on Windows is to download the relevant packages from Christoph Gohlke's Python Extension Packages for Windows repository. You'll find the files for pandas here as well as a list of other required dependencies.

On Linux (Debian / Ubuntu varieties), when NOT installing inside a virtual environment, but in the main system, I find it best to just use the Synaptic Package Manager (because even the --user switch seems to fail when trying to install pandas without sudo). Search for pandas inside Synaptic PM. There's varieties for python 2 and 3.
However, on linux, I have generally found the cleanest, easiest, and overall safest approach to be creating virtual environments and then use pip install <package name> inside the virtual environment. I believe this would be best on Windows too.

I installed Pandas package following procedure listed after the following disclaimer section:
Disclaimer:
- I don't consider myself a computer expert so follow the instructions at your own risk.
- My procedure worked on my windows computer
- My windows computer has python 2.7 installed from python.org
- My python GUI is IDLE
- I don't recall installing pip, it is possible that it installs at the time of installing python 2.7 from python.org (not sure)
- The directory in which my pip.exe is located is under C:\Python27\Scripts
Procedure:
Open a command window for the directory under which you have pip.exe, (in my case is Scripts). The prompt looks like this in my case: C:\Python27\Scripts>
At the prompt type pip install pandas. The prompt looks like this in my case: C:\Python27\Scripts>pip install pandas
Press ENTER key. You should see message: "Collecting pandas" being displayed in the command window.
Once the system completes collecting pandas, you should see message "Successfully installed pandas-0.22.0" , or similar depending on version collected.
Picture shows steps 1 -4 as shown in my computer command window.collecting_pandas