Develop Reference

Should Which Pip and Which Python Return the Same Directory? Zeppelin Configuration On Unix RHEL

This is probably a really dumb question but I am stuck and wasting too much time on this so I would SO appreciate any help.
I am using a RHEL 7 box and installed Apache Zeppelin on it. Everything works except for the life of me I can't import Python packages such as Pandas.
I realized I didn't have PIP so I installed it with these steps: https://pip.pypa.io/en/stable/installing/ (notice I had to use the "--user" argument for the command "python get-pip.py").
Finally, I did "pip install pandas --user" which worked perfectly. I then go into my Zeppelin notebook and I cannot import pandas, even after restarting the Python interpreter.
I did some research and I think the problem is that "which python" and "which pip" are installed in different directories as the former results in "/usr/bin/python" while the latter in "~/.local/bin/pip".
So I suspect the packages installed with pip are basically getting loaded into a different version of python? If it helps, when I do "whereis python" I get 5 different results such as "/usr/bin/python" and "/usr/bin/python2.7" etc.

First thing to understand is: Python packages aren't installed globally, every installed Python has its own set of packages. BTW, pip being a Python package with a script is also not global. If you have a few different pythons you need different pips for them. I don't know Apache Zeppelin so I cannot guess if it uses the system Python (/usr/bin/python) or has its own Python; in the latter case you need to install pip specifically for Zeppelin so its pip install packages available for Zeppelin.
To investigate to what Python pip installs packages you need to find out under what python it runs. Start with shebang:
head -1 `which pip`
The command will prints something like ~/.local/bin/python. If it's not the version of Python you need to install packages for you need to install a different pip using that Python.
The most complex case would be if the shebang is PATH-dependent, something like #!/usr/bin/env python. In that case pip runs Python that you can find with which python.
PS. AFAIK the simplest way to install pip at RedHat is dnf install python-pip.

phd's answer was very helpful but I found that it was just a matter of using the root account to install the python packages. Then my Zeppelin was able to see any packages.

Pip upgrade does not remove the old pip and breaks

Edit: My question is not regarding how to immediately get pip to work. I fixed this by relinking as you can see in the question.
The question is: What is best/pythonic practice on a fresh *NIX system?
One would probably want to have the most recent version of both Python2 and especially Python3 with pip and setuptools and so on. A safeguarded virtual environment seems to be the way, but anyway one has to start somewhere.
So what is the recommendation?
From the comments it seems that one should use the get-pip.py script to install and then use pip to install virtualenv. I guess this has to be done both for Python2 and Python3 if one wants to have both.
This is what happened some days ago:
Ubuntu 18.04 and Python 2.7
I am a heavy R user, but use more and more Python. Virtually every time I set up any kind of customized python environment on Linux or OSX, something breaks and has to be fiddled with.... links, binaries, paths, dependencies. Each and every time. How many thousands of people are sitting on a Ubuntu/Debian box right now and do apt install python pip; pip install --upgrade pip and -duh- it breaks? Really?
Specifically right now: I want to install django on a webserver and started with apt install python-pip.
Then I did the recommended pip install --upgrade pip which installed pip-18.0 but gave the message Not uninstalling pip at /usr/lib/python2.7/dist-packages
After that pip --version threw an error
Traceback (most recent call last): File "/usr/bin/pip", line 9, in
from pip import main
It turned out that the old pip still resides in /usr/bin/pip while the new one is in /usr/local/bin/pip.
I fixed it brute force with:
rm /usr/bin/pip
ln -s /usr/local/bin/pip /usr/bin/pip
Did I do the right thing or is this the way to doom down the road?
Is there a pythonic or elegant way to handle such issues or to prevent them in the first place?
With best regards and thanks for any kind of suggestions.

My conclusion after investigating the hints in the comments is:
Python conflicts with itself.
Do not try to avoid this - cope with it.
Use virtual environments to handle the unavoidable multiple Python versions consistently.
pyenv seems to work best for my purposes. Use the following to install first pyenv (1), update it (2), install a Pythons version (for this example 3.5.6) with pyenv (3), set your user-global python to the installed version (4), upgrade the Python package mangement tools (5) and install anything inside the virtual Python enviroment (6).
curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
pyenv update
CONFIGURE_OPTS=--enable-shared pyenv install 3.5.6
pyenv global 3.5.6
pip install --upgrade pip setuptools wheel
pip install ipython jupyter snowflake-connector-python seaborn
CONFIGURE_OPTS=--enable-shared in (3) is needed for some python packages which depend on shared libraries. seaborn will not install without it.
Note that the pyenvinstalled versions includes the package manager pip. Now every call python or pip everything references to the correct version.
When referencing python in scripts in MacOS bash and other *NIXes use #!/usr/bin/env python.
It is possible to install pyenv install 2.7.15 the latest python 2.7 and change to it whenever necessary. pyenv shell 2.7.15 switches to Python2.7 only for a particular shell session when needed to run an adhoc script.
Since I use this flow I have no headaches anymore.
The same setup works superfine so far both on a local MacOS and on a remote Ububtu based jupyter notebook server.
Disclaimer: I am not a hardcore Python fullstack developer, but use Python whenever necessary in Data Science pipelines. So for application development in Python this might not be the best solution, but for consistent management of a Data Science environment it seems to work well.
Hope this is helpful.

Pip error: Fatal error in launcher: Unable to create process using '"'

I've seen many threads about this, and have tried all options except for completely wiping Python off of my machine and re-downloading everything...
I'm using a Windows 10, 64-bit machine, and had already downloaded Python2.7. Commands like 'C:\>pip install seaborn' were not an issue.
I recently downloaded Python3.6, and now my pip will not work - it returns the error in the title.
I have added C:\Python27, C:\Python36, C:\Python27\Scripts, C:\Python36\Scripts to my Path, and still it won't work.
If I type in the command C:\>python27 -m pip install seaborn, however, the pip works. I am really confused why I can no longer just type in pip install and have it work.
Thanks in advance!

You have two versions of Python added to path. To differentiate between 2.7 and 3.6 you have to tell it which version you want otherwise each pip conflicts with the other (it does not know what to install and where) in other words you type pip you could either mean for Python 2 or for Python 3.
Do not rename pip it will break your system (you should not need to rename). Instead use those already provided..
Your system should have these already:
pip is universal. Best for one installation.
pip3 for Python 3. Best to distinguish between Python 2 and 3
pip3.6 to distinguish between different Python 3 installations.
The same goes for Python 2 installation.
pip, pip2 and pip2.7.
You need to use either pip3 (or pip2) or pip3.6 (or pip2.7) to install in future. This will allow the different versions to be recognised:
For Python 2:
pip2 install seaborn
For Python 3:
pip3 install seaborn
You should also now use shebang lines as well now (if you are not already) to distinguish between versions.

the issue is the ambiguity between the two pip that you've mentioned in the Environments. As you mentioned the issue only started occurring when you installed python3 on the same system where python2 was installed and both have pip and hence when you fire up pip in your cmd, Windows System isn't able to pick one out of the two.
Why does your C:>python27 -m pip install seaborn work?
Well it's quite simple, since you've mentioned the python27 there, windows knows exactly which pip you're talking about.
How to fix it?
see the edits for this section. (I tried this, it didn't work) Removed it from the final answer to avoid confusion.
Alternatively, what you can do is,
rename your python.exe for python 3 to python3. Don't forget to put it inside your PATH environment. Just use python for python 2, python3 for python 3.
Their pip are separated, pip for python 2. pip3 for python 3.
Now, run and see the below commands behave:
# will return the default version of pip
pip --version
# will use the Python 2 version of pip
pip2 --version
# will use the Python 3 version of pip
pip3 --version

Okay so I finally worked it out...
I uninstalled Python3.6 and deleted all relevant folders.
I then went to Control Panel>Programs>Progams and Features and repaired my Python2.7 program. pip works now (I think it got messed up since I tried to rename the universal pip.exe file -> don't do that!!).
After re-downloading Python3.6, I put my universal pip.exe download from Python3 in a different directory so the Path would not get it confused. I now have Paths for both pip2 and pip3 and all is okay.
Thanks for your help!

This is how I solved this issue on my end: (short answer, remove this folder C:\Python27)
Problem: I installed python 3, after uninstalling python 2.7. The issue here is that pip remains behind even after you uninstall python 2.7.
Solution:
1. Uninstall python 3 (if you have it installed).
2. Just in case, I would uninstall python 2.7.
3. This is the key: go to C:\Python27 ... and delete the entire directory (which also contains pip).
This solution is good for those that are fine with ONLY running Python 3 on their machines (which was my case).

Which version of Pip to use with my Python installs?

Being new to Python, I'd love to clear up a few points that I couldn't get from reading various articles and tutorials.
After using Homebrew to install Python3, I noticed that it had installed both Python3 and Python3.4. I was also a little surprised that there are now three versions of pip on my machine too; pip, pip3 and pip3.4.
I created a new virtualenv and told it to use Python3, using the following command:
virtualenv -p /usr/local/bin/python3 mysite
I was also surprised that the version of Python that it installed in my VM was 3.4.
Is it safe to have these multiple version of Python and Pip hanging around on my machine?
Am I right to assume that I should take extra care to use the matching version of pip with Python, for example, pip3.4 with Python3.4?

Yes, it is safe. Python uses this naming like python3.4, python3.5 etc to differentiate between releases. python3 is a symbolic link to the current python3.x version. Pip follows the same convention.
If you're using python3.4 explicitly, you should be using pip3.4 specifically as well. Otherwise, just use python3 and pip3. For Python 2, you can simply use python (which, unless you installed the Homebrew version as well), will be the system Python), and ditto for pip. python2.7 and pip2.7 may also work.
In general, to find out which Python version goes with which pip you're using, try:
pip --version
and you'll see the Python included in the result.

No need to worried about if you have multiple version of Python and Pip installed. just check your version by writing in terminal :
$ brew info python
or to check the version of pip write in terminal :
$ brew info pip
and make sure you have updated your both pip and python version (write in terminal $ brew upgrade pip/python)
and other way to install python is go to https://www.python.org/downloads/ and choose as your requirement, there is two version available 2.7.9 & 3.4.3 ,
after installing python write in terminal $ python -V to check its version :) Hope it will help :)

installing modules in python - pip, distribute, nose, virtualenv

I'm aware that there are similar questions on SO. This one, for example: What's the proper way to install pip, virtualenv, and distribute for Python?
I'd like to install these modules as per my Learn Python the Hard Way tutorial: http://learnpythonthehardway.org/book/ex46.html
I managed (I think) to install pip by using sudo easy_install pip but when I then ran pydoc modules I could not see it. So I'm not even sure it's installed.
The answer above in question 4324558 is difficult for me to understand: what's a bootstrap, what's curl and why would I set up a virtual environment? Yes, as a learner I should try to pick up as much as I can but I don't want to first create the universe, I just want to get the task at hand done.
How do I install these modules? Is it as complicated as it sounds in the quoted answer? The top voted answer says "Install virtualenv into a bootstrap virtual environment. Use the that virtual environment to create more. Since virtualenv ships with pip and distribute, you get everything from one install."
I really don't get what all that means. Isn't there something about the "Zen" of python and a one true way to get things done? Or am I out of context here? What is "the right way" to install these modules?
I tried:
pip install virtualenv in the terminal and received the following output:
Wheel installs require setuptools >= 0.8 for dist-info support.
pip's wheel support requires setuptools >= 0.8 for dist-info support.
Storing debug log for failure in /Users/myname/.pip/pip.log
I'm using a Mac and python 2.7

To solve your issue,
Just install (or upgrade) the setuptools:
sudo easy_install -U setuptools
Then you can run again: pip install virtualenv

Try adding 'sudo' in your command as-
sudo pip install virtualenv
It worked for me.

Have a look at Python Development Environment on Mac OS X Mavericks 10.9.
I followed these steps as well when trying to get Python 2.7 and Python 3.3 installed on OS X. It doesn't tell you how to install nose and distribute, but you should have a working environment and you can pick up from there.
I did have a problem using virtualenv and pip with Python 3, the question and solutions is available here.