Develop Reference

Pip upgrade does not remove the old pip and breaks

Edit: My question is not regarding how to immediately get pip to work. I fixed this by relinking as you can see in the question.
The question is: What is best/pythonic practice on a fresh *NIX system?
One would probably want to have the most recent version of both Python2 and especially Python3 with pip and setuptools and so on. A safeguarded virtual environment seems to be the way, but anyway one has to start somewhere.
So what is the recommendation?
From the comments it seems that one should use the get-pip.py script to install and then use pip to install virtualenv. I guess this has to be done both for Python2 and Python3 if one wants to have both.
This is what happened some days ago:
Ubuntu 18.04 and Python 2.7
I am a heavy R user, but use more and more Python. Virtually every time I set up any kind of customized python environment on Linux or OSX, something breaks and has to be fiddled with.... links, binaries, paths, dependencies. Each and every time. How many thousands of people are sitting on a Ubuntu/Debian box right now and do apt install python pip; pip install --upgrade pip and -duh- it breaks? Really?
Specifically right now: I want to install django on a webserver and started with apt install python-pip.
Then I did the recommended pip install --upgrade pip which installed pip-18.0 but gave the message Not uninstalling pip at /usr/lib/python2.7/dist-packages
After that pip --version threw an error
Traceback (most recent call last): File "/usr/bin/pip", line 9, in
from pip import main
It turned out that the old pip still resides in /usr/bin/pip while the new one is in /usr/local/bin/pip.
I fixed it brute force with:
rm /usr/bin/pip
ln -s /usr/local/bin/pip /usr/bin/pip
Did I do the right thing or is this the way to doom down the road?
Is there a pythonic or elegant way to handle such issues or to prevent them in the first place?
With best regards and thanks for any kind of suggestions.

My conclusion after investigating the hints in the comments is:
Python conflicts with itself.
Do not try to avoid this - cope with it.
Use virtual environments to handle the unavoidable multiple Python versions consistently.
pyenv seems to work best for my purposes. Use the following to install first pyenv (1), update it (2), install a Pythons version (for this example 3.5.6) with pyenv (3), set your user-global python to the installed version (4), upgrade the Python package mangement tools (5) and install anything inside the virtual Python enviroment (6).
curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
pyenv update
CONFIGURE_OPTS=--enable-shared pyenv install 3.5.6
pyenv global 3.5.6
pip install --upgrade pip setuptools wheel
pip install ipython jupyter snowflake-connector-python seaborn
CONFIGURE_OPTS=--enable-shared in (3) is needed for some python packages which depend on shared libraries. seaborn will not install without it.
Note that the pyenvinstalled versions includes the package manager pip. Now every call python or pip everything references to the correct version.
When referencing python in scripts in MacOS bash and other *NIXes use #!/usr/bin/env python.
It is possible to install pyenv install 2.7.15 the latest python 2.7 and change to it whenever necessary. pyenv shell 2.7.15 switches to Python2.7 only for a particular shell session when needed to run an adhoc script.
Since I use this flow I have no headaches anymore.
The same setup works superfine so far both on a local MacOS and on a remote Ububtu based jupyter notebook server.
Disclaimer: I am not a hardcore Python fullstack developer, but use Python whenever necessary in Data Science pipelines. So for application development in Python this might not be the best solution, but for consistent management of a Data Science environment it seems to work well.
Hope this is helpful.

How to install Numpy & pip3 for python3.x when they were installed in python2.7? Using Conda?

I want to write program in python3 (3.5), hence I installed python3 next to the pre-installed python2 (2.7) on Mac OS X El Captian.
Since my terminal runs python2.7 by default and Numpy is already installed for it, I put alias python=python3 and expected to be able to install Numpy for python3. when I type pip install numpy. This was the generated message:
Requirement already satisfied (use --upgrade to upgrade): numpy in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
I also noticed that I have no pip3 even though I am using python3: python --version returned Python 3.5.2, but pip3 install numpy got me -bash: pip3: command not found.
So my questions are:
1) How to install Numpy for python3.x when Numpy is installed on python2.x?
2) How to get pip3?
3) Is it better to use virtual environments, such as Conda, instead of juggling between python2 and python3 on the system?
Thank you from a total n00b
------------------- Update -------------------
Reinstalling python3 also fixed another problem in my case.
When I ran brew doctor, one of the warning message I got was:
Warning: You have unlinked kegs in your Cellar Leaving kegs unlinked can lead to build-trouble and cause brews that depend on those kegs to fail to run properly once built. Run brew link on these: python –
This is a result of me running brew unlink python in order to fix
"Python quit unexpectedly"
when I launch Vim and also
"The ycmd server SHUT DOWN"
Both seem to relate to the YouCompleteMe autocomplete plugin which I downloaded for Python.
I got my idea of removing symlinks from here and here
However, Homebrew evidently did not like the absence of those 39 symlinks.
After uninstall (brew uninstall python3) and then re-install python3 (brew install python3) as Toby suggested, Homebrew gave me
You can install Python packages with
pip3 install <package>
Then when I pip3 install numpy and pip3 install scipy, both executed successfully.
To my surprise, symlinks created during Python installation used to cause aforementioned error messages for Python and YouCompleteMe, but now I open python files using Vim without crash from a fresh Python installation, which definitely created the symlinks.
------------------- Update2 ------------------
After re-installing Anaconda2, the same YouCompleteMe error came back. I suspect Anaconda messed up symlinks.

I would recommend using the Anaconda Python distribution.
The main reasons are as such:
You will have a Python distribution that comes with numpy and the rest of the Scientific Python stack.
The Anaconda Python will be installed under your home directory, with no need for sudo-ing to install other packages.
conda install [put_packagename_here] works alongside pip install [put_packagename_here]; conda install is much 'cleaner' (IMHO, differing opinions are welcome).
If you have a Python 3 environment as your default, then pip works out-of-the-box without needing to remember to do pip3.
conda environments are easier to manage than virtualenv environments, in my opinion. And yes, you can have Python 2 alongside Python 3.
I once messed up my system Python environment - the one that came with my Mac - and it broke iPhoto (back in the day). Since then, I became convinced of needing separate, atomic environments for different projects.
I've detailed more reasons in a personal blog post.
Other distributions, of course, are all good, provided they give you what you need :).

The simplest way on a Mac is with Homebrew:
http://brew.sh/
Install Homebrew, then run:
brew install python3 pip3
Edit --
Python3 includes pip3, but Homebrew occasionally has trouble linking to the correct versions, depending on what has been installed. Running the following command:
brew doctor
And if you see errors relating to python or unlinked kegs, try running:
brew uninstall python python3
And reinstalling after checking brew doctor.
https://unix.stackexchange.com/questions/233519/pip3-linked-to-python-framework-instead-of-homebrew-usr-local-bin

mac following brew install python warning thrown unstable state

Pundits warn against installing python in a mac usr/bin/Frameworks area.
Python self-installers write to Framework by default.
pundits advise using brew install of python to avoid the above.
Brew install python however, results in unstable state
Idle reports tclsh mismatch.
Pundits advise active state installer of correct tclsh. These are high-level python cognoscenti, and real pundits, lilies amidst the thorns.
Active-state installs to Frameworks (can you imagine?).
The said installer allows no other installation directory.
Brew installed python fails to see the active-state tclsh.
However, if one of you admonitory pundits could help me with a logical, non-idiomatic description of a process that will associate the appropriate "tclsh" in usr/bin with python3 in usr/local/bin, I would be ecstatic.

Homebrew's Python build will only attempt to recognize brewed or system Tcl/Tk. To build against Homebrew's Tcl/Tk (and install it first if necessary), install Python with brew install python3 --with-tcl-tk.

Which version of Pip to use with my Python installs?

Being new to Python, I'd love to clear up a few points that I couldn't get from reading various articles and tutorials.
After using Homebrew to install Python3, I noticed that it had installed both Python3 and Python3.4. I was also a little surprised that there are now three versions of pip on my machine too; pip, pip3 and pip3.4.
I created a new virtualenv and told it to use Python3, using the following command:
virtualenv -p /usr/local/bin/python3 mysite
I was also surprised that the version of Python that it installed in my VM was 3.4.
Is it safe to have these multiple version of Python and Pip hanging around on my machine?
Am I right to assume that I should take extra care to use the matching version of pip with Python, for example, pip3.4 with Python3.4?

Yes, it is safe. Python uses this naming like python3.4, python3.5 etc to differentiate between releases. python3 is a symbolic link to the current python3.x version. Pip follows the same convention.
If you're using python3.4 explicitly, you should be using pip3.4 specifically as well. Otherwise, just use python3 and pip3. For Python 2, you can simply use python (which, unless you installed the Homebrew version as well), will be the system Python), and ditto for pip. python2.7 and pip2.7 may also work.
In general, to find out which Python version goes with which pip you're using, try:
pip --version
and you'll see the Python included in the result.

No need to worried about if you have multiple version of Python and Pip installed. just check your version by writing in terminal :
$ brew info python
or to check the version of pip write in terminal :
$ brew info pip
and make sure you have updated your both pip and python version (write in terminal $ brew upgrade pip/python)
and other way to install python is go to https://www.python.org/downloads/ and choose as your requirement, there is two version available 2.7.9 & 3.4.3 ,
after installing python write in terminal $ python -V to check its version :) Hope it will help :)

Python 2.6 in mac corrupted because of failled update

I'm a mac newbie and I tried to update my python version from 2.6 to 2.7. Unsuccessful, I changed my mind and uninstalled the python2.7 I had. I had a previous issue that if I typed something like:
python setup.py install
It would not install the package for python2.6, installing to the removed 2.7 version instead, to make it work I have to put
python2.6 setup.py install
And now when I try to install something with easy_install or pip (by the way, pip I have installed after 2.7 issue) I got the following huge message errors: here and here. I want to know how can I clean up my mess.

Since you were trying to install MySQLdb, how about you give ActivePython a try?
Install ActivePython 2.7 (it co-exists with Apple's System Python 2.6)
Open Terminal and type pypm install mysql-python (see PyPM Index) .. no compilation required
Make sure that /usr/local/bin/ is in front of your $PATH.
To uninstall ActivePython, you can do:
sudo /Library/Frameworks/Python.framework/Versions/2.7/Resources/Scripts/uninstall
Or, use sudo pythonselect 2.6 to switch the default Python in /usr/local/bin (if you have multiple versions of non-System Pythons installed)