Edit: My question is not regarding how to immediately get pip to work. I fixed this by relinking as you can see in the question.
The question is: What is best/pythonic practice on a fresh *NIX system?
One would probably want to have the most recent version of both Python2 and especially Python3 with pip and setuptools and so on. A safeguarded virtual environment seems to be the way, but anyway one has to start somewhere.
So what is the recommendation?
From the comments it seems that one should use the get-pip.py script to install and then use pip to install virtualenv. I guess this has to be done both for Python2 and Python3 if one wants to have both.
This is what happened some days ago:
Ubuntu 18.04 and Python 2.7
I am a heavy R user, but use more and more Python. Virtually every time I set up any kind of customized python environment on Linux or OSX, something breaks and has to be fiddled with.... links, binaries, paths, dependencies. Each and every time. How many thousands of people are sitting on a Ubuntu/Debian box right now and do apt install python pip; pip install --upgrade pip and -duh- it breaks? Really?
Specifically right now: I want to install django on a webserver and started with apt install python-pip.
Then I did the recommended pip install --upgrade pip which installed pip-18.0 but gave the message Not uninstalling pip at /usr/lib/python2.7/dist-packages
After that pip --version threw an error
Traceback (most recent call last): File "/usr/bin/pip", line 9, in
from pip import main
It turned out that the old pip still resides in /usr/bin/pip while the new one is in /usr/local/bin/pip.
I fixed it brute force with:
rm /usr/bin/pip
ln -s /usr/local/bin/pip /usr/bin/pip
Did I do the right thing or is this the way to doom down the road?
Is there a pythonic or elegant way to handle such issues or to prevent them in the first place?
With best regards and thanks for any kind of suggestions.
My conclusion after investigating the hints in the comments is:
Python conflicts with itself.
Do not try to avoid this - cope with it.
Use virtual environments to handle the unavoidable multiple Python versions consistently.
pyenv seems to work best for my purposes. Use the following to install first pyenv (1), update it (2), install a Pythons version (for this example 3.5.6) with pyenv (3), set your user-global python to the installed version (4), upgrade the Python package mangement tools (5) and install anything inside the virtual Python enviroment (6).
curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
pyenv update
CONFIGURE_OPTS=--enable-shared pyenv install 3.5.6
pyenv global 3.5.6
pip install --upgrade pip setuptools wheel
pip install ipython jupyter snowflake-connector-python seaborn
CONFIGURE_OPTS=--enable-shared in (3) is needed for some python packages which depend on shared libraries. seaborn will not install without it.
Note that the pyenvinstalled versions includes the package manager pip. Now every call python or pip everything references to the correct version.
When referencing python in scripts in MacOS bash and other *NIXes use #!/usr/bin/env python.
It is possible to install pyenv install 2.7.15 the latest python 2.7 and change to it whenever necessary. pyenv shell 2.7.15 switches to Python2.7 only for a particular shell session when needed to run an adhoc script.
Since I use this flow I have no headaches anymore.
The same setup works superfine so far both on a local MacOS and on a remote Ububtu based jupyter notebook server.
Disclaimer: I am not a hardcore Python fullstack developer, but use Python whenever necessary in Data Science pipelines. So for application development in Python this might not be the best solution, but for consistent management of a Data Science environment it seems to work well.
Hope this is helpful.
Related
This is probably a really dumb question but I am stuck and wasting too much time on this so I would SO appreciate any help.
I am using a RHEL 7 box and installed Apache Zeppelin on it. Everything works except for the life of me I can't import Python packages such as Pandas.
I realized I didn't have PIP so I installed it with these steps: https://pip.pypa.io/en/stable/installing/ (notice I had to use the "--user" argument for the command "python get-pip.py").
Finally, I did "pip install pandas --user" which worked perfectly. I then go into my Zeppelin notebook and I cannot import pandas, even after restarting the Python interpreter.
I did some research and I think the problem is that "which python" and "which pip" are installed in different directories as the former results in "/usr/bin/python" while the latter in "~/.local/bin/pip".
So I suspect the packages installed with pip are basically getting loaded into a different version of python? If it helps, when I do "whereis python" I get 5 different results such as "/usr/bin/python" and "/usr/bin/python2.7" etc.
First thing to understand is: Python packages aren't installed globally, every installed Python has its own set of packages. BTW, pip being a Python package with a script is also not global. If you have a few different pythons you need different pips for them. I don't know Apache Zeppelin so I cannot guess if it uses the system Python (/usr/bin/python) or has its own Python; in the latter case you need to install pip specifically for Zeppelin so its pip install packages available for Zeppelin.
To investigate to what Python pip installs packages you need to find out under what python it runs. Start with shebang:
head -1 `which pip`
The command will prints something like ~/.local/bin/python. If it's not the version of Python you need to install packages for you need to install a different pip using that Python.
The most complex case would be if the shebang is PATH-dependent, something like #!/usr/bin/env python. In that case pip runs Python that you can find with which python.
PS. AFAIK the simplest way to install pip at RedHat is dnf install python-pip.
phd's answer was very helpful but I found that it was just a matter of using the root account to install the python packages. Then my Zeppelin was able to see any packages.
I am new to Python, so I was just trying to set my Linux Mint up for some Python development. I saw a lot of answers here on SO advising me against touching the pre-existing Python module, which is Python2.7. Hence, I installed Python3.7 separately. When it came to installing virtualenvwrapper, I installed it using the following command
sudo pip install virtualenvwrapper
Following some other article however, I think I installed another version of virtualenvwrapper with the help of Python3.7 using the following command
python3.7 -m pip install virtualenvwrapper
Now there exist two different versions of virtualenvwrapper and I know this because when I run the following
pip list | grep virtualenvwrapper
I get virtualenvwrapper 4.3.1
But when I run the following
python3.7 -m pip list | grep virtualenvwrapper
I get virtualenvwrapper 4.8.2
Now I am confused as to which one to use. I don't even know if there's any point of keeping two versions of the same thing. I don't know how to remove the virtualenvwrapper 4.3.1. I would like to use the one I installed using Python3.7, but not sure how to do that.
Please let me know if
Removing virtualenvwrapper 4.3.1 will break anything
If no is the answer to the question above, how do I remove that specific version of virtualenvwrapper
How to make use of the virtualenvwrapper installed using Python3.7
Thanks for your time
Welcome to Python!
You're traveling along the common axis of Python 2 vs Python 3. It's a future forward choice to begin your development using Python 3.
Removing virtualenvwrapper 4.3.1 will not break anything, especially if you haven't begun using it as part of your development workflow.
& 3. In Linux Mint, Python 2 and Python 3 are completely independent installations. So typically removing a package from one won't affect the other. virtualenvwrapper is a bit of an exception though, because it's more than a Python package - it's a sort of shell extension and it installs to /usr/local/bin/virtualenvwrapper.sh. I would suggest first running pip uninstall virtualenvwrapper to remove the 2.7 installation and then re-running python3.7 -m pip install --upgrade --force-install virtualenvwrapper to ensure the 3.7 one is good. This should leave virtualenvwrapper from 3.7 as the sole survivor and Python 3.7 as the one it uses by default.
As you're just starting into Python development, may I suggest a different path: may I suggest using https://pipenv.readthedocs.io/en/latest/, "an elegant weapon for a more civilized age". It streamlines the management of virtual environments and packages. While I started with virtualenvwrapper seven years ago, I've been using pipenv for the last year and loving it. python3.7 -m pip install pipenv will get you started in Linux Mint.
I want to write program in python3 (3.5), hence I installed python3 next to the pre-installed python2 (2.7) on Mac OS X El Captian.
Since my terminal runs python2.7 by default and Numpy is already installed for it, I put alias python=python3 and expected to be able to install Numpy for python3. when I type pip install numpy. This was the generated message:
Requirement already satisfied (use --upgrade to upgrade): numpy in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
I also noticed that I have no pip3 even though I am using python3: python --version returned Python 3.5.2, but pip3 install numpy got me -bash: pip3: command not found.
So my questions are:
1) How to install Numpy for python3.x when Numpy is installed on python2.x?
2) How to get pip3?
3) Is it better to use virtual environments, such as Conda, instead of juggling between python2 and python3 on the system?
Thank you from a total n00b
------------------- Update -------------------
Reinstalling python3 also fixed another problem in my case.
When I ran brew doctor, one of the warning message I got was:
Warning: You have unlinked kegs in your Cellar Leaving kegs unlinked can lead to build-trouble and cause brews that depend on those kegs to fail to run properly once built. Run brew link on these: python –
This is a result of me running brew unlink python in order to fix
"Python quit unexpectedly"
when I launch Vim and also
"The ycmd server SHUT DOWN"
Both seem to relate to the YouCompleteMe autocomplete plugin which I downloaded for Python.
I got my idea of removing symlinks from here and here
However, Homebrew evidently did not like the absence of those 39 symlinks.
After uninstall (brew uninstall python3) and then re-install python3 (brew install python3) as Toby suggested, Homebrew gave me
You can install Python packages with
pip3 install <package>
Then when I pip3 install numpy and pip3 install scipy, both executed successfully.
To my surprise, symlinks created during Python installation used to cause aforementioned error messages for Python and YouCompleteMe, but now I open python files using Vim without crash from a fresh Python installation, which definitely created the symlinks.
------------------- Update2 ------------------
After re-installing Anaconda2, the same YouCompleteMe error came back. I suspect Anaconda messed up symlinks.
I would recommend using the Anaconda Python distribution.
The main reasons are as such:
You will have a Python distribution that comes with numpy and the rest of the Scientific Python stack.
The Anaconda Python will be installed under your home directory, with no need for sudo-ing to install other packages.
conda install [put_packagename_here] works alongside pip install [put_packagename_here]; conda install is much 'cleaner' (IMHO, differing opinions are welcome).
If you have a Python 3 environment as your default, then pip works out-of-the-box without needing to remember to do pip3.
conda environments are easier to manage than virtualenv environments, in my opinion. And yes, you can have Python 2 alongside Python 3.
I once messed up my system Python environment - the one that came with my Mac - and it broke iPhoto (back in the day). Since then, I became convinced of needing separate, atomic environments for different projects.
I've detailed more reasons in a personal blog post.
Other distributions, of course, are all good, provided they give you what you need :).
The simplest way on a Mac is with Homebrew:
http://brew.sh/
Install Homebrew, then run:
brew install python3 pip3
Edit --
Python3 includes pip3, but Homebrew occasionally has trouble linking to the correct versions, depending on what has been installed. Running the following command:
brew doctor
And if you see errors relating to python or unlinked kegs, try running:
brew uninstall python python3
And reinstalling after checking brew doctor.
https://unix.stackexchange.com/questions/233519/pip3-linked-to-python-framework-instead-of-homebrew-usr-local-bin
Being new to Python, I'd love to clear up a few points that I couldn't get from reading various articles and tutorials.
After using Homebrew to install Python3, I noticed that it had installed both Python3 and Python3.4. I was also a little surprised that there are now three versions of pip on my machine too; pip, pip3 and pip3.4.
I created a new virtualenv and told it to use Python3, using the following command:
virtualenv -p /usr/local/bin/python3 mysite
I was also surprised that the version of Python that it installed in my VM was 3.4.
Is it safe to have these multiple version of Python and Pip hanging around on my machine?
Am I right to assume that I should take extra care to use the matching version of pip with Python, for example, pip3.4 with Python3.4?
Yes, it is safe. Python uses this naming like python3.4, python3.5 etc to differentiate between releases. python3 is a symbolic link to the current python3.x version. Pip follows the same convention.
If you're using python3.4 explicitly, you should be using pip3.4 specifically as well. Otherwise, just use python3 and pip3. For Python 2, you can simply use python (which, unless you installed the Homebrew version as well), will be the system Python), and ditto for pip. python2.7 and pip2.7 may also work.
In general, to find out which Python version goes with which pip you're using, try:
pip --version
and you'll see the Python included in the result.
No need to worried about if you have multiple version of Python and Pip installed. just check your version by writing in terminal :
$ brew info python
or to check the version of pip write in terminal :
$ brew info pip
and make sure you have updated your both pip and python version (write in terminal $ brew upgrade pip/python)
and other way to install python is go to https://www.python.org/downloads/ and choose as your requirement, there is two version available 2.7.9 & 3.4.3 ,
after installing python write in terminal $ python -V to check its version :) Hope it will help :)
I'm aware that there are similar questions on SO. This one, for example: What's the proper way to install pip, virtualenv, and distribute for Python?
I'd like to install these modules as per my Learn Python the Hard Way tutorial: http://learnpythonthehardway.org/book/ex46.html
I managed (I think) to install pip by using sudo easy_install pip but when I then ran pydoc modules I could not see it. So I'm not even sure it's installed.
The answer above in question 4324558 is difficult for me to understand: what's a bootstrap, what's curl and why would I set up a virtual environment? Yes, as a learner I should try to pick up as much as I can but I don't want to first create the universe, I just want to get the task at hand done.
How do I install these modules? Is it as complicated as it sounds in the quoted answer? The top voted answer says "Install virtualenv into a bootstrap virtual environment. Use the that virtual environment to create more. Since virtualenv ships with pip and distribute, you get everything from one install."
I really don't get what all that means. Isn't there something about the "Zen" of python and a one true way to get things done? Or am I out of context here? What is "the right way" to install these modules?
I tried:
pip install virtualenv in the terminal and received the following output:
Wheel installs require setuptools >= 0.8 for dist-info support.
pip's wheel support requires setuptools >= 0.8 for dist-info support.
Storing debug log for failure in /Users/myname/.pip/pip.log
I'm using a Mac and python 2.7
To solve your issue,
Just install (or upgrade) the setuptools:
sudo easy_install -U setuptools
Then you can run again: pip install virtualenv
Try adding 'sudo' in your command as-
sudo pip install virtualenv
It worked for me.
Have a look at Python Development Environment on Mac OS X Mavericks 10.9.
I followed these steps as well when trying to get Python 2.7 and Python 3.3 installed on OS X. It doesn't tell you how to install nose and distribute, but you should have a working environment and you can pick up from there.
I did have a problem using virtualenv and pip with Python 3, the question and solutions is available here.