Develop Reference

Failure importing openpyxl in Spyder 5.1.5

So I am trying to open a.xlsc file in Spyder using pandas. After importing pandas as pd, when using pd.read_excel() command in the console, it showed the error:
ImportError: Missing optional dependency 'openpyxl'. Use pip or.....
Then I installed the module with:
pip install openpyxl
successfully in CMD, but again Spyder shows the same error.
I tried to restart the kernel, close and reopen Spyder, and even restarted my machine to no avail.
Openpyxl is listed in both help ("modules") in python shell as well as in pip list in CMD.
When I tried to import the openpyxl itself in Spyder (import openpyxl)it says "Module not found error: No module named "openpyxl"."
I have only python 3 (3.9) not both 2 & 3 and Spyder is the only IDE I use with python.
I apologize if this question sounds stupid, I'm fairly new to Python.
I would appreciate any help regarding this.
Thank you.

Spyder has its own Python interpreter 3.7.9 so it loads only the modules installed for that Python.
Since I had a separate Python 3.9.7 installation, whenever I used pip via CMD it installed all the modules for the 3.9.7 Python, that's why my Python terminal & CMD were able to list openpyxl but Spyder said it isn't installed.
After ransacking through the internet I switched the Spyder interpreter to the 3.9.7 stand-alone installation by Tools > Preferences > Python interpreter > Use the following interpreter > navigate through your directory and select the stand-alone python 3.7.9 .exe. Afterwards, Spyder said "Your Python environment or installation doesn't have the spyder‑kernels module or the right version of it installed. Use pip install spyder-kernels==2.1. to install..."*. So back to CMD, I installed the spyder-kernels and now everything's fine.
Spyder IPython console shows Python 3.9.7 (Used to be 3.7.9, Spyder's own interpreter). Now I'm able to import openpyxl without any issues.

I had the same issue as Anoban, but the answer provided by Anoban didn't work for me. I was able to resolve it in two ways
1. Changing my Python environment
This is the better, quicker answer IMO. My Spyder environment was somehow using a custom Python environment as its default (see picture).
Once I changed the Python environment (Tools > Preferences> Python interpreter) from "Default" to the following path (it should have been the default Python interpreter in the first place IMO):
"C:\Users\username\miniconda3\python.exe"
...it worked (see image below to what it should look like in the IDE).
I still had to do an install of spyder-kernels=2.3 (see the next point in my post if you have trouble with that)
2. Creating a separate Python environment
I got this way to work before I figured out the method described in #1 above, so I am posting it too, since it may help someone.
I used the help provided on the Spyder website, followed the directions provided on their site exactly, and it worked for me.
FYI, my installation is slightly different than Anoban's - I didn't use pip. I installed miniconda and a stand-alone version of Spyder on my Windows machine. I used the Anaconda prompt as the terminal window, not the Windows Command Prompt.
Hope this helps someone.
update: when I changed the Python interpreter to the new environment, the Python console gave me the following error
The Python environment or installation whose interpreter is located at
C:\Users\rebecca\miniconda3\python.exe
doesn't have the spyder‑kernels module or the right version of it installed (>= 2.3.0 and < 2.4.0). Without this module is not possible for Spyder to create a console for you.
You can install it by activating your environment (if necessary) and then running in a system terminal:
conda install spyder‑kernels=2.3
But it couldn't find the correct channels for installation so I needed to use
conda -c conda-forge spyder-kernels=2.3
then it worked.

#rpinto73
I know it's tad late. I'm wondering how my answer differs from yours lol.
We faced the same issue and came up with the same solutions. The difference is you set the Python interpreter in a conda environment as default in Spyder whereas I set my standalone Python installation as default. Thus we both assigned an external Python interpreter to be used inside Spyder. Since the external interpreters we assigned (a global Python interpreter in my case (that's why I used pip) and a Python interpreter inside a conda virtual environment in your case (so you had to use conda, you could use pip inside a conda virtual environment as well but not a good practice since it could break dependencies) didn't have the spyder-kernels module we had to install it manually.
The major issue with Spyder is it is an IDE WRITTEN in Python. Thus it needs a Python interpreter to load & function; which is the reason for it being painfully slow during loading. Spyder packs some mundane data science libraries like pandas, numpy, scipy with itself, which you make use of when using Spyder's default Python interpreter. However Spyder does not allow users to manually install packages for its default interpreter (as far as I know). Which makes sense since any conflicts in dependencies introduced by users manually installing/upgrading packages might break the functionality of Spyder itself, making it dysfunctional. Thus the best solution is to leave the Spyder's interpreter to Spyder itself and use a custom Python interpreter where we can install and upgrade modules as we please!.
And a free advice, it's okay to use Spyder when you are starting out with Python as beginners, but it is useless when you have to do heavy computations. You will mostly find yourself facing memory errors. It is lightweight IDE best for learners & beginners. It's best to switch to a more general yet capable development environments like Visual Studio Code, Jupyter Lab.. or a full-fledged Python IDE like PyCharm (there is a free community edition) if you want the in-IDE variables pane & plots pane. But be warned PyCharm (generally any IDE from JetBrains) is notorious for having massive memory footprints.

How to reset Python in macOS

Currently, I feel like my Python ecosystem is out of whack. Several years ago, I had shared my computer with someone else, and now I am discovering that my machine has many versions of Python scattered about. From what I could find, starting from Macintosh HD:
anaconda
Applications/Python 3.6
Contains IDLE, Python Launcher, etc...
Library/Python/2.7/site-packages
This contains things like pip and wheel (I am unsure what this is)
Library/Frameworks/Python.framework/Versions
This contains two folders, 3.5 and 3.6.
System/Library/Frameworks/Python.framework/Versions
Inside here there are many alias folders, all which point to 2.7
Is all of this supposed to be normal? I am trying to run Python from the terminal, yet I have been getting messages such as ImportError: No module named site. pip has also not been working.
Is there a way for me to reset the Python on my machine to just that which comes with macOS? I feel like starting over from a blank slate would be helpful, since I want to get things setup in some type of comprehendible way (e.g. a way in which I've set things up so I know what is on my machine instead of many random things put on it by another).

This is very normal. you have a preinstalled python2.7 which comes with macOS and another one "Anaconda" which has installed manually. You first need to check what is your default python path (version) on your macOS, I mean if you are using anaconda or the default pre-installed python2.x. To remind you can check like below:
python --version
output (for me): Python 3.7.6
then if you want to change it to another version/or use another version under conda you can check these two answeres of mine.
How to add anaconda to PATH?
and here:
How to set the default python3 to python3.7?
It is normal that when you install a package for your default python version you don't expect it to be installed on the other one too. Normally it is better to install python2.x as an environment of conda and switch between two environments with "conda activate py2" and "conda deactivate" to go back to your default version. for each of them, you need to be in the environment and then use pip.
if pip is not working, it may need to be installed. On macOS, as you remember, you can use
brew update
brew install pip
or "easy-install" instead of brew (or whatever you use for installation).

Ubuntu - PySide module not found for python2 but works fine for python3

I had PyQt4 running fine with python2 on Ubuntu 12.04. I then installed python-PySide. But the installation test would give me a module not found error. Then I installed python3-PySide and it works fine. So obviously something to do with my environment paths, but I'm not sure what I need to do. I'm guessing PySide is automatically checking if python3 exists and if it does then it'll use it regardless. I need PySide to work with python2.7 because of Qt4.8 compatibility issues. Any suggestions?
some info about my system:
which python
/usr/bin/local/python
which python3
/usr/bin/python3
EDIT:
More details about installation test.
After installation, I bring up the python console and try import PySide, as follows:
python
import PySide
ImportError: No module name PySide
But it works fine for python3:
python3
import PySide
PySide.version
'1.1.2'

You have two independent Python 2.7 installations, one in /usr and one in /usr/local. (And that's on top of the Python 3.x installation you also have.)
This is bound to cause confusion, especially for novices. And it has caused exactly the kind of consuion it was bound to cause.
You've installed PySide into the /usr installation, so it ended up in /usr/lib/python2.7/dist-packages. If you run /usr/bin/python, that import PySide will probably work fine. (If not, see below.)
But the default thing called python and python2.7 on your PATH is the /usr/local installation, hence which python says /usr/local/bin/python, so it can't see PySide at all. So you need to get it installed for the other Python as well.
Unless you know that you need a second Python 2.7 in /usr/local for some reason, the simplest thing to do would be to scrap it. Don't uninstall it and reinstall it; just uninstall it. You've already got a Python 2.7 in /usr, and you don't need two of them.
If you really need to get PySide working with the second 2.7…
Since you still haven't explained how you've been installing PySide despite being asked repeatedly, I can't tell you exactly how to do that. But generally, the key is to make sure to use explicit paths for all Python programs (python itself, python-config, pip, easy_install, etc.) that you have to run. For example, if the docs or blog or voices in your head tell you to run easy_install at some step, run /usr/local/bin/easy_install instead. If there is no such program, then you need to install that. The fact that you already have /usr/bin/easy_install doesn't help—in fact, it hurts.
If you can get rid of the second Python, but that doesn't fix PySide yet, uninstall, rebuild, and reinstall PySide. Or, even simpler… PySide has pre-made, working binary Ubuntu packages for all of the major Python versions that have Ubuntu packages. Just install it that way.

Django - Mac development, environment hell

I was trying to setup Django dev environment on Mac and arrived into a hell. It all started when trying to install PIL, which failed after trying 15 or so different recipes I found on blogs. So I wanted to install the Python, this time 2.7, and reinstall setuptools, easy_install, pip from scratch.
After just installing Python 2.7, and easy_install with setuptools for 2.7, this all in turn created such a mess that is unbelievable. Different version of Python are installed everywhere, easy_install is installed everywhere and points randomly to different python hashbangs (sometimes to #!/usr/bin, #!/usr/local/, #!/Library/...)
Now I can't even do easy_install pip, which I always could. So I'm already in a hell and I haven't even attempted to install MySQL yet.
My question finally is did anyone bump into such problems, it would help enough to know that I'm not alone.
Second, would it be easier to set up the entire environment on Ubuntu than it is on a Mac?
Thirdly, is there any guide that can really clearly explain how to set up but also tear down the stack for Python development on a Mac?

It wouldn't hurt to run a VM with vagrant. This post should tell you more:
http://stevelosh.com/blog/2011/06/django-advice/
Of course using virtualenv should also help alleviate some of these issues.

I've gone through the same hell 2 weeks ago :)
I needed to make working python 2.7 and virtualenv on OSX 10.6.8.
You haven't mentioned virtualenv in your question but I strongly recommend it. That way you minimize amount of globally installed packages. Everything is... cleaner.
My idea is to only have following things globally:
python (from brew)
pip (via easy_install)
virtualenv (via pip)
virtualenvwrapper (via pip)
other through either virtualenv or buildout
I've just checked and pip PIL installs fine within my virtualenv.
Here are notes from this battle (gist.github.com):
#NOTE: .pydistutils.cfg seems to be not compatible with brew install python
#areas I needed to clean before installation
#clean up ~/Library/Python
#clean up .local
brew install python
easy_install pip
pip install virtualenv
pip install virtualenvwrapper
mkdir $HOME/.virtualenvs
Example .bash_profile:
#homebrew
export PATH=/usr/local/bin:/usr/local/sbin:${PATH}
# homebrew python 2.7
export PATH="/usr/local/share/python:${PATH}"
#virtualenv wrapper
export WORKON_HOME=$HOME/.virtualenvs
source /usr/local/share/python/virtualenvwrapper.sh
Good luck!

Second, would it be easier to set up
the entire environment on Ubuntu than
it is on a Mac?
To answer this question (though I never used Mac though): I never had problems setting up a python environment for Django development on Ubuntu. Though in any case you should go with the built-in Python version if possible. Attempting to install any other Python versions usually ends up messy. Luckily with Ubuntu 11.04 the standard version is already 2.7.

I'm using django development environment on a MAC OS X 10.8 with python 2.7. I don't use virtualenv ore some other things.
With all the respect can say that there is NO ANY PROBLEMS to develop on a mac. Mac is a UNIX like system and you've probably seen that all tools for developers have MAC ports.
As for the setup mess. It's a good idea to use virtualenv. As for PIL installation. I needed to compile it with TrueType. As I'm in common with UNIX like environments it was not heavy task for me to compile PIL from sources using GCC (it's already installed on a MAC)... There are some mess with Django to setup virtualenv... There are certainly lots of articles to setup it on Google.
I use Eclipse and write all my PYTHONPATH variables there. You can forget installing everything like in Linux and try not to make anymore mess with installed tools. Try to read THIS article if you feel like you're ok to use Eclipse for your development on a MAC. It also has a recipe to avoid mess with installation of many copies of Python and other dev utils.

Yes I have had problems with MacOS. I think rather than trying to figure it out I just switched to Ubuntu. I use a mac with Ubuntu installed in VMware Fusion. I have developed on both and prefer the Ubuntu because I'm just more comfortable with installing packages and the file structure.
I love using the VM because I'm never scared of having to start over. I can get a whole new OS installed and get the packages with what I use in just a few hours. Not to mention with 6month rollouts I can do complete installs of new versions instead of updates.
Depending on your production environment, it may be beneficial to use an OS that is similar, if you can install a package on ubuntu desktop, you already know how to do it on ubuntu server.

What is the best way to install python 2 on OS X?

A colleague of mine wants to use my python 2 code on his OS X (10.6) machine. My code imports several built-in python packages, including Tkinter and shelve, and also uses third-party packages, including numpy, scipy, matplotlib, and ipython.
I've encountered a few problems with OS X's built-in python. (IDLE doesn't work, for example*). I suspect I should install a more recent version of python, and a different version of Tk.
My questions:
Will having two different versions of python/Tk on the same machine cause problems?
I would like to associate the terminal commands 'python', 'ipython', and 'easy_install' with the more recent version of python. How should I do this?
When I install third-party packages like numpy using a .dmg file, how do I control which version of python numpy installs into?
Is there a better way to do this?
If this process goes well, I'd consider adding OS X instructions to my code's documentation, so I'd like to boil down this process to the simplest, most general approach.
*EDIT: Also, this
EDIT: Thank you everyone for the useful answers. My colleague tried MacPorts, which seems to work well, but has a few speedbumps. First we had to install Xcode from the system install disk. This is not a fast or lightweight install (several GB). Luckily we still had the disk! Once Xcode was installed, MacPorts was easy to install. Python and the python subpackages we needed were also easy to install, but he told me this installation took several hours. Presumably this delay is due to compilation? He had an easy time setting the MacPorts python as default. However, I think we have to change the 'Python Launcher' application by hand, this seems to still default to the system python.
Even though he has a working system now, I'm tempted to ask him to try one of the other solutions. I'm not sure all of my code's potential users will tolerate a multi-hour, multi-gigabyte installation.

I use brew to install all my libraries/compilers/interpreters.
To install python try this:
brew install python
Then add Python's binaries directory to your $PATH in your ~/.profile:
export PATH=`brew --prefix python`/bin:$PATH
I'd recommend you to install pip, virtualenv and virtualenvwrapper to have better control over your environment too.

Have you tried ActivePython?
It includes a package manager (PyPM) that, by default, installs into your home directory (eg: ~/Library/Python/2.7). Main scripts get symlinked in /usr/local/bin; use the included pythonselect to set the active Python version.
You don't have to bother installing .dmg packages, as PyPM is a binary package manager ... therefore you can install non-pure Python packages like NumPy without having to compile things yourself.
ActivePython can use Apple's Tcl/Tk or, if installed, ActiveTcl.
A "simplest, most general approach" in your documentation could be:
Install ActivePython 2.7
Open Terminal and type pypm-2.7 install matplotlib ipython

Using MacPorts, you can install python 2.6, 2.7, 3.1 and 3.2 at the same time, with their own packages, without ever touching the built-in python.
numpy, scipy, matplotlib, and ipython are also available as ports for most of those python versions.
Moreover, if you install the python_select port, you'll be able:
to choose which one of those (plus the built-in python) is the "default" python;
to install python packages through easy_install/pip for the "selected" python, if they're not available as ports.
Add virtualenv to the mix, and you'll have a very, very flexible Python development environment.
As for your questions:
Q1: with MacPorts, no. while not a frequent user, I've installed and used matplotlib in 2.6 and 2.7, switching between the two using python_select.
Q2: easy_install, pip, ipython will be "linked" to the python they were installed by. (but see tip 1)
Q3: it's easier to install one of the py{26,27,xx}-numpy ports, or pip install numpy under your python_select'ed python.
Q4: well, MacPorts is the best thing I know after APT on Debian/Ubuntu... :-)
Now, two tips if you try MacPorts:
MacPorts cleanly installs ports separately from the OS X installation, in an /opt/local directory, and each python version is installed in a /opt/local/Library/Frameworks/Python.framework/Versions/{2.5,2.6,2.7,...} directory. Using python_select cleanly switch the "python" command using links. BUT... the Versions/{2.5,2.6,2.7,...}/bin directory, where python scripts are installed, is not added to the PATH. Just adding: export PATH=/opt/local/Library/Frameworks/Python.framework/Versions/Current/bin:$PATH to your ~/.profile will always give you direct access to the scripts installed for the selected python.
to avoid bad surprises, I've added a echo Selected python is \"$(python_select -s)\" line to my ~/.profile, so I always know which is my currently selected python when opening a session... :-)
Regards,
Georges

In almost all cases, the best python to use is the one from http://python.org/. It sets up the paths correctly and doesn't overwrite anything. DMG package installs usually work automatically, as does python setup.py install, and it's not too hard to get setuptools to work. If you want per-user installs, it is easy to set up .pydistutils.cfg and python automatically recognizes the path install_lib = ~/Library/Python/$py_version_short/site-packages

An addendum regarding the usage of brew:
Since some time, brew install python will install python3.
If you intend to install python2, you want to use
brew install python#2
It is perfectly fine to install both python and python3 using brew!

Here is an old post that answers your questions too.
In general it is not a problem at all to have more than one python installation on your machine. You just have to watch out which one you are calling on the command line.
>> which python
... helps to identify where your python binary is located. The original Mac OS X python is usually at "/usr/bin/python"
I personally use the MacPorts python installation. It also supports you with the installation of modules. (see link above)

I have 4 versions of python on my MacBook Pro. 2 from the original install of OS X 10.6 and a subsequent update, then self installed copies of python 2.7 and 3.2. You can update the python command to point at any of the versions. They all install in separate directories and cause no problems with each other.
I'm not sure what will happen when you install from a .dmg file. I believe it will simply use whatever version python points to.
This post on superuser.com answers your questions on changing default paths.