Suddenly, I can't import pandas in python. I am using anaconda as package manager, but it seems that no matter how many times I uninstall and install pandas, I still get the same error:
(base) C:\>conda install pandas
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: C:\Users\au207178\Anaconda3
added / updated specs:
- pandas
The following NEW packages will be INSTALLED:
blas pkgs/main/win-64::blas-1.0-mkl
bottleneck pkgs/main/win-64::bottleneck-1.3.2-py38h2a96729_1
intel-openmp pkgs/main/win-64::intel-openmp-2021.4.0-haa95532_3556
mkl pkgs/main/win-64::mkl-2021.4.0-haa95532_640
mkl-service pkgs/main/win-64::mkl-service-2.4.0-py38h2bbff1b_0
mkl_fft pkgs/main/win-64::mkl_fft-1.3.1-py38h277e83a_0
mkl_random pkgs/main/win-64::mkl_random-1.2.2-py38hf11a4ad_0
numexpr pkgs/main/win-64::numexpr-2.8.1-py38hb80d3ca_0
numpy pkgs/main/win-64::numpy-1.21.2-py38hfca59bb_0
numpy-base pkgs/main/win-64::numpy-base-1.21.2-py38h0829f74_0
pandas pkgs/main/win-64::pandas-1.3.5-py38h6214cd6_0
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(base) C:\>python
Python 3.8.12 (default, Oct 12 2021, 03:01:40) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\au207178\AppData\Roaming\Python\Python310\site-packages\pandas\__init__.py", line 22, in <module>
from pandas.compat import (
File "C:\Users\au207178\AppData\Roaming\Python\Python310\site-packages\pandas\compat\__init__.py", line 15, in <module>
from pandas.compat.numpy import (
File "C:\Users\au207178\AppData\Roaming\Python\Python310\site-packages\pandas\compat\numpy\__init__.py", line 7, in <module>
from pandas.util.version import Version
File "C:\Users\au207178\AppData\Roaming\Python\Python310\site-packages\pandas\util\__init__.py", line 1, in <module>
from pandas.util._decorators import ( # noqa
File "C:\Users\au207178\AppData\Roaming\Python\Python310\site-packages\pandas\util\_decorators.py", line 14, in <module>
from pandas._libs.properties import cache_readonly # noqa
File "C:\Users\au207178\AppData\Roaming\Python\Python310\site-packages\pandas\_libs\__init__.py", line 13, in <module>
from pandas._libs.interval import Interval
ModuleNotFoundError: No module named 'pandas._libs.interval'
I have also tried running 'conda update conda'.
What's the next thing to do?
Update:
I checked, and the file is definitely there, in "C:\Users\au207178\AppData\Roaming\Python\Python310\site-packages\pandas_libs\interval.pyx".
However, I notice that part of the path to this library says 'python310'. However, my python version is 3.8.8. Is it possible that python is somehow looking in the wrong place when trying to import pandas._libs.interval? I have tried both python 3.10 (couldn't even install pandas) and python 3.9.7 (no effect).
I have also tried conda install --revision 0, which also had no effect...
Yes, it appears to be loading pandas from a user-level installation. User-level installs can leak into Conda environments and lead to unpredictable behavior, such as what you are seeing.
There are two routes of action of which I know. You may want to try the second one first, which would confirm the cause. However, the first option is likely more manageable going forward, since once it's done, the issue should be resolved.
Option 1: Remove External Python(s)
If you would like Conda to simply work as expected, then uninstall the user-level Python. Note that the one detected (Python 3.10) may not be the only one, so you may have to track down multiple copies. I'm not on Windows, so I can't suggest concrete steps for uninstallation.
However, you may be already using this user-level Python for other projects, so this option may not be practicable.
Option 2: Launch Python with Isolation Flags
There are some pertinent flags that Python provides that has it ignore the various sources that can lead to leaking in other site-packages. Here are the three important ones:
$ python --help
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
...
-E : ignore PYTHON* environment variables (such as PYTHONPATH)
-s : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S : don't imply 'import site' on initialization
...
Try them individually, or all together, and see if this fixes the issue. Individually, you may be able to narrow down whether it is due to PYTHONPATH (-E alone would fix) or generic user-level searching (-s alone would fix).
Additional Thoughts
I find it odd that a Python 3.10 installation is leaking into a Python 3.8 interpreter run. Usually, the site module only loads within matching major+minor versions (e.g., 3.8 should only pick up another 3.8). This might be indicative that something else is going on. Perhaps the PYTHONPATH environment is set? (It really never should be.)
Generally, I don't recommend using the base environment for work. This is never documented in Anaconda documentation, but it is the painful truth that many long-term Conda users come to, usually after their base either breaks or get's so overloaded with packages that it takes unreasonably long for them to upgrade the conda package. Instead, create a new environment and have that activate by default instead of base.
Related
I'm running SPSS 22.0.0.1 (64 bit edition) under Linux Mint 20 Ulyana (kernel 5.4.0-62-generic x86_64 GNU/Linux). Now I want to use python models not shipped with the "private" SPSS python edition.
The private Python version shipped with my version of SPSS is 2.7.1. Running the code below from SPSS syntax gives the private python version.
* SPSS syntax .
begin program .
import sys
print(sys.version)
end program .
outputs:
2.7.1 (r271:86832, Jun 14 2013, 00:22:41)
[GCC 4.3.4 [gcc-4_3-branch revision 152973]]
The system python that carries the modules I want use is 2.7.8.
~$ python2.7
outputs:
Python 2.7.18 (default, Aug 4 2020, 11:16:42)
[GCC 9.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
From what I understand, I can add a sitecustomize.py in the private python that points to the system python. The content of the file is just:
import sys
sys.path.append(r"/usr/lib/python2.7/dist-packages")
This seems to add the desired path to my private python. Listing the paths from SPSS syntax (last row of output):
* SPSS syntax .
begin program .
import sys
for p in sys.path:
print(p)
end program .
outputs:
/opt/IBM/SPSS/Statistics/22/extensions
/home/[USER]/.IBM/SPSS/Statistics/22/extensions
/opt/IBM/SPSS/Statistics/22/Python/lib/python2.7/site-packages/readline-6.1.0-py2.7-linux-x86_64.egg
/opt/IBM/SPSS/Statistics/22/Python/lib/python2.7/site-packages
/opt/IBM/SPSS/Statistics/22/Python/lib/python27.zip
/opt/IBM/SPSS/Statistics/22/Python/lib/python2.7
/opt/IBM/SPSS/Statistics/22/Python/lib/python2.7/plat-linux2
/opt/IBM/SPSS/Statistics/22/Python/lib/python2.7/lib-tk
/opt/IBM/SPSS/Statistics/22/Python/lib/python2.7/lib-old
/opt/IBM/SPSS/Statistics/22/Python/lib/python2.7/lib-dynload
/usr/lib/python2.7/dist-packages
However, when I try to import numpy in my SPSS syntax, I get this quite verbatim message:
* SPSS syntax .
begin program .
import numpy
end program .
outputs:
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "/usr/lib/python2.7/dist-packages/numpy/__init__.py", line 142, in <module>
from . import core
File "/usr/lib/python2.7/dist-packages/numpy/core/__init__.py", line 71, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
Here is how to proceed:
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed:
your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
1. Check that you are using the Python you expect (you're using ),
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy versions you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: No module named _multiarray_umath
I also tried to add the system python path directly in SPSS syntax, but no love...
* SPSS syntax .
begin program .
import sys
sys.path.insert(0, "/usr/lib/python2.7/dist-packages")
import numpy
end program .
SPSS is working fine with the private python. And the system python is working fine on its own.
You need to "tell" SPSS to use your system-Python. Be aware that SPSS 22 can only use python 2.7 (not other version of 2.x, and no 3.x). So you need to have that installed on your system. Then, in SPSS, go to Edit/Options/File locations, and set the Python 2.7 location to your system-Python installation folder. You will be able to use, from SPSS syntax, the libraries available in your system-Python.
Alternatively, you could go into your SPSS-python, and install the needed libraries with pip.
The file path: "C:\Users\Disander\anaconda3\python.exe"
When I open python from the path above, python shows the following warning:
Python 3.7.6 (default, Jan 8 2020, 20:23:39) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Warning:
This Python interpreter is in a conda environment, but the environment has
not been activated. Libraries may fail to load. To activate this environment
please see https://conda.io/activation
Type "help", "copyright", "credits" or "license" for more information.
>>>
Then when I try to import numpy or pandas, it throws the following error:
Traceback (most recent call last):
File "C:\Users\Disander\anaconda3\lib\site-packages\numpy\core\__init__.py", line 24, in <module>
from . import multiarray
File "C:\Users\Disander\anaconda3\lib\site-packages\numpy\core\multiarray.py", line 14, in <module>
from . import overrides
File "C:\Users\Disander\anaconda3\lib\site-packages\numpy\core\overrides.py", line 7, in <module>
from numpy.core._multiarray_umath import (
ImportError: DLL load failed: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Disander\anaconda3\lib\site-packages\numpy\__init__.py", line 142, in <module>
from . import core
File "C:\Users\Disander\anaconda3\lib\site-packages\numpy\core\__init__.py", line 54, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy c-extensions failed.
- Try uninstalling and reinstalling numpy.
- If you have already done that, then:
1. Check that you expected to use Python3.7 from "C:\Users\Disander\anaconda3\python.exe",
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy version "1.18.1" you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: DLL load failed: The specified module could not be found.
>>>
If I launch python from the anaconda terminal, python works perfectly fine. I show this by importin numpy and pandas. I import sys and print sys.executable to show that anaconda is using the same python executable python.exe.
(base) C:\Users\Disander>python
Python 3.7.6 (default, Jan 8 2020, 20:23:39) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.executable)
C:\Users\Disander\anaconda3\python.exe
>>>
>>> import numpy
>>> import pandas
>>>
I also tried to activate the (base) environment after executing python.exe. Below, I show what happened when I tried using "os.system('conda activate base')"
>>> import os
>>> os.system('conda activate base')
'conda' is not recognized as an internal or external command,
operable program or batch file.
1
>>>
I would like to know if there is a way to execute the python.exe file as if I executed it from anaconda using the (base) environment. I need this because I have another application depending on this executable. The same application uses it to import modules such as numpy and pandas but it is not working.
You need first be sure that "Scripts" folder is in your path environment (C:\Anaconda3\Scripts, for my setup).
Then
activate environment_name
in CMD should work. Replace 'environment_name' with 'base' if you want 'base' activated.
Open the Anaconda prompt or conda activate. Then print the path environment to the console via
(base) C:\> path
Add every listed folder below the Anaconda installation folder ..\Anaconda3\ to your user environment PATH variable. This will allow Python to also find numpy's C-Libraries.
\Anaconda3;
\Anaconda3\Library\mingw-w64\bin;
\Anaconda3\Library\usr\bin;
\Anaconda3\Library\bin;
\Anaconda3\Scripts;
\Anaconda3\bin;
Of cause this all ruins conda's well elaborated environment concept. So make sure you know what you're doing.
EDIT AFTER QUESTION EDIT:
If you have another application that depends on Python running in an Anacoda environment - like for example PowerBI Desktop - simply run that other application from the Anaconda prompt too.
EDIT AFTER 2ND QUESTION EDIT
If conda isn't recognized from os you clearly missed to initialize your shells via
conda init --all
Basically this is done during Anaconda installation by default, but some people prefer to skip this step for whatever reason.
I am having trouble installing a custom python module I have written.
Here are my steps so far:
Navigate to the module directory C:\Users\myname\repos\mymodulename where the setup.py file is in the anaconda prompt. Type: python setup.py install
The command prompt returns (plus some other things)
Extracting mymodulename-0.1-py3.7.egg to c:\users\myname\appdata\local\continuum\anaconda3\lib\site-packages
mymodulename 0.1 is already the active version in easy-install.pth
Installing myclass-script.py script to C:\Users\myname\AppData\Local\Continuum\anaconda3\Scripts
Installing myclass.exe script to C:\Users\myname\AppData\Local\Continuum\anaconda3\Scripts
Installed c:\users\myname\appdata\local\continuum\anaconda3\lib\site-packages\mymodulename-0.1-py3.7.egg
Processing dependencies for mymodulename==0.1
...
Using c:\users\myname\appdata\local\continuum\anaconda3\lib\site-packages
Finished processing dependencies for mymodulename==0.1
To me that looks like it has installed. Opening up the console and trying to import:
>>> import mymodulename.myclassas ce
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'mymodulename'
>>>
It appears in hasn't. Checking the list of modules in anaconda with help('modules') confirms that it has not been imported.
I thought that perhaps i had installed it to the wrong environment:
(base) C:\Users\myname>conda env list
# conda environments:
#
base * C:\Users\myname\AppData\Local\Continuum\anaconda3
py2 C:\Users\myname\AppData\Local\Continuum\anaconda3\envs\py2
Only a python 2 environment which mymodule wouldn't be compatible with.
Does anyone have any suggestions about what I can try to resolve this? Happy to elaborate on any of the points
Thanks in advance.
EDIT: Some more information that may be relevant.
This package was initially installed in site-packages. I have reinstalled there and the package works. The reason I moved the package is because I am aware it is bad practice to store custom packages there.
I tried for a long time to work with python on this Mac. I tried to remove python completely from my mac. Then I realized it was hard to get rid of this default python. Fortunately, I read in the internet that it's a bad idea to remove that one python that come with Mac before I figured out how to remove it. Then I also installed python, numpy, scipy, matplotlib using *.dmg packages. After all the effort, I still get this error message every time I import pylab. Please help me? Anyone?
$python
Python 2.7.4 (v2.7.4:026ee0057e2d, Apr 6 2013, 11:43:10)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pylab
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/pylab.py", line 1, in <module>
from matplotlib.pylab import *
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/pylab.py", line 221, in <module>
`from matplotlib import mpl # pulls in most modules
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/mpl.py", line 2, in <module>
`from matplotlib import axis
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/axis.py", line 14, in <module>
`import matplotlib.text as mtext
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/text.py", line 31, in <module>
`from matplotlib.backend_bases import RendererBase
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/backend_bases.py", line 48, in <module>
`import matplotlib.textpath as textpath
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/textpath.py", line 9, in <module>
`from matplotlib.mathtext import MathTextParser
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/mathtext.py", line 52, in <module>
`import matplotlib._png as _png
ImportError: dlopen(/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/_png.so, 2): Library not loaded: /opt/local/lib/libpng14.14.dylib
Referenced from: /Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/_png.so
Reason: image not found
Besides, I followed the instruction on http://ipython.org/install.html to install Anaconda. After Anaconda is installed I went to the next step
$ conda update conda
and got this
"-bash: conda: command not found"
I searched in the net, but it seems no one else has this problem. Can anyone give me a hint here? Many thanks!
In my directory /usr/bin I have
python python-config python2.5 python2.5-config python2.6 python2.6-config python2.7 python2.7-config
With command
which python
I got /Library/Frameworks/Python.framework/Versions/2.7/bin/python
with
which ipython
I got /Library/Frameworks/Python.framework/Versions/2.7/bin/ipython
For numpy I downloaded the package numpy-1.7.0-py2.7-python.org-macosx10.6.dmg, scipy-0.12.0-py2.7-python.org-macosx10.6.dmg for scipy, and matplotlib-1.2.0-py2.7-python.org-macosx10.6.dmg for matplotbib.I downloaded *10.6.dmg because that's what I found...I can not find something like *10.7.dmg
I don't have .bashrc or .bashrc_profile. But I think .profile will do, and it reads like this
# MacPorts Installer addition on 2012-03-07_at_18:55:26: adding an appropriate PATH variable for use with MacPorts.
export PATH=/opt/local/bin:/opt/local/sbin:$PATH
# Finished adapting your PATH environment variable for use with MacPorts.
export TERM="xterm-color"
alias ls="ls -G"
export PS1="[\[\e[33m\]\u#\H \[\e[32m\]\w\[\e[0m\]]\n[\[\e[31m\]\!\[\e[0m\]] > "
# Setting PATH for Python 2.7
# The orginal version is saved in .profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH}"
export PATH
When I type in
$port
it says -bash: port: command not found
Should that worry me?
Shall I also out comment the line
export PATH=/opt/local/bin:/opt/local/sbin:$PATH
as well?
You appear to have MacPorts. Perhaps you can use MacPorts to install the Python of your choice, and then use the corresponding pip (probably /opt/local/bin/pip to install the necessary Python packages.
If MacPorts is something you're not using anymore, I suggest to use homebrew instead: http://mxcl.github.io/homebrew/ . Pay attention to the message you get after installing Python: it tells you that some Python scripts will be installed in /usr/local/share/python. In your case, this may not be an issue (the packages you listed don't install Python scripts afaik).
Keep in mind that using homebrew and MacPorts together may still mess with Python and its packages.
Finally, you need to make sure you don't keep on using the other python executable. Therefore, in your .profile, comment out the last two lines like this:
#PATH="/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH}"
#export PATH
Update
Then, to get rid of the MacPorts reference and ensure your PATH picks up /usr/local/bin, change your .profile further to
# MacPorts Installer addition on 2012-03-07_at_18:55:26: adding an appropriate PATH variable for use with MacPorts.
#export PATH=/opt/local/bin:/opt/local/sbin:$PATH # <- comment out
# Set /usr/local/bin explicitly for Homebrew
export PATH=/usr/local/bin:$PATH
Start in a new Terminal (or tab) to have your settings updated.
I have python-matplotlib and python-mpltoolkits.basemap installed from Ubuntu packages. Installing python-mpltoolkits.basemap also installs python-dap as a dependency.
When I import basemap, I get this warning:
>>> import mpl_toolkits.basemap
/usr/lib/pymodules/python2.7/mpl_toolkits/__init__.py:2: UserWarning: Module dap was
already imported from None, but /usr/lib/python2.7/dist-packages is being added to sys.path
__import__('pkg_resources').declare_namespace(__name__)
What does this mean?
EDIT 1:
>>> import sys
>>> print sys.modules['dap']
<module 'dap' (built-in)>
EDIT 2:
$ python -S
Python 2.7.3 (default, Sep 26 2012, 21:53:58)
[GCC 4.7.2] on linux2
>>> import sys
>>> print sys.modules['dap']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'dap'
EDIT 3: yes I seem to have sitecustomize.py:
$ cat /etc/python2.7/sitecustomize.py
# install the apport exception handler if available
try:
import apport_python_hook
except ImportError:
pass
else:
apport_python_hook.install()
EDIT 4: actually I can get the error by:
>>> import pkg_resources
__main__:1: UserWarning: Module dap was already imported from None, but /usr/lib/python2.7/dist-packages is being added to sys.path
so this has nothing to do with mpl_toolkits.basemap as such.
I can't really say that I'd understand the details, but apparently whenever the package python-dap is installed, then trying to import pkg_resources gives this warning. Here is some discussion.
Following advice from here (comment 29 at the end of the page), I added dap as the first line in file /usr/lib/python2.7/dist-packages/dap-2.2.6.7.egg-info/namespace_packages.txt and get no more warnings. Hope this does not break anything.
I recently had to track down a similar problem, and the actual meaning of the error message:
UserWarning: Module dap was already imported from None, but /usr/lib/python2.7/dist-packages is being added to sys.path
Appears to be "While extending the path for 'dap', found an egg-info directory that does not declare 'dap' to be a namespace package".
This suggests two things: python-dap is missing a "namespace_packages=['dap']" declaration in its own setup.py, and setuptools really should give a better error message in this case...
If you don't need the package, simply remove it, e.g. on an Ubuntu or Debian system apt-get remove --purge python-dap removed the package for me and that silenced the warning. It is easy to accidentally install packages that you don't need because of dependency recommendations when installing some packages.
When you try to remove it the packaging system will warn you if removal of the package (in this case python-dap, but other packages could cause this error to happen, too) would also force removal of other packages which depend on it. In my case there are no other packages that directly depended on python-dap and I wasn't using it for anything that I was aware of, so uninstalling it was simple, painless, and silenced the warning.
Other package installers (such as the non-OS packaging systems like pip or easy_install) might make it more difficult to remove the package; you may need to delete the package by hand, so I'd instead recommend the accepted answer as the way to silence the warning unless the apt-get remove method I recommended here works for you.