Develop Reference

Import failure when calling Anaconda Python from Windows shell

I want to call my Anaconda Python from the Windows shell. The goal is to eventually call it from MATLAB using a system call, so I have to use the plain Windows shell, and not the "Anaconda prompt".
I have a simple Python script that I want to run, but it doesn't get past the import statements:
> "C:\ProgramData\Anaconda3\python.exe" lhs.py
Traceback (most recent call last):
File "lhs.py", line 8, in <module>
import numpy
File "C:\ProgramData\Anaconda3\lib\site-packages\numpy\__init__.py", line 140, in <module>
from . import _distributor_init
File "C:\ProgramData\Anaconda3\lib\site-packages\numpy\_distributor_init.py", line 34, in <module>
from . import _mklinit
ImportError: DLL load failed: The specified module could not be found.
EDIT
I made a simplified Python script (error.py) containing only one line:
import numpy
I then call it as:
"C:\ProgramData\Anaconda3\python.exe" error.py

TL;DR
C:\ProgramData\Anaconda3\condabin\activate.bat YOURENVIRONMENTNAME && python N:\Temp\lhs.py
Long Answer
There is a similar issue reported on github. I found this by searching for the last few lines of your message. It is about this:
File "C:\Users\hanna\Anaconda3\lib\site-packages\tensorflow\python_init.py",
line 47, in
import numpy as np
File "C:\Users\hanna\Anaconda3\lib\site-packages\numpy_init_.py", line 140, in
from . import _distributor_init
File "C:\Users\hanna\Anaconda3\lib\site-packages\numpy_distributor_init.py",
line 34, in
from . import _mklinit
There, they link to a conda troubleshooting page:
Error messages like
Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll
Cause
NumPy is unable to load the correct MKL or Intel OpenMP runtime
libraries. This is almost always caused by one of two things:
The environment with NumPy has not been activated.
Another software vendor has installed MKL or Intel OpenMP
(libiomp5md.dll) files into the C:\Windows\System32 folder. These
files are being loaded before Anaconda's and they're not compatible.
If you are not activating your environments, start with doing that.
So let us try this first:
C:\\ProgramData\\Anaconda3\\condabin\\activate.bat
conda run -n yourenvironment C:\\path\\to\\your\\pythonfile\\lhs.py
(Yes, from a normal cmd terminal)
If that seems to work, you should be able to chain the two commands using &&. My own example looks like this:
F:\Programme\Programme\Anaconda3\condabin\activate.bat && conda run -n textrecognition N:\Temp\temp.py
This can be simplified further to
F:\Programme\Programme\Anaconda3\condabin\activate.bat textrecognition && python N:\Temp\temp.py
In case of the problem being number two, you have the option to load conda things before system32. This is supported in anaconda starting the following versions:
Python 2.7.15 build 14
Python 3.6.8 build 7
Python 3.7.2 build 8
To activate that, you need to set some environment variables to 1:
Control environment variables:
CONDA_DLL_SEARCH_MODIFICATION_ENABLE
CONDA_DLL_SEARCH_MODIFICATION_DEBUG
CONDA_DLL_SEARCH_MODIFICATION_NEVER_ADD_WINDOWS_DIRECTORY
CONDA_DLL_SEARCH_MODIFICATION_NEVER_ADD_CWD
To set variables on Windows, you may use either the CLI (Anaconda
Prompt, for example) or a Windows GUI.
CLI:
https://superuser.com/questions/79612/setting-and-getting-windows-environment-variables-from-the-command-prompt/79614
GUI:
http://www.dowdandassociates.com/blog/content/howto-set-an-environment-variable-in-windows-gui/
These should be set to a value of 1 to enable them. For example, in an
anaconda prompt terminal:
set CONDA_DLL_SEARCH_MODIFICATION_ENABLE=1

you need to activate your conda base in cmd before calling the script
call <path>\anaconda\Scripts\activate base
<path>\anaconda\python.exe <path>\scriptname.py

naoqi 2.5.5.5 and PyCharm CE 2017.1

I have an annoying problem with PyCharm CE and naoqi. Following the installation instructions in Aldebaran Guide, I have configured my environment to work with this one.
I have installed the version of python (python 2.7.13) suggested by Aldebaran, I have written the .bash_profile using:
# Setting PATH for Python 2.7
# The original version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH}"
export PATH
#NaoQI 2.5.5
export PYTHONPATH=${PYTHONPATH}:/Users/francesco/Documents/PyLibs/pynaoqi-python2.7-2.5.5.5-mac64/lib/python2.7/site-packages
export DYLD_LIBRARY_PATH=${DYLD_LIBRARY_PATH}:/Users/francesco/Documents/PyLibs/pynaoqi-python2.7-2.5.5.5-mac64/lib
after that, I launch my code in python from Terminal (making sure that the right version of python is called when I type
$ python my_code.py
and all is working (I tried also with virtualenv and all works fine if I launch all from terminal).
So far so good but then comes the poison arrow. I use PyCharm CE like IDE, I have created a new project that use the same environment that works fine in terminal and the magic happens..
The first issue is:
/Users/francesco/Documents/PyEnv/pynaoqi255_python2713/bin/python /Users/francesco/PycharmProjects/PepperRobot/PepperRobot.py
Traceback (most recent call last):
File "/Users/francesco/PycharmProjects/PepperRobot/PepperRobot.py", line 5, in
import qi
ImportError: No module named qi
(Ok seriuosly this is the same environment that in terminal works)
I try to bypass the problem, adding the content of naoqi sdk in the virtualenv's site-package. Ok, the problem is bypassed.. But there is another problem:
/Users/francesco/Documents/PyEnv/pynaoqi255_python2713/bin/python /Users/francesco/PycharmProjects/PepperRobot/PepperRobot.py
Traceback (most recent call last):
File "/Users/francesco/PycharmProjects/PepperRobot/PepperRobot.py", line 5, in
import qi
File "/Users/francesco/Documents/PyEnv/pynaoqi255_python2713/lib/python2.7/site-packages/qi/init.py", line 88, in
from _qi import Application as _Application
ImportError: dlopen(/Users/francesco/Documents/PyEnv/pynaoqi255_python2713/lib/python2.7/site-packages/_qi.so, 2): Library not loaded: #loader_path/libqipython.dylib
Referenced from: /Users/francesco/Documents/PyEnv/pynaoqi255_python2713/lib/python2.7/site-packages/_qi.so
Reason: image not found
Process finished with exit code 1
I did not understand why...
I have also include in
Preferences -> console -> Python Console -> Environment Variables
the same variables included in .bash_profile but to no avail.
How can I solve this tedious problem ?

I have solved my problem editing run/debug configuration and putting here the environment variable DYLD_LIBRARY_PATH. All works fine.

Python line_profiler not finding module

I recognize that this is an installation failure on my part, and I'm sorry to lay this uninteresting and inconsequential question at your feet, but for the life of me I can't manage to figure out what is going wrong and I've run out of ideas. I'm hoping someone will be able to quickly point out the obvious.
I am trying to profile a python script (using Kern's line_profiler), and the script needs to load the netCDF4 module. I have installed both line_profiler and netCDF4 with pip. Both are reported as present and updated when I queue pip for the list of installed packages.
Without using the profiler, my script runs without problems, meaning that the netCDF4 module is loaded properly. However, if I run 'kernprof -l -v myscript.py' from the "myscript" directory, I get the following error:
Traceback (most recent call last):
File "/usr/local/bin/kernprof", line 9, in <module>
load_entry_point('line-profiler==1.0', 'console_scripts', 'kernprof')()
File "Library/Python/2.7/site-packages/kernprof.py", line 221, in main
execfile(script_file, ns, ns)
File "myscript.py", line 5, in <module>
from netCDF4 import Dataset
ImportError: No module named netCDF4
I am running Python from an installation at /opt/local/bin/python, which is listed first in my PATH.
So, in any case, if the default Python version that I have set is the same as that which appears first in my PATH, and that default version is able to access the netCDF4 module, why isn't line_profiler?

kernprof has a shebang that redirects to the default python install which doesn't have all the required modules.
You can force the use of your "complete" python install by doing:
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/‌​Resources/Python.app‌​/Contents/MacOS/Pyth‌​on /usr/local/bin/kernprof -l -v myscript.py
So shebang is ignored, and you run the profiler with the version of python containing all the required packages.

Error with TensorFlow MNIST [duplicate]

Since updating from Homebrew Python 2.7.11 (from 2.7.10) I'm suddenly unable to test register my package on PyPi from the PyCharm IDE console.
Running (as an "External Tool")
python -B setup.py register -r pypitest
I now get
Traceback (most recent call last):
File "setup.py", line 22, in <module>
from setuptools import setup
File "/usr/local/lib/python2.7/site-packages/setuptools/__init__.py", line 12, in <module>
from setuptools.extension import Extension
File "/usr/local/lib/python2.7/site-packages/setuptools/extension.py", line 8, in <module>
from .dist import _get_unpatched
File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 16, in <module>
from setuptools.depends import Require
File "/usr/local/lib/python2.7/site-packages/setuptools/depends.py", line 6, in <module>
from setuptools import compat
File "/usr/local/lib/python2.7/site-packages/setuptools/compat.py", line 17, in <module>
import httplib
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 80, in <module>
import mimetools
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/mimetools.py", line 6, in <module>
import tempfile
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tempfile.py", line 32, in <module>
import io as _io
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/io.py", line 51, in <module>
import _io
ImportError: dlopen(/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so, 2): Symbol not found: __PyCodecInfo_GetIncrementalDecoder
Referenced from: /usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so
Expected in: flat namespace
in /usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so
Process finished with exit code 1
I'm not sure how to proceed. I only get this issue if I execute from within my IDE's console. If I do it directly at the system command line (Terminal on OS X) I have no problems.
OS X 10.11.3; Homebrew Python 2.7.11; PyCharm 5.0.3

tl;dr: Fix this issue by doing one of the following:
type hash -r python, OR
log out and log in.
EDIT: An answer to my related question makes it clear what's happening here. When you install a new version of python, you may need to run hash -r python to tell bash to reset the "cached" location to the python executable.
In my case, I was typing python, which was on my $PATH at /usr/local/bin/python. But bash was still using the old cache location /usr/bin/python. So, the old executable was called, but the new path was provided to python in sys.argv[0]. This means that the old executable was running, but the new sys.executable value caused all the wrong modules to get loaded (including the io module).
I'm having the same problem. I installed python 2.7.11 via an installer from Python.org. Strangely, the issue seems to be related to some subtle difference between how OSX launches python when I invoke it from the shell using the full path vs. using just the word python.
So, for me, this works (invoking python via the full path /usr/local/bin/python):
$ which python
/usr/local/bin/python
$ /usr/local/bin/python -c "import io"
$
... but this doesn't:
$ python -c "import io"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/io.py", line 51, in <module>
import _io
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so, 2): Symbol not found: __PyCodecInfo_GetIncrementalDecoder
Referenced from: /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so
Expected in: flat namespace
in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so
So, as a workaround, you can try doing the same thing.
Elsewhere, I've posted a separate question about this puzzling behavior. Maybe somehow merely calling python invokes some strange mix of the 2.7.11 executable with the 2.7.10 dylibs??

According to https://github.com/klen/python-mode/issues/634:
I had the same issue, but successfully fixed. In my case I compiled
python and vim with homebrew, when PYTHON_PATH has been specified and
set to one of my dev environments, where I also had some libraries,
including io. Workaround was simple: open new terminal, make sure that
you do not have custom PYTHON_PATH, uninstall python, uninstall vim.
Reinstall both of them.
and
Problem solved.
Culprit is the update from python 2.7.10 to 2.7.11.
If you are using conda package control, simply run "conda install
python=2.7.10" will solve this problem.
This doesn't give the root cause though. Since this happens with _io, this looks like a bug in python 2.7.11 (unlikely, there would be a world-scale outcry and a prompt fix if it was) or some packaging bug or version mismatch specifically with the homebrew version (and maybe some related ones, too).
Try to import _io in the console and if it succeeds, check if it was loaded from the same path.

Reinstall python.
brew unlink python && brew reinstall python
Secure the path
export PYTHONPATH=$PYTHONPATH:/usr/local/bin/
BACKUP and Change the order of "paths" file.
sudo nano /etc/paths
it seems, the order of paths, it is decisive to run python properly. In my case, the result was:
#sudo nano /etc/paths
/usr/bin
/usr/local/bin
/bin
/usr/sbin
/sbin
On my mac, path is like this.
$ which python
/usr/local/bin/python
Now I can run both:
$ /usr/local/bin/python -c "import io"
$ python -c "import io"

I had the same issue, it is successfully fixed by just replacing the _io.so file.
sudo find / -name _io.so
copy the path of the _io.so file which DOES NOT belong to python-2.7.11. For example, copy the path of _io.so which is under python-2.7.5:
/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so
Replace the /usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so file with the _io.so that you just found.

This happened to me as well in MacVim. I solved it by making sure :python print(sys.path) is using system Python (e.g. /Library/Python/2.7/...)
Since I installed MacVim via Homebrew, I just did that by:
Spawn a new shell that had which python -> /usr/bin/python. For my case I needed to remove the pyenv line from my .bash_profile. If you installed Python via Homebrew you may want to brew unlink python first
brew reinstall macvim

If your problem is caused by anaconda, it is unnecessary to remove //anaconda directory.
Just open your ~/.bash_profile, find the line
export PATH="//anaconda/bin:$PATH
and comment it out, then restart your terminal session.

Another quick workaround if you don't mind sticking with Python 2.7.10 is to specify the path of the Python interpreter executable that will be used for the virtualenv. On OSX that path is usually /usr/bin/python:
virtualenv venv --python=/usr/bin/python

Can't add comment (?) so this just to share my exp., downgrade to 2.7.10 works fr me.

I got this error after a failed NLTK download, I needed to uninstall anaconda:
sudo rm -rf ~/anaconda
update PATH variable

This happened when I already had tried to create a venv in a folder, and mistakenly was trying to initialize a second one! So I just removed venv directory and re-ran the command. Very likely this is not the answer to this solution, but searching my error brought me here, so it may help some others who are stuck.

I solved this issue by removing the symbolic link that was in /usr/local/bin and copying the actual python binary, that was pointed to by said link, there.

I had the same issue when I tried to use PyCharm. Solved by setting "python interpreter" in project configuration to point to the python virtual env I wanted to use, which was an Anaconda env. Somehow the interpreter path was missing the "anaconda" portion of ~/.../anaconda/.../_io.so. No need to uninstall anaconda.

Cannot import mysql-python in crontab Mac OS

I wrote a script which runs ok, but when I run it from crontab I get an import error.
Traceback (most recent call last):
File "/Users/.../Desktop/test.py", line 3, in <module>
import MySQLdb as mysql
File "build/bdist.macosx-10.7-intel/egg/MySQLdb/__init__.py", line 19, in <module>
File "build/bdist.macosx-10.7-intel/egg/_mysql.py", line 7, in <module>
File "build/bdist.macosx-10.7-intel/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/.../.python-eggs/MySQL_python-1.2.3-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so, 2): Library not loaded: libmysqlclient.18.dylib
Referenced from: /Users/.../.python-eggs/MySQL_python-1.2.3-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so
Reason: image not found
I tried adding this #!/usr/bin/python to the top of my script but the problem is still there.
I am working on Mac OS version 10.7
Edit: Crontab try to load mysql-python from current directory (which is my Home folder) lopen(/Users/.../.python-eggs/MySQL_python-1.2.3-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so, 2), while the library locate in /Library/Python/2.7/site-packages/MySQL_python-1.2.3-py2.7-macosx-10.7-intel.egg, i tried to use PYTHONPATH=/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7 in crontab but it's not working, any suggestion?
Thanks.

I solved this by using the following solution from : https://unix.stackexchange.com/questions/27289/how-can-i-run-a-cron-command-with-existing-environmental-variables
In my case it was because Cron didn't have access to my Environment variables that point to the correct place the Mysql library.
I used:
8 10 * * * . $HOME/.bash_profile; /Path/To/Script/pythonDbScript.py

I think you should check for your python version. On mac, it maybe installed several version of python. e.g. 2.5, 2.6, 2.7
When you run on terminal, you may using python2.6, but for the cron job, it my use the 2.7
use python -V to see which version of python are you using.
use which python to locate the python path
use shell autocomplete to see how many pythons did you install
when you type python, then press "tab", you will find the result.

From the Mysqldb faq: http://mysql-python.sourceforge.net/FAQ.html
Another thing that can cause this: The MySQL libraries may not be on your system path.
Solutions:
set the LD_LIBRARY_PATH environment variable so that it includes the path to the MySQL libraries.
set static=True in site.cfg for static linking
This probably means that in your crontab (remember minimal shell environment) the mysql libraries are not accessible (not on the path, not in the LD_LIBRARY_PATH)
check your environment variables between the 'regular' shell and the 'crontab' shell

try below:
sudo install_name_tool -change libmysqlclient.18.dylib /usr/local/mysql/lib/libmysqlclient.18.dylib