Develop Reference

Error with TensorFlow MNIST [duplicate]

Since updating from Homebrew Python 2.7.11 (from 2.7.10) I'm suddenly unable to test register my package on PyPi from the PyCharm IDE console.
Running (as an "External Tool")
python -B setup.py register -r pypitest
I now get
Traceback (most recent call last):
File "setup.py", line 22, in <module>
from setuptools import setup
File "/usr/local/lib/python2.7/site-packages/setuptools/__init__.py", line 12, in <module>
from setuptools.extension import Extension
File "/usr/local/lib/python2.7/site-packages/setuptools/extension.py", line 8, in <module>
from .dist import _get_unpatched
File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 16, in <module>
from setuptools.depends import Require
File "/usr/local/lib/python2.7/site-packages/setuptools/depends.py", line 6, in <module>
from setuptools import compat
File "/usr/local/lib/python2.7/site-packages/setuptools/compat.py", line 17, in <module>
import httplib
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 80, in <module>
import mimetools
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/mimetools.py", line 6, in <module>
import tempfile
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tempfile.py", line 32, in <module>
import io as _io
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/io.py", line 51, in <module>
import _io
ImportError: dlopen(/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so, 2): Symbol not found: __PyCodecInfo_GetIncrementalDecoder
Referenced from: /usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so
Expected in: flat namespace
in /usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so
Process finished with exit code 1
I'm not sure how to proceed. I only get this issue if I execute from within my IDE's console. If I do it directly at the system command line (Terminal on OS X) I have no problems.
OS X 10.11.3; Homebrew Python 2.7.11; PyCharm 5.0.3

tl;dr: Fix this issue by doing one of the following:
type hash -r python, OR
log out and log in.
EDIT: An answer to my related question makes it clear what's happening here. When you install a new version of python, you may need to run hash -r python to tell bash to reset the "cached" location to the python executable.
In my case, I was typing python, which was on my $PATH at /usr/local/bin/python. But bash was still using the old cache location /usr/bin/python. So, the old executable was called, but the new path was provided to python in sys.argv[0]. This means that the old executable was running, but the new sys.executable value caused all the wrong modules to get loaded (including the io module).
I'm having the same problem. I installed python 2.7.11 via an installer from Python.org. Strangely, the issue seems to be related to some subtle difference between how OSX launches python when I invoke it from the shell using the full path vs. using just the word python.
So, for me, this works (invoking python via the full path /usr/local/bin/python):
$ which python
/usr/local/bin/python
$ /usr/local/bin/python -c "import io"
$
... but this doesn't:
$ python -c "import io"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/io.py", line 51, in <module>
import _io
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so, 2): Symbol not found: __PyCodecInfo_GetIncrementalDecoder
Referenced from: /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so
Expected in: flat namespace
in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so
So, as a workaround, you can try doing the same thing.
Elsewhere, I've posted a separate question about this puzzling behavior. Maybe somehow merely calling python invokes some strange mix of the 2.7.11 executable with the 2.7.10 dylibs??

According to https://github.com/klen/python-mode/issues/634:
I had the same issue, but successfully fixed. In my case I compiled
python and vim with homebrew, when PYTHON_PATH has been specified and
set to one of my dev environments, where I also had some libraries,
including io. Workaround was simple: open new terminal, make sure that
you do not have custom PYTHON_PATH, uninstall python, uninstall vim.
Reinstall both of them.
and
Problem solved.
Culprit is the update from python 2.7.10 to 2.7.11.
If you are using conda package control, simply run "conda install
python=2.7.10" will solve this problem.
This doesn't give the root cause though. Since this happens with _io, this looks like a bug in python 2.7.11 (unlikely, there would be a world-scale outcry and a prompt fix if it was) or some packaging bug or version mismatch specifically with the homebrew version (and maybe some related ones, too).
Try to import _io in the console and if it succeeds, check if it was loaded from the same path.

Reinstall python.
brew unlink python && brew reinstall python
Secure the path
export PYTHONPATH=$PYTHONPATH:/usr/local/bin/
BACKUP and Change the order of "paths" file.
sudo nano /etc/paths
it seems, the order of paths, it is decisive to run python properly. In my case, the result was:
#sudo nano /etc/paths
/usr/bin
/usr/local/bin
/bin
/usr/sbin
/sbin
On my mac, path is like this.
$ which python
/usr/local/bin/python
Now I can run both:
$ /usr/local/bin/python -c "import io"
$ python -c "import io"

I had the same issue, it is successfully fixed by just replacing the _io.so file.
sudo find / -name _io.so
copy the path of the _io.so file which DOES NOT belong to python-2.7.11. For example, copy the path of _io.so which is under python-2.7.5:
/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so
Replace the /usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so file with the _io.so that you just found.

This happened to me as well in MacVim. I solved it by making sure :python print(sys.path) is using system Python (e.g. /Library/Python/2.7/...)
Since I installed MacVim via Homebrew, I just did that by:
Spawn a new shell that had which python -> /usr/bin/python. For my case I needed to remove the pyenv line from my .bash_profile. If you installed Python via Homebrew you may want to brew unlink python first
brew reinstall macvim

If your problem is caused by anaconda, it is unnecessary to remove //anaconda directory.
Just open your ~/.bash_profile, find the line
export PATH="//anaconda/bin:$PATH
and comment it out, then restart your terminal session.

Another quick workaround if you don't mind sticking with Python 2.7.10 is to specify the path of the Python interpreter executable that will be used for the virtualenv. On OSX that path is usually /usr/bin/python:
virtualenv venv --python=/usr/bin/python

Can't add comment (?) so this just to share my exp., downgrade to 2.7.10 works fr me.

I got this error after a failed NLTK download, I needed to uninstall anaconda:
sudo rm -rf ~/anaconda
update PATH variable

This happened when I already had tried to create a venv in a folder, and mistakenly was trying to initialize a second one! So I just removed venv directory and re-ran the command. Very likely this is not the answer to this solution, but searching my error brought me here, so it may help some others who are stuck.

I solved this issue by removing the symbolic link that was in /usr/local/bin and copying the actual python binary, that was pointed to by said link, there.

I had the same issue when I tried to use PyCharm. Solved by setting "python interpreter" in project configuration to point to the python virtual env I wanted to use, which was an Anaconda env. Somehow the interpreter path was missing the "anaconda" portion of ~/.../anaconda/.../_io.so. No need to uninstall anaconda.

pdfminer - ImportError: No module named pdfminer.pdfdocument

I am trying to install pdfMiner to work with CollectiveAccess. My host (pair.com) has given me the following information to help in this quest:
When compiling, it will likely be necessary to instruct the
installation to use your account space above, and not try to install
into the operating system directories. Typically, using "--
home=/usr/home/username/pdfminer" at the end of the install command
should allow for that.
I followed this instruction when trying to install.
The result was:
running install
running build
running build_py
running build_scripts
running install_lib
running install_scripts
changing mode of /usr/home/username/pdfminer/bin/latin2ascii.py to 755
changing mode of /usr/home/username/pdfminer/bin/pdf2txt.py to 755
changing mode of /usr/home/username/pdfminer/bin/dumppdf.py to 755
running install_egg_info
Removing /usr/home/username/pdfminer/lib/python/pdfminer-20140328.egg-info
Writing /usr/home/username/pdfminer/lib/python/pdfminer-20140328.egg-info
I don't see anything wrong with that (I'm very new to python), but when I try to run the sample command $ pdf2txt.py samples/simple1.pdf I get this error:
Traceback (most recent call last): File "pdf2txt.py", line 3, in <module>
from pdfminer.pdfdocument import PDFDocument ImportError: No module named pdfminer.pdfdocument
I'm running python 2.7.3. I can't install from root (shared hosting). The most recent version of pdfminer, which is 2014/03/28.
I've seen some posts on similar issues ("no module named. . . " but nothing exactly the same. The proposed solutions either don't help (such as installing with sudo - not an option; specifying the path for python (which doesn't seem to be the issue), etc.).
Or is this a question for my host? (i.e., something amiss or different about their setup)

I had an error like this:
No module named 'pdfminer.pdfinterp'; 'pdfminer' is not a package
My problem was that I had named my script pdfminer.py which for the reasons that I don't know, Python took it for the original pdfminer package files and tried to compiled it.
I renamed my script to something else, deleted all the *.pyc file and __pycache__ directory and my problem was solved.

use this command worked for me and removed the error
pip install pdfminer.six

Since the package pdfminer is installed to a non-standard/non-default location, Python won't be be able to find it. In order to use it, you will need to add it to your 'pythonpath'. Three ways:
At run time, put this in your script pdf2txt.py:
import sys
# if there are no conflicting packages in the default Python Libs =>
sys.path.append("/usr/home/username/pdfminer")
or
import sys
# to always use your package lib before the system's =>
sys.path.insert(1, "/usr/home/username/pdfminer")
Note: The install path specified with --home is used as the Lib for all packages which you might want to install, not just this one. You should delete that folder and re-install with --
home=/usr/home/username/myPyLibs (or any generic name) so that when you install other packages with that install path, you would only need the one path to add to your local Lib to be able to import them:
import sys
sys.path.insert(1, "/usr/home/username/myPyLibs")
Add it to PYTHONPATH before executing your script:
export PYTHONPATH="${PYTHONPATH}:/usr/home/username/myPyLibs"
And then put that in your ~/.bashrc file (/usr/home/username/.bashrc) or .profile as applicable. This may not work for programs which are not executed from the console.
Create a VirtualEnv and install the packages you need to that.

I have a virtual environment and I had to activate it before I did a pip3 install to have the venv see it.
source ~/venv/bin/activate

Unable to import git in python

I am facing these issues. Can you help me with the same ?
Why am I seeing this error ? Do I have to add anything in the requirements.txt file ?
>>> import git
Traceback (most recent call last):
File "<pyshell#7>", line 1, in <module>
import git
File "git\__init__.py", line 29, in <module>
_init_externals()
File "git\__init__.py", line 23, in _init_externals
raise ImportError("'gitdb' could not be found in your PYTHONPATH")
ImportError: 'gitdb' could not be found in your PYTHONPATH
>>> from git import Repo
Traceback (most recent call last):
File "<pyshell#8>", line 1, in <module>
from git import Repo
File "git\__init__.py", line 29, in <module>
_init_externals()
File "git\__init__.py", line 23, in _init_externals
raise ImportError("'gitdb' could not be found in your PYTHONPATH")
ImportError: 'gitdb' could not be found in your PYTHONPATH

I already had gitdb and smmap installed so I had to reinstall them.
You can reinstall them by running the following command in your terminal:
pip3 install --upgrade --force-reinstall gitdb; pip3 install --upgrade --force-reinstall smmap

I also got the message ImportError: 'gitdb' could not be found in your PYTHONPATH (when trying to use GitPython).BUT I had gitdb already installed!
Thanks to this hint I figured out that gitdb silently failed because it was missing smmap.
So I installed this and it worked.

You need to install gitdb package.
$ sudo easy_install gitdb

I had the same problem. However, gitdb and smmap were already installed by pip. As I used brew to install python and its dependencies on my mac, when I checked brew doctor command, it said that my /usr/local/sbin directory is not in my PATH. So I added it to my PATH (though it didn't have anything to do with the python) and everything worked out eventually.

MS Windows Versions of this problem can occur because of the order of Python versions in your system PATH, as it did for me. I did not realize that when I installed another program, it installed a newer version of Python for its own usage, and it appended my system PATH with the address to the newer version. I noticed it when I looked at the PATH variable and found two versions of Python being called. Windows uses the first it finds, and if the first doesn't match what your program expects, it gets confused and can't find the right path to the module. This is what I did to resolve it:
To check: an easy way to test if this is your problem is to see if the paths separated by semicolons are in the right order. That can be seen in the System Variables of Windows or by printing your PATH variable in your CMD shell like in this example:
C:> path
PATH=C:\Program Files (x86)\Python37-32\Scripts;C:\Program Files (x86)\Python37-32;C:\Program Files\Python38\Scripts;C:\WINDOWS
Temporary solution:
To see if it is going to fix your computer, change it in your CMD window. Your variable change will be discarded when the window is closed. One way to do this test is to copy the paths, move the Python references to the order they are needed, and write it back:
C:> set path = C:\WINDOWS;C:\Program Files (x86)\Python37-32;C:\Program Files\Python38\Scripts;C:\Program Files (x86)\Python37-32\Scripts\
Then run the Python program to see if this was your problem. Note that this is only an example; do not copy & paste it. Your path is customized for the programs on your computer.
Permanent solution: If the above test resolves your problem, you must change your System Variables to make the change permanent. For me that usually requires a reboot afterwards in order to make the variables appear in all new windows.

virtualenv and python path

i did a virtualenv in c:\users\devtool.virtualenv\devenv
After i run activate.bat.
And last i wrote a simple py file to test environment:
import os,sys
for x in sys.path:
print x
print os.executable
and result is weird
C:\Users\devtool\.virtualenv\devenv\Scripts
C:\Python27\lib\site-packages\setuptools-0.6c11-py2.7.egg
C:\Python27\lib\site-packages\virtualenv-1.8.4-py2.7.egg
C:\Python27\lib\site-packages\pip-1.2.1-py2.7.egg
C:\Users\developer\.virtualenv\devenv\Lib\site-packages\django
C:\Windows\system32\python27.zip
C:\Python27\DLLs
C:\Python27\lib
C:\Python27\lib\plat-win
C:\Python27\lib\lib-tk
C:\Python27
C:\Python27\lib\site-packages
['C:\\Users\\developer\\.virtualenv\\devenv\\Lib\\site-packages\\django']
C:\Python27\python.exe
in fact it is just using base python install.
What is the point of using virtualenv?
i am getting errors with django, installed in virtualenv.
i can solve it, just need to rewrite django-admin.py to append to search path my virtualenv folder, but still what is need of virtualenv in this case.
another thing that i cant understand. In the python search path, there is a line
C:\Users\devtools\.virtualenv\devenv\Lib\site-packages\django
and when i use django-admin.py i get
Traceback (most recent call last):
File "C:\Users\devtools\.virtualenv\devenv\Scripts\django-admin.py", line 12, in module>
from django.core import management
ImportError: No module named django.core
but django folder is in the path

I have also faced the same issue few days before. From the base folder, you have to execute the following command.
C:\Users\devtools\.virtualenv\devenv> python Scripts\django-admin.py startproject myproj

virtualenv extends/overrides your system python environment with its paths prepended to the paths of the system python installation. You see, you .virtualenv site-packages are listed before the system site-packages, that's how it works.
The thing you have to keep in mind that activate patches your current command line environment, so you must run activate before running python code depending on your virtualenv.

Workaround read-only site-packages for cx_freeze

I'm currently trying to make cx_freeze to work on a Solaris workstation I have to work with, in order to make an executable from a Python script I have. Problem is, I'm not administrator of this machine, and installation of cx_freeze requests write to site-packages, which is read-only for me.
So, obviously, I get this error:
creating /usr/local/lib/python2.6/site-packages/cx_Freeze
error: could not create '/usr/local/lib/python2.6/site-packages/cx_Freeze': Read-only file system
And if I try to run it anyway, it fails:
bash-3.00$ python /home/xxxx/cx_freeze-4.2.3/cxfreeze --target-dir cx_dist src/p_tool.py
Traceback (most recent call last):
File "/home/xxxx/cx_freeze-4.2.3/cxfreeze", line 5, in <module>
main()
File "/home/xxxx/cx_freeze-4.2.3/cx_Freeze/main.py", line 187, in main
silent = options.silent)
File "/home/xxxx/cx_freeze-4.2.3/cx_Freeze/freezer.py", line 91, in __init__
self._VerifyConfiguration()
File "/home/xxxx/cx_freeze-4.2.3/cx_Freeze/freezer.py", line 371, in _VerifyConfiguration
self._GetInitScriptFileName()
File "/home/xxxx/cx_freeze-4.2.3/cx_Freeze/freezer.py", line 283, in _GetInitScriptFileName
raise ConfigError("no initscript named %s", name)
cx_Freeze.freezer.ConfigError: no initscript named Console
Obviously, this is linked to the failed installation. So, here's my question:
Without installation of virtualenv, could I avoid the writing to site-packages, and make cx_freeze to execute from my home folder?
EDIT I had a look at site.py documentation, and PYTHONPATH filling should be equivalent to use of site-packages. So my question is now more something like: what is the path to be added to PYTHONPATH, so that cx_freeze could be executed from any location?
Notes:
I would like to avoid to deal with virtualenv, as I'm already struggling to understand the executable tools...
I saw this question, but this still requires access to site-packages folder, plus it's not user-specific;
I tried adding the following path to PYTHONPATH, but this does not work: /home/xxxx/cx_freeze-4.2.3/build/lib.solaris-2.10-sun4v-2.6;
I'm also trying to use PyInstaller but have dependency problems (and the administrator is not really helping me).

This works like a charm for me :
$ python setup.py install --home=$HOME
Run in the source directory of cx_freeze found on the Sourceforge download page.