Develop Reference

Python import working on host but not inside a VM

Pythonproject directory structure is like
--test
--upperlevel
-- __init__.py
-- manager.py
-- UpperLevel.py
this files in turn contains
# __init__.py
msg = "YAYY printing !!!"
print msg
# UpperLevel.py
from upperlevel import msg
# manager.py
import UpperLevel
So in my local MAC book with python 2.7.10, started a python shell in test directory.
From that shell,
Python 2.7.10 (default, Jul 30 2016, 19:40:32)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import upperlevel.manager
YAYY printing !!!
>>>
it worked !!!!
However i started a virtual machine (ubuntu 14.04 and python 2.7.10) with vagrant and added same test directory to it.
so if i did the same thing
Python 2.7.10 (default, Jul 13 2017, 19:26:24)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import upperlevel.manager
YAYY printing !!!
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "upperlevel/manager.py", line 1, in <module>
import UpperLevel
File "upperlevel/UpperLevel.py", line 1, in <module>
from upperlevel import msg
File "upperlevel/upperlevel.py", line 1, in <module>
from upperlevel import msg
ImportError: cannot import name msg
>>>
So my questions are
1) why it is not working in the later case, i tried the same in docker and getting the same error
2) there is no such file in my project, File "upperlevel/upperlevel.py", line 1, in
3) why it is searching for upperlevel.py instead of UpperLevel.py
FYI
It looks like if we do "import upperlevel" from UpperLevel.py it is refering back to itself instead of going to upperlevel/init.py.
UPDATE:
I understood where the problem is from.... my test directory(volume) is being shared between mac and vagrant/docker, somehow UpperLevel.pyc is being treated as upperlevel.pyc in that shared volume.
Instead of running in a shared directory i created same folders/files in /home/vagrant and it worked.

It seems you are running from a Mac environment, and it is possible that the Python default search paths are different for those builds, despite the version being similar.
Try comparing:
import sys
print(sys.path)
It is probable that the default installation search paths might differ.
You can use the environment variable $PYTHONPATH to add additional import paths, while I don't really like this method it can be sufficient in most cases.
You can also setup your package in a proper module installation path.

Finally answering my own question...the problem is mac has a case insensitive file system and when it is mounted on linux, python is trying to use ubuntu mode of module reading like in the case sensitive way on a case insensitive File system.
After a lot of research found this link for docker https://github.com/docker/for-mac/issues/320 so those when using ubuntu docker with python on a mac be careful with your naming conventions.

ImportError: No module named package

I found importing modules in Python complicated, so I'm doing experiments to clear it up. Here is my file structure:
PythonTest/
package/
__init__.py
test.py
Content of __init__.py:
package = 'Variable package in __init__.py'
from package import test
Content of test.py:
from package import package
print package
When I stay out of package(in PythonTest), and execute python package/test.py, I get:
Traceback (most recent call last):
File "package/test.py", line 1, in <module>
from package import package
ImportError: No module named package
The expected output is Variable package in __init__.py. What am I doing wrong?
However, I can get the expected output in the interactive mode:
sunqingyaos-MacBook-Air:PythonTest sunqingyao$ python
Python 2.7.10 (default, Oct 23 2015, 19:19:21)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import package
Package in __init__.py

First Let's see how Python search for packages and modules. sys.path
A list of strings that specifies the search path for modules. Initialized from the environment variable PYTHONPATH, plus an installation-dependent default.
That's the search paths. Therefore, if your module/package is located in one of sys.path, python interpreter is able to find and import it. The doc says more:
As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first.
I modified test.py as an example.
import sys; import pprint
pprint.pprint(sys.path)
from package import package
print package
There are two cases:
$ python package/test.py
['/Users/laike9m/Dev/Python/TestPython/package',
'/usr/local/lib/python2.7/site-packages/doc2dash-2.1.0.dev0-py2.7.egg',
'/usr/local/lib/python2.7/site-packages/zope.interface-4.1.3-py2.7-macosx-10.10-x86_64.egg',
'/usr/local/lib/python2.7/site-packages/six-1.10.0-py2.7.egg',
'/usr/local/lib/python2.7/site-packages/colorama-0.3.3-py2.7.egg',
As you see, path[0] is /Users/laike9m/Dev/Python/TestPython/package, which is the directory containing the script test.py that was used to invoke the Python interpreter.
$ python
Python 2.7.12 (default, Jun 29 2016, 14:05:02)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import package
['',
'/usr/local/lib/python2.7/site-packages/doc2dash-2.1.0.dev0-py2.7.egg',
'/usr/local/lib/python2.7/site-packages/zope.interface-4.1.3-py2.7-macosx-10.10-x86_64.egg',
'/usr/local/lib/python2.7/site-packages/six-1.10.0-py2.7.egg',
'/usr/local/lib/python2.7/site-packages/colorama-0.3.3-py2.7.egg',
...
Now comes the second case, when invoked interactively, "path[0] is the empty string, which directs Python to search modules in the current directory first." What's the current directory? /Users/laike9m/Dev/Python/TestPython/.(look this is the path on my machine, it's equivalent to the path to PythonTest in your case)
Now you know the answers:
Why did python package/test.py give ImportError: No module named package?
Because the interpreter does not "see" the package. For the interpreter to be aware of package package, PythonTest has to be in sys.path, but it's not.
Why did this work in interactive mode?
Because now PythonTest is in sys.path, so the interpreter is able to locate package package.

Ubuntu VTK for Python compiled from source

I compiled VTK 7.0 (6.3 has the same effect) over cmake with following params:
-LIBRARY_OUTPUT_PATH:PATH="" -CMAKE_INSTALL_PREFIX:PATH="/usr/local" -VTK_ENABLE_VTKPYTHON:BOOL="1" -Module_vtkPython:BOOL="1" - -VTK_Group_Qt:BOOL="1" -CMAKE_OBJCOPY:FILEPATH="/usr/bin/objcopy" -VTK_RENDERING_BACKEND:STRING="OpenGL2" -VTK_INSTALL_PYTHON_MODULE_DIR:PATH="/usr/local/lib/python2.7/site-packages" -DVTK_EGL_DEVICE_INDEX:STRING="0" -VTK_WRAP_PYTHON:BOOL="1" -Module_vtkGUISupportQtOpenGL:BOOL="1"
Now i can find the binary "vtkpython" at /usr/local/bin .
Good news:
I am allowed to enter python shell with the command "vtkpython" out of this directory (/usr/local/bin) with all the needed vtk bindings.
markovich#markovich-desktop:~$ cd /usr/local/bin/
markovich#markovich-desktop:/usr/local/bin$ vtkpython
vtk version 7.0.0
Python 2.7.10 (default, Oct 14 2015, 16:09:02)
[GCC 5.2.1 20151010] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import vtk
>>> vtk
<module 'vtk' from '/usr/local/lib/python2.7/site-packages/vtk/__init__.py'>
>>>
Thats a bit irritating because i am expecting to run my default python environement and the vtk bindings are available.
so the bad news:
if I type python in my shell or vtkpython from another location on my system the shell says "no modulen named vtk found" on calling import vtk .
markovich#markovich-desktop:~$ vtkpython
vtk version 7.0.0
Python 2.7.10 (default, Oct 14 2015, 16:09:02)
[GCC 5.2.1 20151010] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import vtk
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named vtk
>>>
Question 1: Maybe I missed something in the make configuration ?
Question 2: If I take the actual status (which is somehow working): Is it possible integrate the "vtkpython" bindings in my default python environment? If I am not totally wrong. The binding is correctly loaded out of my python2.7 path like you can see in the terminal:
<module 'vtk' from '/usr/local/lib/python2.7/site-packages/vtk/__init__.py'>
So how can I add this module to be loaded in Python environment?

Since loading vtkpython clearly shows that you have the module available somewhere on your system, you should be able to add the location of the vtk module to your PYTHONPATH variable.
Find where the vtk module is installed (try /usr/local/lib/python2.7/site-packages, you should see a /vtk folder). If you're unsure, you could try to find it in vtkpython with
import vtk
import imp
imp.find_module('vtk')
You can check what paths are stored in PYTHONPATH by entering in terminal:
echo $PYTHONPATH
(In my install, it was empty by default.)
Then you can add the vtk folder location to your PYTHONPATH in terminal:
export PYTHONPATH=$PYTHONPATH:/usr/local/lib/python2.7/site-packages
Check if vtk is availabe:
$ python
>>> import vtk
If it works, you can add the export... line above into your ~/.bashrc or ~/.profile (as appropriate per your distro installation) to permanently load the option in PYTHONPATH.

Python path confusion: can import directory but not files?

I'm struggling with Python paths. I want to be able to import a file from a directory. I can do this when it's available from the filesystem, but not when I'm relying on my PATH (update: or PYTHONPATH) to find it.
Here are the symptoms. First of all, check that my current directory is on my PATH:
(.venv)~/Dropbox/gds/myapp $ pwd
/Users/me/Dropbox/gds/myapp
(.venv)~/Dropbox/gds/myapp $ echo $PATH
... :/usr/local/go/bin:.:/Users/me/Dropbox/gds/myapp
And check that test/test_app is where I think it is (and has an __init__.py in the directory:
(.venv)~/Dropbox/gds/myapp $ ls tests/
__init__.py __pycache__ test_app.py test_views.py
Now check that Python can import test/test_app from this directory:
(.venv)~/Dropbox/gds/myapp $ python
Python 3.4.2 (default, Jan 29 2015, 13:46:46)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tests
>>> import tests.test_app
>>> exit()
All good so far. Now move to a different directory, and see if I can import test/test_app:
(.venv)~/Dropbox/gds/myapp $ cd ..
(.venv)~/Dropbox/gds $ python
Python 3.4.2 (default, Jan 29 2015, 13:46:46)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tests
>>> import tests.test_app
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'tests.test_app'
But /Users/me/Dropbox/gds/myapp is on my PATH, and I can import the directory, so why can't I import this file?
UPDATE:
Thanks for the suggestions. Just tried doing export PYTHONPATH=$PYTHONPATH:/Users/me/Dropbox/gds/myapp: but still see the same problem when I run Python.

The directory needs to be in PYTHONPATH, not PATH (which is for regular non-Python command lookup).

There is probably another tests Python package in sys.path. Check tests.__file__ to confirm. To fix it, use more specific names e.g., myapp.tests instead i.e., move tests package inside myapp Python package.

Python doesn't use PATH. It uses the PYTHONPATH variable to look up modules:
Augment the default search path for module files. The format is the same as the shell’s PATH: one or more directory pathnames separated by os.pathsep (e.g. colons on Unix or semicolons on Windows).
PATH is a variable used by your shell to look up binaries; only the format (how paths are specified and separated) is shared.

Only able to import python module inside the installation directory

I installed 64bit package of cefpython in Ubuntu 12.04 LTS (http://code.google.com/p/cefpython/). The problem is I am not able to run the examples. It says no module named wx.
But when I navigate to the directory /usr/local/lib/python2.7/dist-packages/cefpython1 and do import wx it works. So basically I am not able to import wx outside that directory. I am using python interpreter on terminal.
rishi:cefpython1 ls
cefclient cefpython_py27.pyc chrome.pak examples __init__.pyc LICENSE.txt wx
cefpython_py27.py cefpython_py27.so devtools_resources.pak __init__.py libcef.so locales
rishi:cefpython1 python
Python 2.7.3 (default, Sep 26 2013, 20:03:06)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import wx
>>>
[2]+ Stopped python
rishi:cefpython1 cd ..
rishi:dist-packages python
Python 2.7.3 (default, Sep 26 2013, 20:03:06)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import wx
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named wx
>>>
KeyboardInterrupt
>>>
My PATH is as follows and PYTHONPATH is empty:
installed 64bit package of cefpython in Ubuntu 12.04 LTS (http://code.google.com/p/cefpython/). The problem is I am not able to run the examples. It says no module named wx. But when I navigate to the directory /usr/local/lib/python2.7/dist-packages/cefpython1 and do import wx it works. So basically I am not able to import wx outside that directory. I am using python interpreter on terminal.
rishi:dist-packages echo $PATH
/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
rishi:dist-packages echo $PYTHONPATH
rishi:dist-packages echo $PYTHONPATH
rishi:dist-packages

Your problem here is that python has no idea where you are importing from. Refer to This post for instructions. Basically, what is happening is that when you're in the directory, python knows to look for it (it looks for python files and packages in the directory). You need to add the python libraries into the PYTHONPATH.