Python import working on host but not inside a VM - python

Pythonproject directory structure is like
--test
--upperlevel
-- __init__.py
-- manager.py
-- UpperLevel.py
this files in turn contains
# __init__.py
msg = "YAYY printing !!!"
print msg
# UpperLevel.py
from upperlevel import msg
# manager.py
import UpperLevel
So in my local MAC book with python 2.7.10, started a python shell in test directory.
From that shell,
Python 2.7.10 (default, Jul 30 2016, 19:40:32)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import upperlevel.manager
YAYY printing !!!
>>>
it worked !!!!
However i started a virtual machine (ubuntu 14.04 and python 2.7.10) with vagrant and added same test directory to it.
so if i did the same thing
Python 2.7.10 (default, Jul 13 2017, 19:26:24)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import upperlevel.manager
YAYY printing !!!
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "upperlevel/manager.py", line 1, in <module>
import UpperLevel
File "upperlevel/UpperLevel.py", line 1, in <module>
from upperlevel import msg
File "upperlevel/upperlevel.py", line 1, in <module>
from upperlevel import msg
ImportError: cannot import name msg
>>>
So my questions are
1) why it is not working in the later case, i tried the same in docker and getting the same error
2) there is no such file in my project, File "upperlevel/upperlevel.py", line 1, in
3) why it is searching for upperlevel.py instead of UpperLevel.py
FYI
It looks like if we do "import upperlevel" from UpperLevel.py it is refering back to itself instead of going to upperlevel/init.py.
UPDATE:
I understood where the problem is from.... my test directory(volume) is being shared between mac and vagrant/docker, somehow UpperLevel.pyc is being treated as upperlevel.pyc in that shared volume.
Instead of running in a shared directory i created same folders/files in /home/vagrant and it worked.


It seems you are running from a Mac environment, and it is possible that the Python default search paths are different for those builds, despite the version being similar.
Try comparing:
import sys
print(sys.path)
It is probable that the default installation search paths might differ.
You can use the environment variable $PYTHONPATH to add additional import paths, while I don't really like this method it can be sufficient in most cases.
You can also setup your package in a proper module installation path.


Finally answering my own question...the problem is mac has a case insensitive file system and when it is mounted on linux, python is trying to use ubuntu mode of module reading like in the case sensitive way on a case insensitive File system.
After a lot of research found this link for docker https://github.com/docker/for-mac/issues/320 so those when using ubuntu docker with python on a mac be careful with your naming conventions.

Related

Python on Gentoo is missing sqlite

I use a Gentoo-based Docker image for CI with multiple versions of Python. Recently, I've started experiencing errors because one tool (coveralls) requires sqlite, which is missing. sqlite is part of the Python standard library.
This can be checked from the command line
>>> removing all .pyc files
>>> executing command
me#5b35f99c08af /source $ python
Python 3.6.9 (default, Dec 27 2019, 12:15:49)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'sqlite3'
I couldn't find any notes in the Gentoo packaging database about this, but I'm not really familiar with it as an OS. I assume there must have been a problem building some relevant library.
Python has been installed like this:
RUN emerge -q -u dev-lang/python:3.6
But the error occurs for all the versions I'm currently testing with: >= 3.5. Any ideas as to what I'm doing wrong?

Dropbox API not working on python script run but is working on interactive mode

A simple check if the Dropbox API works, I have below dropbox.py script created
import dropbox
dbx = dropbox.Dropbox('MY_TOKEN')
dbx.users_get_current_account()
Running it in normal script mode using terminal, I have to use below command.
username$ python3 dropbox.py
This returns below error:
Traceback (most recent call last):
File "dropbox.py", line 1, in <module>
import dropbox
It works okay when using the interactive mode with below command
username$ python3
Python 3.6.4 (default, Jan 6 2018, 11:51:15)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dropbox
>>> dbx = dropbox.Dropbox('MY_TOKEN')
>>> dbx.users_get_current_account()
FullAccount displayed here successfully
Why is there a difference in interactive versus script mode? How to get the script mode working?

Probably, the issue is name clutch between your file dropbox.py and module.
When running dropbox.Dropbox, python tries to create instance of Dropbox class from your file (which is treated as module too), and you have no such.
The motivation behind this order of imports is ability to "override" pre-installed modules with your own.
TL;DR: renaming your file should help.

Can't find the test module in Anaconda Python

I cannot find the test module in my Anaconda's version of Python. Can anyone help me fix this. This module is used by the dpkt library that I am trying to use.
Python 2.7.8 |Anaconda 2.1.0 (x86_64)| (default, Aug 21 2014, 15:21:46)
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
>>> import test
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named test

Quick Fix: You can checkout a copy of pystone.py from the cpython github repository and copy it to a test directory that is present in your PYTHONPATH. Or you would copy it to a test directory in your python project's root (ugly way).
Perhaps Anaconda Python does not ship with a copy of the test module. This is a standard part of Python 2.7. Other times, users accidentally overwrite their Python standard library's test module with something else. You can try to use the Python version that is shipped with OS X instead. If that fails as well, then try to see which test module is being loaded, and go from there.
import test
print test

Alpine Linux ships a python2-tests package.
the quickest way to fetch it if you don't have an Alpine lxc container is from a main repo here (or apk fetch python2-tests inside lxc).
the .apk can be uncompressed with an archiver to a .tar.gz & then just uncompress again.

Python turtle graphics malfunction on a Mac running OSX 10.6

This is the result when I attempt to run Python turtle graphics on my Mac.
Has anyone else seen this. Are there suggestions for fixing this problem. Thanks
in advance!
$ python
Python 2.6.1 (r261:67515, Aug 2 2010, 20:10:18)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import turtle
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-tk/turtle.py", line 114, in <module>
from copy import deepcopy
File "/Users/morrison/copy.py", line 3, in <module>
Interface summary:
IndexError: list index out of range
>>>

Is your current directory /Users/morrison/ when you run this?
If so, the problem is that the interpreter's current working directory is being used for looking up python modules at runtime in addition to the standard locations. You have a file copy.py in this directory and it is being imported when the standard library copy module is what was intended by the turtle module.
This happens because when you run the python interpreter interactively, it will add the current working directory to the front of sys.path automatically. (There's a likewise effect if you were trying to run a script in this dir from a different directory - the script's directory would be added to the front of sys.path.)
The easiest way around this is to just rename copy.py to mycopy.py (and don't forget to remove copy.pyc in that dir).

OSX Port Python Library Path? Cant find ctypes

I am trying to use an application that has a dependency of ctypes, but am getting this error:
$ python peach.py -t ~/Desktop/fuzz/wav/template.xml
] Peach 2.3.6 Runtime
] Copyright (c) Michael Eddington
Traceback (most recent call last):
File "peach.py", line 335, in <module>
from Peach.Engine import *
File "/opt/Peach-2.3.6/Peach/__init__.py", line 40, in <module>
import Publishers, Transformers
File "/opt/Peach-2.3.6/Peach/Publishers/__init__.py", line 37, in <module>
import file, sql, stdout, tcp, udp, com, process, http, icmp, raw, remote, dll, smtp
File "/opt/Peach-2.3.6/Peach/Publishers/file.py", line 37, in <module>
from Peach.Engine.engine import Engine
File "/opt/Peach-2.3.6/Peach/Engine/engine.py", line 835, in <module>
from Peach.Engine.state import StateEngine
File "/opt/Peach-2.3.6/Peach/Engine/state.py", line 38, in <module>
import sys, re, types, time, struct, ctypes
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/ctypes/__init__.py", line 10, in <module>
from _ctypes import Union, Structure, Array
ImportError: No module named _ctypes
I have installed py-ctypes from ports, but it seems to only be a Python 2.4 version:
$ port contents py-ctypes
Port py-ctypes contains:
/opt/local/lib/python2.4/site-packages/_ctypes.so
/opt/local/lib/python2.4/site-packages/_ctypes_test.so
/opt/local/lib/python2.4/site-packages/ctypes/__init__.py
/opt/local/lib/python2.4/site-packages/ctypes/__init__.pyc
/opt/local/lib/python2.4/site-packages/ctypes/_endian.py
/opt/local/lib/python2.4/site-packages/ctypes/_endian.pyc
/opt/local/lib/python2.4/site-packages/ctypes/macholib/__init__.py
/opt/local/lib/python2.4/site-packages/ctypes/macholib/__init__.pyc
/opt/local/lib/python2.4/site-packages/ctypes/macholib/dyld.py
/opt/local/lib/python2.4/site-packages/ctypes/macholib/dyld.pyc
/opt/local/lib/python2.4/site-packages/ctypes/macholib/dylib.py
/opt/local/lib/python2.4/site-packages/ctypes/macholib/dylib.pyc
I then tried to run the application via python2.4, but it seems the application uses syntax that is only available in 2.5:
$ python2.4 peach.py -t ~/Desktop/fuzz/wav/template.xml
File "peach.py", line 498
finally:
^
SyntaxError: invalid syntax
My python install is also from OSX ports, and I noticed in the Peach application, it defines python as:
#!/usr/bin/python
Will that mess with anything if my default python executable points to my port installation (and I am running 'python peach.py')?
$ which python
/opt/local/bin/python
Is there any work around for this?
ctypes for python2.5?
Ability to add 2.4 libraries to 2.5 path?

A simple solution would be to use the native Python build that is included with Mac OS. This definitely works with the latest release of Mac OS X 10.6.4, which has Python 2.6.
Here is an example showing that '_ctypes' is being imported successfully:
mariah:~ joet3ch$ /usr/bin/python
Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from _ctypes import Union, Structure, Array
>>>
If you have issues after this, try looking at the sys.path attribute to see which modules and versions are in your path.
Here is an example of viewing the contents of sys.path on a fresh Mac OS 10.6.4 build:
mariah:~ joet3ch$ /usr/bin/python
Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python26.zip', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-old', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload', '/Library/Python/2.6/site-packages', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjC', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/wx-2.8-mac-unicode']
>>>

ctypes is a standard Python library since version 2.5, so py-ctypes is not needed. The line at which you get an ImportError still exists in my 2.6.5 installation.
I do not own OSX, so my question is: does /opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5 belong to the standard installation of Python 2.5, or may this belong to a possibly broken installation of some framework?
In a running python shell, you could inspect the value of sys.path. Maybe there is some broken library that precedes the standard library.
The shebang line #!/usr/bin/python is interpreted by the OS if it is the first line of an executable script that is invoked directly like an ordinary program. In all other cases, this is just a comment. In particular, the line is ignored if you invoke the script as in python myscript.py or if it is imported by other Python code.

Categories

Resources