I have a Python program originally built on and for Linux, which I'm now trying to port over to Windows. I am running the program in a virtual environment which contains all of the dependencies (my program is installed as a wheel with pip install --find-links wheels my_module). The program is launched with
(venv) C:\>venv\Scripts\python.exe -m base_module.Launcher arg1 arg2
The base_module loads my module as interpreted by the arguments provided, and his relevant code is:
from multiprocessing.managers import SyncManager
import OtherCustomClass
class BaseModule(object):
def __init__(self, arg1, arg2):
self.manager = SyncManager()
self.manager.start(ignore_interrupt)
def main(argv=None):
ret = -1
try:
basmod = BaseModule(argv[0], argv[1])
ret = basmod.run()
except Exception, err:
print("error: " + str(err))
print(traceback.format_exc())
return ret
if __name__ == "__main__":
exitCode = main(sys.argv[1:])
sys.exit(exitCode)
This has worked fine in Linux, but on Windows I get the following exception:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Python27\Lib\multiprocessing\forking.py", line 380, in main
prepare(preparation_data)
File "C:\Python27\Lib\multiprocessing\forking.py", line 505, in prepare
'__parents_main__', file, path_name, etc
File "build/bdist.linux-x86_64/egg/base_module/BaseModule.py", line 2, in <module>
ImportError: No module named OtherCustomClass
exception in main:
Traceback (most recent call last):
File "build/bdist.linux-x86_64/egg/base_module/BaseModule.py", line 12, in main
File "build/bdist.linux-x86_64/egg/base_module/BaseModule.py", line 7, in __init__
File "C:\Python27\Lib\multiprocessing\managers.py", line 528, in start
self._address = reader.recv()
EOFError
The latter EOFError is caused by the unexpected early termination from the forking in SyncManager, where the true error is being unable to import OtherCustomClass. I have confirmed that OtherCustomClass exists in the base_module's folder within venv/lib/site-packages, and this error isn't happening when I launch the module first as Python would never reach the instructions in main() or init if the script wouldn't compile.
I've done some research, and I know this problem has hit other people (often using third party libraries, who fixed the issue without posting the solution). It seems to trace back to Windows' lack of a fork(), and python's handling of multiprocessing on Windows - see also http://docs.python.org/library/multiprocessing.html#windows. But I'm at a loss as to how to fix this.
This is the latest Python 2.7 branch (2.7.8), running on Windows 7 x64.
You can work around this by using an absolute import for OtherCustomClass:
from base_module import OtherCustomClass
I'm not exactly sure why, but it seems that when multiprocessing spawns a new process and imports your __main__, it's not able to handle the implicit relative import you're using with OtherCustomClass. If you explicitly import it from base_module, it works fine. My guess is that the spawned child process is not recognized as being part of the base_module package, so the implicit import fails, but that's just a guess.
Note that you shouldn't be using implicit relative imports anyway (they're altogether removed from Python 3), so switching to an absolute import isn't a bad thing.
Also of note, that doing an explicit relative import works on Python 3.4:
from . import OtherCustomClass
But it fails on Python 2.7:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\python27\lib\multiprocessing\forking.py", line 380, in main
prepare(preparation_data)
File "C:\python27\lib\multiprocessing\forking.py", line 495, in prepare
'__parents_main__', file, path_name, etc
File "C:\Users\oreild1\Desktop\base_module\Launcher.py", line 5, in <module>
from . import OtherCustomClass
ValueError: Attempted relative import in non-package
error:
Traceback (most recent call last):
File "C:\Users\oreild1\Desktop\base_module\Launcher.py", line 18, in main
basmod = BaseModule(argv[0], argv[1])
File "C:\Users\oreild1\Desktop\base_module\Launcher.py", line 10, in __init__
self.manager.start()
File "C:\python27\lib\multiprocessing\managers.py", line 528, in start
self._address = reader.recv()
EOFError
Related
There was a problem when starting a microservice written in the Nameko framework.
When starting the microservice via the command
nameko run name_file_with_class_microservice
The following exception is thrown:
Traceback (most recent call last):
File "/home/user/.pyenv/versions/project/bin/nameko", line 33, in <module>
sys.exit(load_entry_point('nameko==2.12.0', 'console_scripts', 'nameko')())
File "/home/user/.pyenv/versions/project/lib/python3.9/site-packages/nameko/cli/main.py", line 112, in main
args.main(args)
File "/home/user/.pyenv/versions/project/lib/python3.9/site-packages/nameko/cli/commands.py", line 110, in main
main(args)
File "/home/user/.pyenv/versions/project/lib/python3.9/site-packages/nameko/cli/run.py", line 181, in main
import_service(path)
File "/home/user/.pyenv/versions/project/lib/python3.9/site-packages/nameko/cli/run.py", line 46, in import_service
__import__(module_name)
File "./main.py", line 18, in <module>
from commands.object import ObjectUpdateCommand, ObjectExportCommand
File "./commands/object/__init__.py", line 1, in <module>
from .object_export import ObjectExportCommand
File "./commands/object/object_export.py", line 6, in <module>
from ...exception import ProxyException
ImportError: attempted relative import beyond top-level package
I understand this has to do with the "sys.path" variable and where the launch is coming from. However, I do not understand how to make imports workable, both for the entire project as a whole, and for a separate microservice in it (yes, microservices are combined into one repository, for each of which its own docker-file is written)
Everything works if you specify an absolute path relative to the starting point of the microservice. However, this is not suitable for the whole project, PyCharm indicates import curves, autocompletion does not work, and so on.
I am trying to setup PyGears, I have installed the package using pip. I am trying to execute the bellow two lines of code using command line.
from pygears import gear, Intf
from pygears.typing import Uint
This is the error I received:
Traceback (most recent call last):
File "test.py", line 1, in <module>
from pygears import gear, Intf
File "C:\Users\Nikola\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pygears\__init__.py", line 15, in <module>
import pygears.typing
File "C:\Users\Nikola\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pygears\typing\__init__.py", line 1, in <module>
from pygears.conf import PluginBase, reg
File "C:\Users\Nikola\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pygears\conf\__init__.py", line 5, in <module>
from .trace import pygears_excepthook, register_issue, MultiAlternativeError
File "C:\Users\Nikola\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pygears\conf\trace.py", line 400, in <module>
elif PYPY:
NameError: name 'PYPY' is not defined
To my understanding, PyPy is an alternative interpreter to CPython, but I am unsure whether the symbol mentioned above is related to this. Should I try configuring PyPy as my interpreter or is this issue caused by something else?
Thank you in advance
Update: this has been fixed by https://github.com/bogdanvuk/pygears/commit/a27df7ce62cfe568978fad1f39d87372b5b9a531
This looks like a bug in pygears. If you try with python >= 3.7, you should not run into this error.
I submitted a bug report: https://github.com/bogdanvuk/pygears/issues/8
I am currently trying to install google cloud sdk on my macbook. I am using scikit-learn and anaconda for machine learning so on my laptop python points to python3.5. However google cloud sdk requires python 2.7 for installation, and that's why I am facing the problem of not be able to install it.
when I ran the install.sh file the error occurred and I have no clue how to deal with it:
Adam | ~ $ git/google-cloud-sdk/install.sh
Welcome to the Google Cloud SDK!
Traceback (most recent call last):
File "/Users/AdamLiu/Git/google-cloud-sdk/lib/third_party/enum/__init__.py", line 364, in __getattr__
return cls._member_map_[name]
KeyError: '_convert'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/AdamLiu/git/google-cloud-sdk/bin/bootstrapping/install.py", line 8, in <module>
import bootstrapping
File "/Users/AdamLiu/Git/google-cloud-sdk/bin/bootstrapping/bootstrapping.py", line 9, in <module>
import setup
File "/Users/AdamLiu/Git/google-cloud-sdk/bin/bootstrapping/setup.py", line 34, in <module>
from googlecloudsdk.core.util import platforms
File "/Users/AdamLiu/Git/google-cloud-sdk/lib/googlecloudsdk/core/util/platforms.py", line 18, in <module>
import platform
File "/Users/AdamLiu/anaconda/lib/python3.5/platform.py", line 117, in <module>
import sys, os, re, subprocess
File "/Users/AdamLiu/anaconda/lib/python3.5/subprocess.py", line 364, in <module>
import signal
File "/Users/AdamLiu/anaconda/lib/python3.5/signal.py", line 8, in <module>
_IntEnum._convert(
File "/Users/AdamLiu/Git/google-cloud-sdk/lib/third_party/enum/__init__.py", line 366, in __getattr__
raise AttributeError(name)
AttributeError: _convert
Super thanks in advance!
Set the CLOUDSDK_PYTHON environment variable to the location of your Python 2.x executable before running install.sh.
Or make sure that you have python2 in your path so that which python2 can find it.
I'm a beginner in OpenGL/OpenCL.
I'm trying to execute code from this
example, but there is an error:
Traceback (most recent call last):
File "/home/anka-rybalko/workspace/bla/openGL.py", line 99, in initializeGL
self.initialize_buffers()
File "/home/anka-rybalko/workspace/bla/openGL.py", line 61, in initialize_buffers
self.ctx, self.queue = clinit()
File "/home/anka-rybalko/workspace/bla/openGL.py", line 37, in clinit
+ get_gl_sharing_context_properties())
File "/usr/lib64/python2.7/site-packages/pyopencl-2014.1-py2.7-linux-x86_64.egg/pyopencl/tools.py", line 422, in get_gl_sharing_context_properties
from OpenGL import platform as gl_platform, GLX, WGL
File "/usr/lib/python2.7/site-packages/OpenGL/WGL/__init__.py", line 1, in <module>
from OpenGL.raw.WGL.VERSION.WGL_1_0 import *
File "/usr/lib/python2.7/site-packages/OpenGL/raw/WGL/VERSION/WGL_1_0.py", line 48, in <module>
#_p.types(_cs.c_int,_cs.HDC,ctypes.POINTER(_cs.PIXELFORMATDESCRIPTOR))
File "/usr/lib/python2.7/site-packages/OpenGL/raw/WGL/VERSION/WGL_1_0.py", line 13, in _f
return _p.createFunction( function,_p.PLATFORM.WGL,'WGL_VERSION_WGL_1_0',error_checker=_errors._error_checker)
AttributeError: 'GLXPlatform' object has no attribute 'WGL'
As I understand, WGL is an API for Windows OS and not for Linux. Should I somehow specify my platform before? Or how can I fix this?
Thanks in advance!
The code you are using (PyOpenGL 2014.1) is importing two platform-specific modules (WGL and GLX). PyOpenGL should have raised that error as an ImportError (rather than the AttributeError), but it still would have failed.
PyOpenCL has, in the meantime, worked around the issue in their github repository. If you have git installed (and pip) then this should get you a new version (note: not set up for OpenCL development here, so can't actually test this):
pip install git+https://github.com/pyopencl/pyopencl#egg=pyopencl
$ bin/python src/app/utils/loader.py newTasks checkForStuff
Traceback (most recent call last):
File "bin/python", line 135, in <module>
execfile(__file__)
File "src/app/utils/loader.py", line 154, in <module>
module = __import__(args.module)
ImportError: No module named newTasks
from the following loader.py:
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Runs python')
parser.add_argument('module', metavar='module', help="the target module")
args = parser.parse_args()
module = __import__(args.module)
newTasks.py is in the same directory as loader.py. It works on my dev box, I did an svn export to staging and the above happened. Any pointers?
Only material difference I can work out is this time I'm running out of virtualenv.
EDIT:
I changed the command line to:
$ bin/python src/app/utils/loader.py utils.newTasks checkForStuff
and the module was found ok. But my next command is to call one of the module functions checkForStuff:
parser.add_argument('func',metavar='function', help="the target function name")
...
func_param = getattr(module,args.func)
and now it cannot find the function?
Traceback (most recent call last):
File "bin/python", line 135, in <module>
execfile(__file__)
File "src/app/utils/loader.py", line 156, in <module>
func_param = getattr(module,args.func)
AttributeError: 'module' object has no attribute 'checkForStuff'
If I go into bin/python:
>>> import utils.newTasks as nt
>>> nt.checkForStuff
<function checkForStuff at 0x29b2f50>
This is annoying!
bin/python is not the python binary on your staging machine. Check whether it inserts the scripts directory into sys.path as ordinary python does.