I placed a ClientConnectionError exception in a multiprocessing.Queue that was generated by asyncio. I did this to pass an exception generated in asyncio land back to a client in another thread/process.
My assumption is that this exception occurred during the deserialization process reading the exception out of the queue. It looks pretty much impossible to reach otherwise.
Traceback (most recent call last):
File "model_neural_simplified.py", line 318, in <module>
main(**arg_parser())
File "model_neural_simplified.py", line 314, in main
globals()[command](**kwargs)
File "model_neural_simplified.py", line 304, in predict
next_neural_data, next_sample = reader.get_next_result()
File "/project_neural_mouse/src/asyncs3/s3reader.py", line 174, in get_next_result
result = future.result()
File "/usr/lib/python3.6/concurrent/futures/_base.py", line 432, in result
return self.__get_result()
File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "model_neural_simplified.py", line 245, in read_sample
f_bytes = s3f.read(read_size)
File "/project_neural_mouse/src/asyncs3/s3reader.py", line 374, in read
size, b = self._issue_request(S3Reader.READ, (self.url, size, self.position))
File "/project_neural_mouse/src/asyncs3/s3reader.py", line 389, in _issue_request
response = self.communication_channels[uuid].get()
File "/usr/lib/python3.6/multiprocessing/queues.py", line 113, in get
return _ForkingPickler.loads(res)
File "/usr/local/lib/python3.6/dist-packages/aiohttp/client_exceptions.py", line 133, in __init__
super().__init__(os_error.errno, os_error.strerror)
AttributeError: 'str' object has no attribute 'errno'
I figure it's a long shot to ask, but does anyone know anything about this issue?
Python 3.6.8, aiohttp.__version__ == 3.6.0
Update:
I managed to reproduce the issue (credit to Samuel in comments for improving the minimal reproducible test case, and later xtreak at bugs.python.org for furthing distilling it to a pickle-only test case):
import pickle
ose = OSError(1, 'unittest')
class SubOSError(OSError):
def __init__(self, foo, os_error):
super().__init__(os_error.errno, os_error.strerror)
cce = SubOSError(1, ose)
cce_pickled = pickle.dumps(cce)
pickle.loads(cce_pickled)
./python.exe ../backups/bpo38254.py
Traceback (most recent call last):
File "/Users/karthikeyansingaravelan/stuff/python/cpython/../backups/bpo38254.py", line 12, in <module>
pickle.loads(cce_pickled)
File "/Users/karthikeyansingaravelan/stuff/python/cpython/../backups/bpo38254.py", line 8, in __init__
super().__init__(os_error.errno, os_error.strerror)
AttributeError: 'str' object has no attribute 'errno'
References:
https://github.com/aio-libs/aiohttp/issues/4077
https://bugs.python.org/issue38254
OSError has a custom __reduce__ implementation; unfortunately, it's not subclass friendly for subclasses that don't match the expected arguments. You can see the intermediate state of the pickling by calling __reduce__ manually:
>>> SubOSError.__reduce__(cce)
(modulename.SubOSError, (1, 'unittest'))
The first element of the tuple is the callable to call, the second is the tuple of arguments to pass. So when it tries to recreate your class, it does:
modulename.SubOSError(1, 'unittest')
having lost the information about the OSError you were originally created with.
If you must accept arguments that don't match what OSError.__reduce__/OSError.__init__ expects, you're going to need to write your own __reduce__ override to ensure the correct information is pickled. A simple version might be:
class SubOSError(OSError):
def __init__(self, foo, os_error):
self.foo = foo # Must preserve information for pickling later
super().__init__(os_error.errno, os_error.strerror)
def __reduce__(self):
# Pickle as type plus tuple of args expected by type
return type(self), (self.foo, OSError(*self.args))
With that design, SubOSError.__reduce__(cce) would now return:
(modulename.SubOSError, (1, PermissionError(1, 'unittest')))
where the second element of the tuple is the correct arguments needed to recreate the instance (the change from OSError to PermissionError is expected; OSError actually returns its own subclasses based on the errno).
This issue was fixed and merged to master in aiohttp on 25 Sep 2019. I'll update this answer in the future if I note a version that the fix goes into (feel free to edit this answer in the future to note a version containing this update).
Git issue with the fix:
https://github.com/aio-libs/aiohttp/issues/4077
Related
I have a small chunk of code which not being a python guru I am not sure what it is really 'doing', but it fails under python3:
if indSoundsToPlay:
indSoundsToPlay = list(indSoundsToPlay)
indSoundsToPlay.sort()
soundsToPlay = list(zip(*indSoundsToPlay)[1]) <-- fails
soundsToPlay.reverse() # since played from back to front
self.playSounds(soundsToPlay)
This worked fine in Python2, but upon converting it did not like the list zip line at all. So I converted that line to:
soundsToPlay = list(zip(*indSoundsToPlay))
It was okay with that, I am not sure what the list zip * is really doing in python. Though the error subsided but now I realized that I painted myself into either a) another error or b) a new error just near the same code. The slice of code that now breaks is :
What error I get now is on the function call to playSounds:
def playSounds(self, sounds):
"""Play one or more of a set of sounds; played in order from last to first.
"""
if not sounds:
return
soundFunc = sounds.pop(-1)
soundFunc()
if sounds:
self._soundTimer.start(_SoundInterval, self.playSounds, sounds)
KeyVar('tcc', 'AxisCmdState') <bound method AxisStatusWdg.setAxisCmdState of <TUI.TCC.StatusWdg.AxisStatus.AxisStatusWdg object .!toplevel46.!statuswdg.!axisstatuswdg>>(*(['Drifting', 'Drifting', 'Drifting'],), **{'isCurrent': True, 'keyVar': KeyVar('tcc', 'AxisCmdState', 3, str)}) failed: 'tuple' object is not callable
Traceback (most recent call last):
File "/Users/st/TUI3/RO/AddCallback.py", line 77, in safeCall2
return func(*args, **kwargs)
File "/Users/st/TUI3/TUI/TCC/StatusWdg/AxisStatus.py", line 300, in setAxisCmdState
self.playSounds(soundsToPlay)
File "/Users/st/TUI3/TUI/TCC/StatusWdg/AxisStatus.py", line 359, in playSounds
soundFunc()
TypeError: 'tuple' object is not callable
KeyVar('tcc', 'AxisCmdState') <bound method AxisStatusWdg.setAxisCmdState of <TUI.TCC.StatusWdg.AxisStatus.AxisStatusWdg object .!toplevel46.!statuswdg.!axisstatuswdg>>(*(['Tracking', 'Tracking', 'Tracking'],), **{'isCurrent': True, 'keyVar': KeyVar('tcc', 'AxisCmdState', 3, str)}) failed: 'tuple' object is not callable
Traceback (most recent call last):
File "/Users/st/TUI3/RO/AddCallback.py", line 77, in safeCall2
return func(*args, **kwargs)
File "/Users/st/TUI3/TUI/TCC/StatusWdg/AxisStatus.py", line 300, in setAxisCmdState
self.playSounds(soundsToPlay)
File "/Users/st/TUI3/TUI/TCC/StatusWdg/AxisStatus.py", line 359, in playSounds
soundFunc()
TypeError: 'tuple' object is not callable
I don't know if it's what I did to indSoundsToPlay with the changing of that list(zip command or if it is a separate error in general. So not really sure where to start with this one besides learning what list(zip(* means in python.
I am running under Python 3.7 on Linux Ubuntu 18.04 under Eclipse 4.8 and Pydev.
The declaration:
args:Dict[str: Optional[Any]] = {}
is in a module that is imported from my testing code. and it is flagged with the following error message from typing.py:
TypeError: Parameters to generic types must be types. Got slice(<class 'str'>, typing.Union[typing.Any, NoneType], None). The stack trace follows: Finding files... done. Importing test modules ... Traceback (most recent call last): File "/Data/WiseOldBird/Eclipse/pool/plugins/org.python.pydev.core_7.0.3.201811082356/pysrc/_pydev_runfiles/pydev_runfiles.py", line 468, in __get_module_from_str
mod = __import__(modname) File "/Data/WiseOldBird/Workspaces/WikimediaAccess/WikimediaLoader/Tests/wiseoldbird/loaders/TestWikimediaLoader.py", line 10, in <module>
from wiseoldbird.application_controller import main File "/Data/WiseOldBird/Workspaces/WikimediaAccess/WikimediaLoader/src/wiseoldbird/application_controller.py", line 36, in <module>
args:Dict[str: Optional[Any]] = {} File "/usr/local/lib/python3.7/typing.py", line 251, in inner
return func(*args, **kwds) File "/usr/local/lib/python3.7/typing.py", line 626, in __getitem__
params = tuple(_type_check(p, msg) for p in params) File "/usr/local/lib/python3.7/typing.py", line 626, in <genexpr>
params = tuple(_type_check(p, msg) for p in params) File "/usr/local/lib/python3.7/typing.py", line 139, in _type_check
raise TypeError(f"{msg} Got {arg!r:.100}.") TypeError: Parameters
This prevents my testing module from being imported.
What am I doing wrong?
The proper syntax for a dict's type is
Dict[str, Optional[Any]]
When you write [a: b], Python interprets this as a slice, i.e. the thing that makes taking parts of arrays work, like a[1:10]. You can see this in the error message: Got slice(<class 'str'>, typing.Union[typing.Any, NoneType], None).
I have a function that is supposed to build a window according to which option in a dropdown menu is selected:
def buildview():
value = StringVar()
options = ["one", "two", "three"]
menu = OptionMenu(*(root, value) + tuple(options))
### Some window building accoring to the value selected... ###
value.trace_variable("w", buildview)
The exception that is raised looks like this (EDIT: Entire Traceback):
Traceback (most recent call last):
File "D:\Dropbox\PRO\infograbber\Infograbber 0.1.py", line 102, in <module>
mainloop()
File "C:\Python35\Python3564\lib\tkinter\__init__.py", line 405, in mainloop
_default_root.tk.mainloop(n)
File "C:\Python35\Python3564\lib\tkinter\__init__.py", line 1553, in __call__
self.widget._report_exception()
AttributeError: 'StringVar' object has no attribute '_report_exception'
What exactly is causing this? Can I not have a method call back itself like this? I don't even know where to start fixing this problem, so I'd appreciate any help.
I'm using Python 3.5 64 bit, Sublime Text 2, Windows 10.
EDIT:
Added a test callback function:
def test(*args):
print("test")
and changed the above trace to
value.trace_variable("w", test)
Now the exception changed to this:
Traceback (most recent call last):
File "C:\Python35\Python3564\lib\tkinter\__init__.py", line 1549, in __call__
return self.func(*args)
File "D:\Dropbox\PRO\infograbber\Infograbber 0.1.py", line 56, in buildview
root.trace_variable("w", self.printcurrentarticle)
File "C:\Python35\Python3564\lib\tkinter\__init__.py", line 1948, in __getattr__
return getattr(self.tk, attr)
AttributeError: '_tkinter.tkapp' object has no attribute 'trace_variable'
I'm not entirely sure if this is the only problem, but it's definitely a problem. When a trace fires it passes in three arguments. The function you've defined takes no arguments. You need to pass in a reference to a function that takes at least three arguments.
You also have the problem that each time the trace fires, you are creating another variable and another trace. On the surface that seems like a bad idea, unless you really do want to create a new option menu every time an optionmenu changes.
I'm trying to debug an application which makes use of the pynetdicom library. I'm not sure how relevant that specific detail is, however what IS relevant is that it makes heavy use of multithreading to run background socket listener tasks without blocking the main thread. The storescp.py example can be used to reproduce this.
Whenever I place a breakpoint that gets encountered (regardless of what thread, main or child, it gets encountered in), I get the following traceback:
Traceback (most recent call last):
File "/Applications/Aptana Studio 3/plugins/org.python.pydev_2.7.0.2013012902/pysrc/pydevd.py", line 1397, in <module>
debugger.run(setup['file'], None, None)
File "/Applications/Aptana Studio 3/plugins/org.python.pydev_2.7.0.2013012902/pysrc/pydevd.py", line 1090, in run
pydev_imports.execfile(file, globals, locals) #execute the script
File "/Users/alexw/Development/Python/kreport2/KReport2/dicomdatascraper.py", line 183, in <module>
oldDicomList = copy.copy(newData)
File "/Users/alexw/Development/Python/kreport2/KReport2/dicomdatascraper.py", line 183, in <module>
oldDicomList = copy.copy(newData)
File "/Applications/Aptana Studio 3/plugins/org.python.pydev_2.7.0.2013012902/pysrc/pydevd_frame.py", line 135, in trace_dispatch
self.doWaitSuspend(thread, frame, event, arg)
File "/Applications/Aptana Studio 3/plugins/org.python.pydev_2.7.0.2013012902/pysrc/pydevd_frame.py", line 25, in doWaitSuspend
self._args[0].doWaitSuspend(*args, **kwargs)
File "/Applications/Aptana Studio 3/plugins/org.python.pydev_2.7.0.2013012902/pysrc/pydevd.py", line 832, in doWaitSuspend
self.processInternalCommands()
File "/Applications/Aptana Studio 3/plugins/org.python.pydev_2.7.0.2013012902/pysrc/pydevd.py", line 360, in processInternalCommands
thread_id = GetThreadId(t)
File "/Applications/Aptana Studio 3/plugins/org.python.pydev_2.7.0.2013012902/pysrc/pydevd_constants.py", line 140, in GetThreadId
return thread.__pydevd_id__
File "/Users/alexw/.virtualenvs/kreport2dev/devlibs/pynetdicom/source/netdicom/applicationentity.py", line 73, in __getattr__
obj = eval(attr)()
File "<string>", line 1, in <module>
NameError: name '__pydevd_id__' is not defined
My thought is that, perhaps, in order to make things work, PyDev monkey-patches a __pydevd_id__ into spawned threads, however fails to patch those into these threads because they are, in fact, subclasses and not direct instances of threading.Thread (in this case, the worker is an instance of class Association(threading.Thread):).
Of course, I don't know PyDev well enough to confirm this theory, or else fix it. And it seems neither does the internet.
Is subclassing Thread so rarely used a pattern that it's simply not considered in the PyDev architecture? Without re-architecting the library, how could this issue be remedied?
I simply needed to look harder at that traceback.
The pynetdicom library, in its subclassing of threading.Thread, overrode __getattr__ and somewhat broke it. The problem was:
def __getattr__(self, attr):
#while not self.AssociationEstablished:
# time.sleep(0.001)
obj = eval(attr)
# do some stuff
return obj
when a nonexistent attribute is passed, a NameError is raised. This isn't caught by pydev's monkeypatching routine (if thread.__pydevd_id__ raises AttributeError, thread.__pydevd_id__ = stuff)
The solution was to update that section thusly:
def __getattr__(self, attr):
#while not self.AssociationEstablished:
# time.sleep(0.001)
try:
obj = eval(attr)
except NameError:
raise AttributeError
# do some stuff
return obj
This intercepts the NameError and raises an AttributeError instead, as __getattr__ should if the queried attribute doesn't exist.
I'm trying to get the name of a WMI win32 class. But the __name__ attribute is not defined for it.
>> import wmi
>> machine = wmi.WMI()
>> machine.Win32_ComputerSystem.__name__
I get the following error:
Traceback (most recent call last):
File "<pyshell#21>", line 1, in <module>
machine.Win32_ComputerSystem.__name__
File "C:\Python27\lib\site-packages\wmi.py", line 796, in __getattr__
return _wmi_object.__getattr__ (self, attribute)
File "C:\Python27\lib\site-packages\wmi.py", line 561, in __getattr__
return getattr (self.ole_object, attribute)
File "C:\Python27\lib\site-packages\win32com\client\dynamic.py", line 457, in __getattr__
raise AttributeError(attr)
AttributeError: __name__
I thought that the __name__ attribute is defined for all Python functions, so I don't know what the problem is here. How is it possible that this function doesn't have that attribute?
OK, The reason that I thought it was a method is because machine.Win32_ComputerSystem() is defined, but I guess that isn't enough for something to be a method. I realise that it isn't a method.
However, this doesn't work:
>> machine.Win32_ComputerSystem.__class__.__name__
'_wmi_class'
I want it to return 'Win32_ComputerSystem'. How can I do this?
From what I can tell looking at the documentation (specifically, based on this snippet), wmi.Win32_ComputerSystem is a class, not a method. If you want to get its name you could try:
machine.Win32_ComputerSystem.__class__.__name__
I've found a way to get the output that I want, however it doesn't satisfy me.
repr(machine.Win32_ComputerSystem).split(':')[-1][:-1]
returns: 'Win32_ComputerSystem'
There must be a more Pythonic way to do this.