Develop Reference

VI_ERROR_TMO (-1073807339) on Anritsu OSA

I'm working on interfacing an old Optical Spectrum Analyzer Anritsu MS9710B with a RS232 connection. A year ago, I managed to comunicate with him, send SCPI command using pyvisa and receive data. Today, I execute the exact same code with libraries up to date and I get an error Timeout when I query IDN or anything. The RS232-USB drivers are ok, I manage to open a communication but a query or read fail.
I changed the parameter to "RS232C" on the OSA parameters and my communication parameters are the same between OSA and program. Following advices from the forum, I tried the connection with NI-VISA where I get the same error. I tried to change timeout parameter, write_termination and read_termination but there isn't any change. The manual is very hard to understand termination value, but when it worked I didn't add anything.
I don't know anymore what to do.
Any advices and help would be warmly welcome to fix my problem !
import pyvisa
from pyvisa.constants import StopBits, Parity
rm = pyvisa.ResourceManager()
print(rm.list_resources())
my_instrument = rm.open_resource('ASRL5::INSTR')
my_instrument.baud_rate=9600
my_instrument.data_bits=8
my_instrument.parity=Parity.even
my_instrument.stop_bits=StopBits.one
my_instrument.write('*IDN?')
print(my_instrument.read())```
'''
('ASRL5::INSTR',)
Traceback (most recent call last):
File "PremiereComm.py", line 26, in <module>
print(my_instrument.read())
File "C:\Program Files\Python38\lib\site-packages\pyvisa\resources\messagebased.py", line 486, in read
message = self._read_raw().decode(enco)
File "C:\Program Files\Python38\lib\site-packages\pyvisa\resources\messagebased.py", line 442, in _read_raw
chunk, status = self.visalib.read(self.session, size)
File "C:\Program Files\Python38\lib\site-packages\pyvisa\ctwrapper\functions.py", line 2337, in read
ret = library.viRead(session, buffer, count, byref(return_count))
File "C:\Program Files\Python38\lib\site-packages\pyvisa\ctwrapper\highlevel.py", line 222, in _return_handler
return self.handle_return_value(session, ret_value) # type: ignore
File "C:\Program Files\Python38\lib\site-packages\pyvisa\highlevel.py", line 251, in handle_return_value
raise errors.VisaIOError(rv)
pyvisa.errors.VisaIOError: VI_ERROR_TMO (-1073807339): Timeout expired before operation completed.
>>> '''

Using VS Code to debug python files. Exception thrown on breakpoint and breakpoint is ignored

Tried with multiple different python files. Every time I try to use the debugger in vs code and set breakpoints the breakpoint gets ignored and exception gets raised and the script continues on. I've been googling and tinkering for over 2 hours and can't seem to figure out what's going on here. Tried rebooting PC, running vs code as admin, uninstall/reinstall the python extension for vs code. Tried to dig into the files mentioned in the traceback and pinpointed the function that seems to be raising the exception but I can't figure out where it's being called from or why it's raising the exception. I'm still new-ish to Python. Debugging works properly on my laptop but for whatever reason my desktop is having this issue.
Traceback (most recent call last):
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd_file_utils.py", line 529, in _original_file_to_client
return cache[filename]
KeyError: 'c:\\users\\joel\\local settings\\application data\\programs\\python\\python37-32\\lib\\runpy.py'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_comm.py", line 330, in _on_run
self.process_net_command_json(self.py_db, json_contents)
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_process_net_command_json.py", line 190, in process_net_command_json
cmd = on_request(py_db, request)
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_process_net_command_json.py", line 771, in on_stacktrace_request
self.api.request_stack(py_db, request.seq, thread_id, fmt=fmt, start_frame=start_frame, levels=levels)
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_api.py", line 214, in request_stack
if internal_get_thread_stack.can_be_executed_by(get_current_thread_id(threading.current_thread())):
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_comm.py", line 661, in can_be_executed_by
py_db, self.seq, self.thread_id, frame, self._fmt, must_be_suspended=not timed_out, start_frame=self._start_frame, levels=self._levels)
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_net_command_factory_json.py", line 213, in make_get_thread_stack_message
py_db, frames_list
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_net_command_factory_xml.py", line 175, in _iter_visible_frames_info
new_filename_in_utf8, applied_mapping = pydevd_file_utils.norm_file_to_client(filename_in_utf8)
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd_file_utils.py", line 531, in _original_file_to_client
translated = _path_to_expected_str(get_path_with_real_case(_AbsFile(filename)))
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd_file_utils.py", line 221, in _get_path_with_real_case
return _resolve_listing(drive, iter(parts))
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd_file_utils.py", line 184, in _resolve_listing
dir_contents = cache[resolved_lower] = os.listdir(resolved)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\Joel\\Local Settings'
So I get this traceback every time a breakpoint is hit. Taking a peek at the "_original_file_to_client" function in "pydevd_file_utils.py" we get this:
def _original_file_to_client(filename, cache={}):
try:
return cache[filename]
except KeyError:
translated = _path_to_expected_str(get_path_with_real_case(_AbsFile(filename)))
cache[filename] = (translated, False)
return cache[filename]
I wasn't able to figure out where this function was being called from or what the expected output was supposed to be. Any help with this would be greatly appreciated!
Edit: Forgot to mention I'm using Windows 10 if it wasn't obvious from the trace

This is a similar question. The spaces in the filename cause this problem:
"Local Settings", "application data"

How to solve "control message failed, win error: A device which does not exist was specified" error PyUSB

I'm trying to connect a POS receipt printer to python and have been using python-escpos to do so. I'm running windows 10.
My code in my file 'print.py' is:
from escpos.printer import Usb
p = Usb(0x0456,0x0808)
p.text('Hello World\n')
but when i run it i get the following error log:
Traceback (most recent call last):
File "C:\Users\Harry\AppData\Local\Programs\Python\Python36-32\lib\site-packages\usb\core.py", line 223, in get_interface_and_endpoint
return self._ep_info[endpoint_address]
KeyError: 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "print.py", line 5, in <module>
p.text('Hello World\n')
File "C:\Users\Harry\AppData\Local\Programs\Python\Python36-32\lib\site-packages\escpos\escpos.py", line 437, in text
self._raw(txt.encode())
File "C:\Users\Harry\AppData\Local\Programs\Python\Python36-32\lib\site-packages\escpos\printer.py", line 73, in _raw
self.device.write(self.out_ep, msg, self.timeout)
File "C:\Users\Harry\AppData\Local\Programs\Python\Python36-32\lib\site-packages\usb\core.py", line 940, in write
intf, ep = self._ctx.setup_request(self, endpoint)
File "C:\Users\Harry\AppData\Local\Programs\Python\Python36-32\lib\site-packages\usb\core.py", line 102, in wrapper
return f(self, *args, **kwargs)
File "C:\Users\Harry\AppData\Local\Programs\Python\Python36-32\lib\site-packages\usb\core.py", line 215, in setup_request
intf, ep = self.get_interface_and_endpoint(device, endpoint_address)
File "C:\Users\Harry\AppData\Local\Programs\Python\Python36-32\lib\site-packages\usb\core.py", line 102, in wrapper
return f(self, *args, **kwargs)
File "C:\Users\Harry\AppData\Local\Programs\Python\Python36-32\lib\site-packages\usb\core.py", line 225, in get_interface_and_endpoint
for intf in self.get_active_configuration(device):
File "C:\Users\Harry\AppData\Local\Programs\Python\Python36-32\lib\site-packages\usb\core.py", line 102, in wrapper
return f(self, *args, **kwargs)
File "C:\Users\Harry\AppData\Local\Programs\Python\Python36-32\lib\site-packages\usb\core.py", line 239, in get_active_configuration
bConfigurationValue=self.backend.get_configuration(self.handle)
File "C:\Users\Harry\AppData\Local\Programs\Python\Python36-32\lib\site-packages\usb\backend\libusb0.py", line 510, in get_configuration
100)
File "C:\Users\Harry\AppData\Local\Programs\Python\Python36-32\lib\site-packages\usb\backend\libusb0.py", line 593, in ctrl_transfer
timeout
File "C:\Users\Harry\AppData\Local\Programs\Python\Python36-32\lib\site-packages\usb\backend\libusb0.py", line 431, in _check
raise USBError(errmsg, ret)
usb.core.USBError: [Errno None] b'libusb0-dll:err [control_msg] sending control message failed, win error: A device which does not exist was specified.\r\n\n'
Does anyone know what i need to do to make this work.
I have installed libusb into my System32 and SysWOW64 folders, aswell as installing a libusb-32 driver for my printer with zadig.
The printer i'm using is a 'Hoin 58mm POS Printer'.
I would be grateful if someone could explain to me why i'm having this error and how to solve it!

Firstly, you can check this link for the complete thread.
As suggested by KM4YRI,
I ran into the same No backend available exception when trying to follow the tutorial on a Windows 7 machine. The following worked for me, per one of the StackOverflow replies:
1. Download the latest Windows binary: https://sourceforge.net/projects/libusb/files/libusb-1.0/libusb-1.0.21/libusb-1.0.21.7z/download
2. Unzip using 7zip into a temp dir
3. If on 64-bit Windows, copy `MS64\dll\libusb-1.0.dll` into `C:\windows\system32`. If on 32-bit windows, copy `MS32\dll\libusb-1.0.dll` into `C:\windows\SysWOW64`.
I've verified the above answer on Windows-10 64-bit version.

I had a tight experience with that error using at least 3 different USB Storage Devices. After hours of trials I ended up to a feeling that USB transmission errors are the cause of this error.
The USB bus will return it in a random pattern for a low quality connection as if a device remove has presumably happened.
In my scenario I was bulk writing sectors to a \\.\PhysicalDriveN device and I noticed that after a short period the connection reestablished and the file handle was still valid and going.
To overcome the error I've ended up with the following steps:
Seek To File Position;
Write Sector;
For any 433 error:
250 ms delay;
Seek To file Position, ignoring ERROR_NO_SUCH_DEVICE and ERROR_NOT_READY errors and trying for at least 1000 ms before giving up;
Read Sector, ignoring ERROR_NO_SUCH_DEVICE and ERROR_NOT_READY errors and retrying for up to 1000 ms before giving up;
Specially for my scenario: I've compared input buffer and read buffer and stop if matching;
Repeat the whole procedure up to 5 times.
So the idea here is the following: The device driver returns ERROR_NO_SUCH_DEVICE assuming a device remove scenario, but still in a pending state. As the problem is simply quality of physical connection, the link continues and the bus driver starts a renegotiation, in which case a ERROR_NOT_READY is temporarily returned. When the link is stable and negotiated, communication continues without the need to reopen the device handle.

I would recommend adding a sleep statement of a second or two in between the instantiation of the USB printer, and the attempt to print to it. e.g:
from escpos.printer import Usb
from time import sleep
p = Usb(0x0456,0x0808)
sleep(1)
p.text('Hello World\n')

pyspark got Py4JNetworkError("Answer from Java side is empty") when exit python

Background:
spark standalone cluster mode on k8s
spark 2.2.1
hadoop 2.7.6
run code in python, not in pyspark
client mode, not cluster mode
The pyspark code in python, not in pyspark env.
Every code can work and get it down. But 'sometimes', when the code finish and exit, below error will show up even time.sleep(10) after spark.stop().
{{py4j.java_gateway:1038}} INFO - Error while receiving.
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/py4j-0.10.4-py2.7.egg/py4j/java_gateway.py", line 1035, in send_command
raise Py4JNetworkError("Answer from Java side is empty")
Py4JNetworkError: Answer from Java side is empty
[2018-11-22 09:06:40,293] {{root:899}} ERROR - Exception while sending command.
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/py4j-0.10.4-py2.7.egg/py4j/java_gateway.py", line 883, in send_command
response = connection.send_command(command)
File "/usr/lib/python2.7/site-packages/py4j-0.10.4-py2.7.egg/py4j/java_gateway.py", line 1040, in send_command
"Error while receiving", e, proto.ERROR_ON_RECEIVE)
Py4JNetworkError: Error while receiving
[2018-11-22 09:06:40,293] {{py4j.java_gateway:443}} DEBUG - Exception while shutting down a socket
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/py4j-0.10.4-py2.7.egg/py4j/java_gateway.py", line 441, in quiet_shutdown
socket_instance.shutdown(socket.SHUT_RDWR)
File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
File "/usr/lib64/python2.7/socket.py", line 170, in _dummy
raise error(EBADF, 'Bad file descriptor')
error: [Errno 9] Bad file descriptor
I guess the reason is parent process python try to get log message from terminated child process 'jvm'. But the wired thing is the error not always raise...
Any suggestion?

This root-cause is 'py4j' log-level.
I set python log-level to DEBUG, this let the 'py4j' client & 'java' raise connection error when close pyspark.
So setting python log-level to INFO or more higher level will resolve this problem.
ref: Gateway raises an exception when shut down
ref: Tune down the logging level for callback server messages
ref: PySpark Internals

Error with multiprocessing, atexit and global data

Sorry in advance, this is going to be long ...
Possibly related:
Python Multiprocessing atexit Error "Error in atexit._run_exitfuncs"
Definitely related:
python parallel map (multiprocessing.Pool.map) with global data
Keyboard Interrupts with python's multiprocessing Pool
Here's a "simple" script I hacked together to illustrate my problem...
import time
import multiprocessing as multi
import atexit
cleanup_stuff=multi.Manager().list([])
##################################################
# Some code to allow keyboard interrupts
##################################################
was_interrupted=multi.Manager().list([])
class _interrupt(object):
"""
Toy class to allow retrieval of the interrupt that triggered it's execution
"""
def __init__(self,interrupt):
self.interrupt=interrupt
def interrupt():
was_interrupted.append(1)
def interruptable(func):
"""
decorator to allow functions to be "interruptable" by
a keyboard interrupt when in python's multiprocessing.Pool.map
**Note**, this won't actually cause the Map to be interrupted,
It will merely cause the following functions to be not executed.
"""
def newfunc(*args,**kwargs):
try:
if(not was_interrupted):
return func(*args,**kwargs)
else:
return False
except KeyboardInterrupt as e:
interrupt()
return _interrupt(e) #If we really want to know about the interrupt...
return newfunc
#atexit.register
def cleanup():
for i in cleanup_stuff:
print(i)
return
#interruptable
def func(i):
print(i)
cleanup_stuff.append(i)
time.sleep(float(i)/10.)
return i
#Must wrap func here, otherwise it won't be found in __main__'s dict
#Maybe because it was created dynamically using the decorator?
def wrapper(*args):
return func(*args)
if __name__ == "__main__":
#This is an attempt to use signals -- I also attempted something similar where
#The signals were only caught in the child processes...Or only on the main process...
#
#import signal
#def onSigInt(*args): interrupt()
#signal.signal(signal.SIGINT,onSigInt)
#Try 2 with signals (only catch signal on main process)
#import signal
#def onSigInt(*args): interrupt()
#signal.signal(signal.SIGINT,onSigInt)
#def startup(): signal.signal(signal.SIGINT,signal.SIG_IGN)
#p=multi.Pool(processes=4,initializer=startup)
#Try 3 with signals (only catch signal on child processes)
#import signal
#def onSigInt(*args): interrupt()
#signal.signal(signal.SIGINT,signal.SIG_IGN)
#def startup(): signal.signal(signal.SIGINT,onSigInt)
#p=multi.Pool(processes=4,initializer=startup)
p=multi.Pool(4)
try:
out=p.map(wrapper,range(30))
#out=p.map_async(wrapper,range(30)).get() #This doesn't work either...
#The following lines don't work either
#Effectively trying to roll my own p.map() with p.apply_async
# results=[p.apply_async(wrapper,args=(i,)) for i in range(30)]
# out = [ r.get() for r in results() ]
except KeyboardInterrupt:
print ("Hello!")
out=None
finally:
p.terminate()
p.join()
print (out)
This works just fine if no KeyboardInterrupt is raised. However, if I raise one, the following exception occurs:
10
7
9
12
^CHello!
None
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "test.py", line 58, in cleanup
for i in cleanup_stuff:
File "<string>", line 2, in __getitem__
File "/usr/lib/python2.6/multiprocessing/managers.py", line 722, in _callmethod
self._connect()
File "/usr/lib/python2.6/multiprocessing/managers.py", line 709, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
File "/usr/lib/python2.6/multiprocessing/connection.py", line 143, in Client
c = SocketClient(address)
File "/usr/lib/python2.6/multiprocessing/connection.py", line 263, in SocketClient
s.connect(address)
File "<string>", line 1, in connect
error: [Errno 2] No such file or directory
Error in sys.exitfunc:
Traceback (most recent call last):
File "/usr/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "test.py", line 58, in cleanup
for i in cleanup_stuff:
File "<string>", line 2, in __getitem__
File "/usr/lib/python2.6/multiprocessing/managers.py", line 722, in _callmethod
self._connect()
File "/usr/lib/python2.6/multiprocessing/managers.py", line 709, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
File "/usr/lib/python2.6/multiprocessing/connection.py", line 143, in Client
c = SocketClient(address)
File "/usr/lib/python2.6/multiprocessing/connection.py", line 263, in SocketClient
s.connect(address)
File "<string>", line 1, in connect
socket.error: [Errno 2] No such file or directory
Interestingly enough, the code does exit the Pool.map function without calling any of the additional functions ... The problem seems to be that the KeyboardInterrupt isn't handled properly at some point, but it is a little confusing where that is, and why it isn't handled in interruptable. Thanks.
Note, the same problem happens if I use out=p.map_async(wrapper,range(30)).get()
EDIT 1
A little closer ... If I enclose the out=p.map(...) in a try,except,finally clause, it gets rid of the first exception ... the other ones are still raised in atexit however. The code and traceback above have been updated.
EDIT 2
Something else that does not work has been added to the code above as a comment. (Same error). This attempt was inspired by:
http://jessenoller.com/2009/01/08/multiprocessingpool-and-keyboardinterrupt/
EDIT 3
Another failed attempt using signals added to the code above.
EDIT 4
I have figured out how to restructure my code so that the above is no longer necessary. In the (unlikely) event that someone stumbles upon this thread with the same use-case that I had, I will describe my solution ...
Use Case
I have a function which generates temporary files using the tempfile module. I would like those temporary files to be cleaned up when the program exits. My initial attempt was to pack each temporary file name into a list and then delete all the elements of the list with a function registered via atexit.register. The problem is that the updated list was not being updated across multiple processes. This is where I got the idea of using multiprocessing.Manager to manage the list data. Unfortunately, this fails on a KeyboardInterrupt no matter how hard I tried because the communication sockets between processes were broken for some reason. The solution to this problem is simple. Prior to using multiprocessing, set the temporary file directory ... something like tempfile.tempdir=tempfile.mkdtemp() and then register a function to delete the temporary directory. Each of the processes writes to the same temporary directory, so it works. Of course, this solution only works where the shared data is a list of files that needs to be deleted at the end of the program's life.