Develop Reference

How to apply safe multi-threading in tkinter?

I have a system that is connected with multiple hardware devices(cameras, router, sensors etc.). So I thought about having a separate GUI that pings the IP addresses of these devices almost every second in order to check whenever a device gets disconnected, and I need to ping them all together.
This is the ping script:
def testRouter(self):
response = os.popen(f"ping {self.router_ip} {self.count} 1").read()
if "Received = 1" in response:
self.router_status.configure(image=self.connected)
self.router_button.configure(command=self.clickedNone)
else:
self.router_status.configure(image=self.disconnected)
self.router_button.configure(command=self.clickedRouter)
self.root.after(5000, self.testRouter)
But the GUI freezes for more than a minute till it displays the content, and many other times it just freezes forever. So I tried to implement a thread for each ping but I came across the fact that tkinter is not a thread-safe library with an error RuntimeError: main thread is not in main loop
Any idea on how to implement this? (without using mtTkinter please)

Interrupt a pySerial readline in another thread

I am using pySerial to communicate to a microcontroller over USB, Most of the communication is initiated by the desktop python script which sends a command packet and waits for reply.
But there is also a alert packet that may be sent by the microcontroller without a command from the python script. In this case I need to monitor the read stream for any alerts.
For handling alerts, I dedicate a seperate process to call readline() and loop around it, like so:
def serialMonitor(self):
while not self.stopMonitor:
self.lock.acquire()
message = self.stream.readline()
self.lock.release()
self.callback(message)
inside a class. The function is then started in a seperate process by
self.monitor = multiprocessing.Process(target = SerialManager.serialMonitor, args = [self])
Whenever a command packet is send, the command function needs to take back control of the stream, for which it must interrupt the readline() call which is in blocking. How do I interrupt the readline() call? Is there any way to terminate a process safely?

You can terminate a multiprocessing process with .terminate(). Is this safe? Probably it's alright for a readline case.
However, this is not how I would handle things here. As I read your scenario, there are two possibilities:
MCU initiates alert package
Computer sends data to MCU (and MCU perhaps responds)
I assume the MCU will not send an alert package whilst an exchange is going on initiated by the computer.
So I would just initiate the serial object with a small timeout, and leave it in a loop when I'm not using it. My overall flow would go like this:
ser = Serial(PORT, timeout=1)
response = None
command_to_send = None
running = True
while running: # event loop
while running and not command_to_send and not line:
try:
line = ser.readline()
except SerialTimeoutException:
pass
if not command_to_send:
process_mcu_alert(line)
else:
send_command(command_to_send)
command_to_send = None
response = ser.readline()
This is only a sketch, as it would need to be run in a thread or subprocess, since readline() is indeed blocking, so you need some thread-safe way of setting command_to_send and running (used to exit gracefully) and getting response, and you likely want to wrap all this state up in a class. The precise implementation of that depends upon what you are doing, but the principle is the same---have one loop which handles reading and writing to the serial port, have it timeout to respond relatively quickly (you can set a smaller timeout if you need to), and have it expose some interface you can handle.
Sadly to my knowledge python has no asyncio compatible serial library, otherwise that approach would seem neater.

Python Sockets - How to shut down the server?

I tried to make a simple chat system with the socket module in Python. everything works, except, that i need to kill the process everytime when i want to shutdown the server. And i don't want to do this everytime.
So my question is:
How can i make a function, that when i type shutdown in the server terminal, it shutdowns the whole server?
I already tried to do this:
def close(self):
server.close(self)
server.shutdown(self)
But it doesn't work. When i type close(), nothing happens. Nothing.
Heres the full code of the server.py:
https://pastebin.com/gA4QYmQe
Every help is useful. Thanks.

Well... there are many problems with the code (your "MY IP" and "SERVERIP" are probably not what you want, but this is beside the point.
Your close() function has a "self" parameter, which is pointless as this is not a class. You also need to move the close function to the beginning of your code if you want to call it from your try-except -structure. You need to call shutdown() first and then close(), and shutdown takes an argument. I modified your close() to do this and it works.
def close():
server.shutdown(socket.SHUT_RDWR)
server.close()
print ("closed")
When you open your socket, you should also set SO_REUSEADDR to make the address reusable (meaning you can start the server again if you shut it down, instead of waiting for a minute for TIME_WAIT status to finish with your server port):
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
But how exactly are you calling close() when you type "shutdown" somewhere? You are not doing this. Your program is in the socket loop and is not reading keyboard input.
I see no point whatsoever adding keyboard input to this program. First, it adds complexity as you are operating with two possibly blocking inputs (socket input and keyboard input) and you would need to manage this. It is possible but definitely complicated. Second, it is faster to press Control + C instead of typing "shutdown" and hitting enter.
You currently do not call close after a keyboard interrupt. I added this to the inner KeyboardInterrupt (the outer you can remove - it is not doing anything and is never reached) and it now shuts down your program neatly, closing all connections. Remember to move close() function from the bottom of your code to the front before the try: statement:
except KeyboardInterrupt:
print("[!] Keyboard Interrupted!")
close()
break
If you want a remote shutdown (server shuts down if you type "shutdown" to the socket), you can add this to your server loop:
if message == "shutdown":
close()
exit(0)
There are other problems as well. For example, if you start your server, connect to it and shut down the connection, your server exits as it does not return to listen().
This is also (in my opinion), somewhat bad programming, as you use "server" as a global variable. I would rather create a class and put all socket operations in it, but if style is not important, this should work.
Hannu

One other approach would be having a default admin-client that can control the server. Admin-client will be created when the server starts and from that client admin can shutdown and do any of the admin tasks on the server.

PySide QTextEdit or QPlainTextEdit update faster?

I am now trying to make a GUI on my PC communicate with a Server per sockets.
here is part of the code of GUI:
def listenToServer(self):
""" keep listening to the server until receiving 'All Contracts Finished' """
self.feedbackWindow.appendPlainText('--Executing the Contracts, Listening to Server--')
contentsListend = ''
while contentsListend != 'All Contracts Finished':
#keep listen from the socket
contentsListend = self.skt.recv(1024)
#make the GUI show the text
self.feedbackWindow.appendPlainText(contentsListend)
On the Other side, the server will send data one by one but with some interval. Here is the test code simulating the server:
for i in range(7):
print 'send back msg, round: ', i # this will be printed on the screen of the server, to let me know that the server works
time.sleep(1) # make some interval
# c is the connected socket, which can send messages
# just send the current loop number
c.send('send back msg' + str(i))
c.send('All Contracts Finished')
c.close()# Close the connection
Now, everything works except the problem that, the GUI will only show the received messages after the whole for loop in the server.
Once I run the server and the GUI. The server side print the messages onto the screen with correct speed one by one, but the GUI has no response, it does not update. Till the end of the program, all the 7 lines occurs all at once at GUI side. I want them to appear one by one, so that later I can inspect the state of the server with this GUI on my PC.
Can anybody help, thanks a lot!

This has nothing to do with "fast" or "slow".
The GUI runs on the same thread as your listenToServer method - so as long as it's running nothing can happen on the GUI thread. You'll notice that you can't move, resize or click anything in the GUI while you're waiting socket input.
You'll have to run your listenToServer method on a thread separate from the GUI. The proper way to do that would be to implement a Worker object that receives data from the socket and notifies you textEdit via a Signal->Slot connection that there's data ready to receive.
I answered a similar question a while back, that might help
https://stackoverflow.com/a/24821300/2319400
A really quick and dirty alternative would be to process all queued events when you've appended new data, via:
QApplication.processEvents()
This gives Qt time to e.g. repaint the GUI on the screen with new text.
Your GUI will however not respond to any events while python is waiting for data to come from the socket!

signal.alarm() handler causing problem with pyserial

so i have a motion sensor connected to an avr micro that is communicating with my python app via usb. im using pyserial to do the comm. during my script i have an infinate loop checking for data from the avr micro. before this loop i start a timer with signal.alarm() that will call a function to end a subprocess. when this alarm goes it interrupts the pyserial comm and the program exits completly. i get the error that pyserial read() is interrupted. is there any way around this issue. any help would be awesome

The problem is that your alarm will interrupt the read from the serial port, which isn't at all what you want.
It sounds like you probably want to break this into two threads that do work separately.

You are using alarm(), which send a signal, and pyserial, which does reads and writes to a serial port. When you are reading or writing to a device like that, and the SIGALRM signal is received, a read() or a write() call is interrupted so the signal can be handled.
As signals are handled in userspace, and reading and writing is actually handled by the kernel, this makes things rather ugly. This is a know wart of the way signals are handled, and dates back to the very early UNIX days.
Code that handles signals correctly in python may look like:
import errno
while True:
try:
data = read_operation()
except OSError, e:
if getattr(e, 'errno', errno.EINTR):
continue
raise
else:
break