I have an application written in Python, using Tkinter. One of the features allows a serial port to be opened, after which any messages received over the serial port are displayed in a text window. This works fine. The problem comes when I close the window, which doesn't kill the thread monitoring the serial port. It then has to be killed manually (alternatively, unplugging the USB-serial cable causes an exception which kills the process).
I assume I'm missing something simple here, but I would have thought closing the application would close all associated threads. I can't seem to find anything about this in the documentation, but I'm probably looking in the wrong place?
Code for the serial thread in case it's relevant:
class SerialThread(threading.Thread):
def __init__(self, queue, sp):
threading.Thread.__init__(self)
self.queue = queue
self.ser_handle = sp;
def run(self):
while True:
if self.ser_handle.inWaiting():
text = self.ser_handle.readline(self.ser_handle.inWaiting())
self.queue.put(text)
time.sleep(0.2)
You must have a way to ask the thread for stop, you can accomplish that using: threading.Event.
class SerialThread(threading.Thread):
def __init__(self, queue, sp):
threading.Thread.__init__(self)
self.queue = queue
self.event = threading.Event() # An event object.
self.ser_handle = sp;
def stop(self):
self.event.set()
def run(self):
while not self.event.isSet():
if self.ser_handle.inWaiting():
text = self.ser_handle.readline(self.ser_handle.inWaiting())
self.queue.put(text)
time.sleep(0.2)
Then in the on close event of your window, call your_thread.stop()
Related
Suppose I have some consumer daemon threads that constantly take objects from a queue whenever the main thread puts them there and performs some long operation (a couple of seconds) with them.
The problem is that whenever the main thread is done, the daemon threads are killed before they finish processing whatever is left in the queue.
I know that one way to solve this could be to wait for the daemon threads to finish processing whatever is left in the queue and then exit, but I am curious if there is any way for the daemon threads to "clean up" after themselves (i.e. finish processing whatever is left in the queue) when the main thread exits, without explicitly having the main thread tell the daemon threads to start cleaning up.
The motivation behind this is that I made a python package that has a logging handler class that puts items into a queue whenever the user tries to log something (e.g. with logging.info("message")), and the handler has a daemon thread that sends the logs over the network. I'd prefer if the daemon thread could clean up by itself, so users of the package wouldn't have to manually make sure to make their main thread wait for the log handler to finish its processing.
Minimal working example
# this code is in my package
class MyHandler(logging.Handler):
def __init__(self, level):
super().__init__(level=level)
self.queue = Queue()
self.thread = Thread(target=self.consume, daemon=True)
self.thread.start()
def emit(self, record):
# This gets called whenever the user does logging.info, or similar
self.queue.put(record)
def consume(self):
while True:
record = self.queue.get()
send(record) # send record over network, can take a few seconds (assume it never raises)
self.queue.task_done()
# This is user's main code
# user will have to keep a reference to the handler for later. I want to avoid this.
my_handler = MyHandler()
# set up logging
logging.basicConfig(..., handlers=[..., my_handler])
# do some stuff...
logging.info("this will be sent over network")
# some more stuff...
logging.error("also sent over network")
# even more stuff
# before exiting must wait for handler to finish sending
# I don't want user to have to do this
my_hanler.queue.join()
You can use threading.main_thread.join() which will wait until shutdown like so:
import threading
import logging
import queue
class MyHandler(logging.Handler):
def __init__(self, level):
super().__init__(level=level)
self.queue = queue.Queue()
self.thread = threading.Thread(target=self.consume) # Not daemon
# Shutdown thread
threading.Thread(
target=lambda: threading.main_thread().join() or self.queue.put(None)
).start()
self.thread.start()
def emit(self, record):
# This gets called whenever the user does logging.info, or similar
self.queue.put(record)
def consume(self):
while True:
record = self.queue.get()
if record is None:
print("cleaning")
return # Cleanup
print(record) # send record over network, can take a few seconds (assume it never raises)
self.queue.task_done()
Quick test code:
logging.getLogger().setLevel(logging.INFO)
logging.getLogger().addHandler(MyHandler(logging.INFO))
logging.info("Hello")
exit()
You can use atexit to wait until the daemon thread shuts down:
import queue, threading, time, logging, atexit
class MyHandler(logging.Handler):
def __init__(self, level):
super().__init__(level=level)
self.queue = queue.Queue()
self.thread = threading.Thread(target=self.consume, daemon=True)
# Right before main thread exits, signal cleanup and wait until done
atexit.register(lambda: self.queue.put(None) or self.thread.join())
self.thread.start()
def emit(self, record):
# This gets called whenever the user does logging.info, or similar
self.queue.put(record)
def consume(self):
while True:
record = self.queue.get()
if record is None: # Cleanup requested
print("cleaning")
time.sleep(5)
return
print(record) # send record over network, can take a few seconds (assume it never raises)
self.queue.task_done()
# Test code
logging.getLogger().setLevel(logging.INFO)
logging.getLogger().addHandler(MyHandler(logging.INFO))
logging.info("Hello")
I am using one XBee S2 as coordinator (API mode), 3 XBee S2 as routers (AT mode). The routers are connected to Naze32 board (using MSP).
On the computer side, I have a GUI using wxpython to send out command to request data.
The GUI will send out command to XBee (Coordinator) to request data from the routers every second.
I am using python-xbee library to do the send and receive frame job on computer side. Once new data received, it will notify the GUI to update some labels with the new data.
Currently I am able to send and receive frames outside a thread, but once I move the send and receive functions to a thread, it will never be able to read a frame any more. As I don't want to let the serial stop the GUI or make it not responding. Another thing is if I close the thread, then start new thread with xbee again, it will not work any more.
The communication is controlled by a button on the GUI; once "Turn on" clicked, the "self._serialOn" will set to True, then create new thread; once "Turn off" clicked, "self._serialOn" will set to False and thread is stopped.
How can I fix this problem?
Thanks in advance.
class DataExchange(object):
def __init__(self):
self._observers = []
self._addressList = [['\x00\x13\xA2\x00\x40\xC1\x43\x0F', '\xFF\xFE'],[],[]]
self._serialPort = ''
self._serialOn = False
self.workerSerial = None
# serial switch
def get_serialOn(self):
return self._serialOn
def set_serialOn(self, value):
self._serialOn = value
print(self._serialOn)
if self.serialOn == True:
EVT_ID_VALUE = wx.NewId()
self.workerSerial = WorkerSerialThread(self, EVT_ID_VALUE, self.serialPort, self.addressList)
self.workerSerial.daemon = True
self.workerSerial.start()
elif self.serialOn == False:
self.workerSerial.stop()
del(self.workerSerial)
self.workerSerial = None
serialOn = property(get_serialOn, set_serialOn)
class WorkerSerialThread(threading.Thread):
def __init__(self, notify_window, id, port, addresslist):
threading.Thread.__init__(self)
self.id = id
self.serialPort = port
self.addressList = addresslist
self.counter = 0
self._notify_window = notify_window
self.abort = False
self.sch = SerialCommunication(self.serialPort, self.addressList)
try:
self.sch.PreLoadInfo()
except:
print('failed')
def run(self):
while not self.abort:
self.counter += 1
print('Serial Working on '+str(self.id))
self.sch.RegularLoadInfo()
#wx.PostEvent(self._notify_window, DataEvent(self.counter, self.id))
time.sleep(1)
def stop(self):
self.sch.board.stop()
self.abort = True
This question was finally solved with multiprocessing rather than threading of python.
In the manual of python-xbee, it mentioned that "... Make sure that updates to external state are thread-safe...". Also in the source code, threading was used.
So I guess in this case threading will cause problem.
Anyway, using multiprocessing it finally works.
I've searched this site (and others) up and down but I can't seem to find the right solution.
I have a client program that connects to a server and automatically sends a message every few seconds, as well as on user command. I'm using multiple threads for this.
Enter Tkinter: Once I hit the 'Connect' button, my UI freezes, either until the connection attempt times out or until the end of time, should the client connect to the server.
I've tried calling the thread from the button's command parameter, from inside the main loop, and outside the main loop. I've tried putting the main loop in a thread and then creating a new thread for the connection from there.
Nothing seems to be working...the UI continues to hang.
class EventSim(Frame):
def __init__(self, parent):
self.queue = Queue
Frame.__init__(self, parent)
self.parent = parent
def initUI(self,IP_Address,Port_Number,Events_Directory):
#...
self.Con_Button = Button(frame7,text='Connect', command = lambda:
self.connect(IP_Text.get(),Port_Text.get(),))
def connect(self,IP,Port):
ConnectionThread = Thread(eventsim.con_thread(IP,Port))
ConnectionThread.start()
def main():
root = Tk()
root.geometry("300x310+750+300")
Sim = EventSim(root)
eventsim.readconfig()
Sim.initUI(eventsim.ipa,eventsim.portnum,eventsim.event_dir)
root.mainloop()
You pass the result of eventsim.con_thread(IP,Port) to Thread(...) function, so it will wait until the execution of eventsim.con_thread(...) completes. Try changing:
def connect(self, IP, Port):
ConnectionThread = Thread(eventsim.con_thread(IP,Port))
ConnectionThread.start()
to:
def connect(self, IP, Port):
ConnectionThread = Thread(target=lambda ip=IP, port=Port: eventsim.con_thread(ip,port))
ConnectionThread.start()
I'm trying to multithread my Python application. This is how i thought the application would work:
A list of ipv4 addresses is created by the user
For each ipv4 address, the application establishes an SSH connection and logs in. This part would benefit from multithreading since each device takes about 10 seconds to complete. The ssh bit is all handled by my ConfDumper class.
in each thread, a bit of data is fetched from the network device and should be returned to the main thread (where there is a list of devices)
Once all threads are done, a result is presented.
Being new to Python and having no experience with multithreading, I've tried something like this:
import threading
import confDumper
class MyThread (threading.Thread):
device = None
# A device object is sent as agument
def __init__(self, device):
threading.Thread.__init__(self)
self.device = device
def run(self):
print "Starting scan..."
self.sshscan()
print "Exiting thread"
def sshscan(self):
s = confDumper.ConfDumper(self.device.mgmt_ip, self.device.username, self.device.password, self.device.enable_password)
t = s.getConf()
if t:
# We got the conf, return it to the main thread, somehow...
It seems to be working when I debug the code and step though the lines one by one, but once the thread is closed all results from the thread are lost. How do I return the result to the main thread?
You can use a Queue:
import Queue
import threading
import random
import time
class Worker(threading.Thread):
def __init__(self, queue):
super(Worker, self).__init__()
self._queue = queue
def run(self):
time.sleep(5.0 * random.random())
self._queue.put(str(self))
queue = Queue.Queue()
workers = [Worker(queue) for _ in xrange(10)]
for worker in workers:
worker.start()
for worker in workers:
worker.join()
while queue.qsize():
print queue.get()
This was much easier than I thought. As far as I can see you don't have to return anything, the object sent to the thread is the same as the source.
The documentation for python's select.select says:
Note that on Windows, it only works for sockets; on other operating
systems, it also works for other file types (in particular, on Unix,
it works on pipes).
My group is developing a simplistic multiplayer game using pygame and sockets. (We are not using Twisted or zeromq or any similar libraries; this being the only constraint).
Now, for the game design; we want the player to send data to the server when a key event occurs in the pygame screen. The client/player side's socket will otherwise be hooked to server and listen for changes occurring on other players' side. For this task, I'd need the pygame and socket to work parallely. I was recommended to use select module from several users on #python.
Can I do something like:
inp = [self.sock, pygame.event.get]
out = [self.server]
i, o, x = select.select( inp, out, [] )
If not, what should be the way to go?
You could use threads for this task. Is it necessary to process server messages and pygame events in series (not concurrently)? If so, you could do this:
class SocketListener(threading.Thread):
def __init__(self, sock, queue):
threading.Thread.__init__(self)
self.daemon = True
self.socket = sock
self.queue = queue
def run(self):
while True:
msg = self.socket.recv()
self.queue.put(msg)
class PygameHandler(threading.Thread):
def __init__(self, queue):
threading.Thread.__init__(self)
self.queue = queue
self.daemon = True
def run(self):
while True:
self.queue.put(pygame.event.wait())
queue = Queue.Queue()
PygameHandler(queue).start()
SocketListener(queue).start()
while True:
event = queue.get()
"""Process the event()"""
If not, you could process the events inside the run methods of the PygameHandler and SocketListener classes.