Develop Reference

How to detect if a Python Thread is being killed?

I have my own Thread called TimeBasedLogThread. I would like to fire a function my_function when the TimeBasedLogThread is being killed because the main process is exiting. I would like to do it from within this object. Is it possible to do so?
Here is my current approach:
class TimeBasedBufferingHandler(MemoryHandler):
# This is a logging-based handler that buffers logs to send
# them as emails
# the target of this handler is a SMTPHandler
def __init__(self, capacity=10, flushLevel=logging.ERROR, target=None,
flushOnClose=True, timeout=60):
MemoryHandler.__init__(self, capacity=capacity, flushLevel=flushLevel,
target=target, flushOnClose=flushOnClose)
self.timeout = timeout # in seconds (as time.time())
def flush(self):
# Send the emails that are younger than timeout, all together
# in the same email
class TimeBasedLogThread(Thread):
def __init__(self, handler, timeout=60):
Thread.__init__(self)
self.handler = handler
self.timeout = timeout
def run(self):
while True:
self.handler.flush()
time.sleep(self.timeout)
def my_function(self):
print("my_function is being called")
self.handler.flush()
def setup_thread():
smtp_handler = SMTPHandler()
new_thread = TimeBasedLogThread(smtp_handler, timeout=10)
new_thread.start()
In my main thread, I have:
setup_thread()
logging.error("DEBUG_0")
time.sleep(5)
logging.error("DEBUG_1")
time.sleep(5)
logging.error("DEBUG_2")
The time.sleep(5) releases the main thread 5 seconds before the timeout of my other thread. So, I receive the first 2 emails with "DEBUG_0" and "DEBUG_1", but not the last one "DEBUG_2" because the main process exits before the timeout has finished.
I would like to link the class TimeBasedLogThread and the function my_function that will flush (send the emails) before exiting. How can I do that? I looked at the source code of threading but I did not understand what method I could use.

Build your function as a Thread too. (Ex: AfterDeadThread)
You have two strategy here:
TimeBasedLogThread call AfterDeadThread before die
AfterDeadThread check if TimeBasedLogThread is alive or not, if not it will run some methods

Extend run() method (representing the thread’s activity) to fire the on_terminate handler passed to custom thread’s constructor as keyword argument.
On a slightly changed custom thread class (for demonstration):
from threading import Thread
import time, random
class TimeBasedLogThread(Thread):
def __init__(self, handler, timeout=2, on_terminate=None):
Thread.__init__(self)
self.handler = handler
self.timeout = timeout
self.terminate_handler = on_terminate
def run(self):
while True:
num = self.handler()
if num > 5:
break
time.sleep(self.timeout)
print(num)
if self.terminate_handler:
self.terminate_handler()
def my_term_function():
print("my_function is being called")
f = lambda: random.randint(3, 10)
tlog_thread = TimeBasedLogThread(f, on_terminate=my_term_function)
tlog_thread.start()
tlog_thread.join()
Sample output:
3
4
5
4
5
my_function is being called

pyserial time based repeated data request

I am writing an application in python to acquire data using a serial communication. I use the pyserial library for establishing the communication. What is the best approach to request data in an interval (eg every 2 seconds). I always have to the send a request, wait for the answer and start the process again.

if this a "slow" process, that does not accurate time precision, use a while loop and time.sleep(2) to timeout the process for 2 seconds.

I thought about using a separate thread to prevent the rest of the applicaiton from freezing. The thread takes a function which requests data from the instrument.
class ReadingThread(Thread):
'''Used to read out from the instrument, interrupted by an interval'''
def __init__(self, controller, lock, function, queue_out, interval=3 , *args, **kwargs):
Thread.__init__(self)
self.lock = lock
self.function = function
self.queue_out = queue_out
self.interval = interval
self.args = args
self.kwargs = kwargs
self.is_running = True
def run(self):
while self.is_running:
with self.lock:
try:
result = self.function(self.args, self.kwargs)
except Exception as e:
print(e)
else:
self.queue_out.put(result)
time.sleep(self.interval)
def stop(self):
self.is_running = False

Python: Continuously running a section of a script

I have written a python script that idles waiting for gmail to push a notification indicating that an email has been received. Then the contents are parsed and a database is searched, then database data is emailed back to the original sender.
Currently once an email is received, the script cannot process another email until the script has emailed back. I am wondering if there is a way for the script to be continually listening for an email. As at this point in time if two emails are received at similar times the second one will not be processed.
I think multi-threading might be a solution but I am not sure. Possibly create a new thread for the processEmail.py section of code below?
Sorry if I have explained badly, I am struggling to explain this adequately, feel free to ask for more information.
EDIT: Instead of down voting me could you help me by commenting telling me what more information I should include?
EDIT 2: Here is a code example, I am trying to have the ability to still listen for an email while an email is being processed in processEmail.py
import imaplib2
import time
import subprocess
from threading import *
from subprocess import call
import processEmail
class Idler(object):
def __init__(self, conn):
self.thread = Thread(target=self.idle)
self.M = conn
self.event = Event()
def start(self):
self.thread.start()
def stop(self):
self.event.set()
def join(self):
self.thread.join()
def idle(self):
while True:
if self.event.isSet():
return
self.needsync = False
def callback(args):
if not self.event.isSet():
self.needsync = True
self.event.set()
self.M.idle(callback=callback)
self.event.wait()
if self.needsync:
self.event.clear()
self.dosync()
def dosync(self):
print "An email has been received, please wait...\n"
self.execute()
def execute(self):
processEmail.main()
M = imaplib2.IMAP4_SSL("imap.gmail.com")
M.login("email_address","email_pass")
M.select("Folder")
idler = Idler(M)
idler.start()
x = False
while not x: time.sleep(0.1)

You can directly inherit from threading.Thread and override its run method:
class SomeTask(threading.Thread):
def run(self):
# Will be executed in separate thread
Start the thread via its start method.
Communication between threads should be handled via queues.

Python: return result from a thread

I'm trying to multithread my Python application. This is how i thought the application would work:
A list of ipv4 addresses is created by the user
For each ipv4 address, the application establishes an SSH connection and logs in. This part would benefit from multithreading since each device takes about 10 seconds to complete. The ssh bit is all handled by my ConfDumper class.
in each thread, a bit of data is fetched from the network device and should be returned to the main thread (where there is a list of devices)
Once all threads are done, a result is presented.
Being new to Python and having no experience with multithreading, I've tried something like this:
import threading
import confDumper
class MyThread (threading.Thread):
device = None
# A device object is sent as agument
def __init__(self, device):
threading.Thread.__init__(self)
self.device = device
def run(self):
print "Starting scan..."
self.sshscan()
print "Exiting thread"
def sshscan(self):
s = confDumper.ConfDumper(self.device.mgmt_ip, self.device.username, self.device.password, self.device.enable_password)
t = s.getConf()
if t:
# We got the conf, return it to the main thread, somehow...
It seems to be working when I debug the code and step though the lines one by one, but once the thread is closed all results from the thread are lost. How do I return the result to the main thread?

You can use a Queue:
import Queue
import threading
import random
import time
class Worker(threading.Thread):
def __init__(self, queue):
super(Worker, self).__init__()
self._queue = queue
def run(self):
time.sleep(5.0 * random.random())
self._queue.put(str(self))
queue = Queue.Queue()
workers = [Worker(queue) for _ in xrange(10)]
for worker in workers:
worker.start()
for worker in workers:
worker.join()
while queue.qsize():
print queue.get()

This was much easier than I thought. As far as I can see you don't have to return anything, the object sent to the thread is the same as the source.

Thread doesn't close on exiting tk window

I have an application written in Python, using Tkinter. One of the features allows a serial port to be opened, after which any messages received over the serial port are displayed in a text window. This works fine. The problem comes when I close the window, which doesn't kill the thread monitoring the serial port. It then has to be killed manually (alternatively, unplugging the USB-serial cable causes an exception which kills the process).
I assume I'm missing something simple here, but I would have thought closing the application would close all associated threads. I can't seem to find anything about this in the documentation, but I'm probably looking in the wrong place?
Code for the serial thread in case it's relevant:
class SerialThread(threading.Thread):
def __init__(self, queue, sp):
threading.Thread.__init__(self)
self.queue = queue
self.ser_handle = sp;
def run(self):
while True:
if self.ser_handle.inWaiting():
text = self.ser_handle.readline(self.ser_handle.inWaiting())
self.queue.put(text)
time.sleep(0.2)

You must have a way to ask the thread for stop, you can accomplish that using: threading.Event.
class SerialThread(threading.Thread):
def __init__(self, queue, sp):
threading.Thread.__init__(self)
self.queue = queue
self.event = threading.Event() # An event object.
self.ser_handle = sp;
def stop(self):
self.event.set()
def run(self):
while not self.event.isSet():
if self.ser_handle.inWaiting():
text = self.ser_handle.readline(self.ser_handle.inWaiting())
self.queue.put(text)
time.sleep(0.2)
Then in the on close event of your window, call your_thread.stop()