Develop Reference

Problem with closing a running python thread

I have a single-line encrypted code in form of string and a function that decodes & execute the encrypted code (the code establishes backdoor connection to another computer in the network, but that isn't very relevent here) Once the encrypted code runs the program won't proceed to next line because the code does not stop (in order to keep the connection alive).
What I want to do is run the code for 10 seconds, then terminate it, and re-run it.
But I am not being able to terminate the code because it's encrypted and can't be changed. I thought I could run it on a separate thread then forcibly terminate it after 10 seconds, but the problem is all the solutions mentioned on the internet would require changing the encrypted code slightly.
Python does not seem to have a way to forcibly close a running thread after 'n' seconds, which would solve my problem very easily.
Following code demonstrates my problem:
import threading, time
encrypted_code = ")'cba'(tnirp :eurT elihw"
def decrypt_and_execute(encrypted_code):
exec(encrypted_code[::-1])
def run(encrypted_code, run_time):
my_thread = threading.Thread(target=decrypt_and_execute, args=(encrypted_code,))
my_thread.start()
time.sleep(run_time)
# CODE TO STOP 'my_thread' without changing 'encrypted_code' NEEDED HERE#
while True:
threading.Thread(target=run, args=(encrypted_code, 10)).start()
time.sleep(10)
Above was my attempted solution (encrypted code here is an infinite while loop that keeps printing abc).
Any code I tried to stop 'my_thread' without closing the whole application, did not work.

Maybe that technique of a thread with traces works for you, is a Thread subclass that you can stop. From https://www.geeksforgeeks.org/python-different-ways-to-kill-a-thread/
Or use subprocesses, they have terminate. That same post covers that too nicely.
Edit: for some reason, this failed to kill the thread in the case in question. The multiprocess module solved the issue: killing processes is a reliable way to stop things on all operating systems, and multiprocess module has a spawn method that gives a Thread like API for subprocesses, and can start new processes with Python code also on Windows without having Python installed.
Anyway for other cases this technique with threading should work too. Threads are rarely actually needed but sometimes they truly are.
# Python program using
# traces to kill threads
import sys
import trace
import threading
import time
class thread_with_trace(threading.Thread):
def __init__(self, *args, **keywords):
threading.Thread.__init__(self, *args, **keywords)
self.killed = False
def start(self):
self.__run_backup = self.run
self.run = self.__run
threading.Thread.start(self)
def __run(self):
sys.settrace(self.globaltrace)
self.__run_backup()
self.run = self.__run_backup
def globaltrace(self, frame, event, arg):
if event == 'call':
return self.localtrace
else:
return None
def localtrace(self, frame, event, arg):
if self.killed:
if event == 'line':
raise SystemExit()
return self.localtrace
def kill(self):
self.killed = True
def func():
while True:
print('thread running')
t1 = thread_with_trace(target = func)
t1.start()
time.sleep(2)
t1.kill()
t1.join()
if not t1.isAlive():
print('thread killed')

Python Paramiko Ctrl Z is not working for threading [duplicate]

I am testing Python threading with the following script:
import threading
class FirstThread (threading.Thread):
def run (self):
while True:
print 'first'
class SecondThread (threading.Thread):
def run (self):
while True:
print 'second'
FirstThread().start()
SecondThread().start()
This is running in Python 2.7 on Kubuntu 11.10. Ctrl+C will not kill it. I also tried adding a handler for system signals, but that did not help:
import signal
import sys
def signal_handler(signal, frame):
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
To kill the process I am killing it by PID after sending the program to the background with Ctrl+Z, which isn't being ignored. Why is Ctrl+C being ignored so persistently? How can I resolve this?

Ctrl+C terminates the main thread, but because your threads aren't in daemon mode, they keep running, and that keeps the process alive. We can make them daemons:
f = FirstThread()
f.daemon = True
f.start()
s = SecondThread()
s.daemon = True
s.start()
But then there's another problem - once the main thread has started your threads, there's nothing else for it to do. So it exits, and the threads are destroyed instantly. So let's keep the main thread alive:
import time
while True:
time.sleep(1)
Now it will keep print 'first' and 'second' until you hit Ctrl+C.
Edit: as commenters have pointed out, the daemon threads may not get a chance to clean up things like temporary files. If you need that, then catch the KeyboardInterrupt on the main thread and have it co-ordinate cleanup and shutdown. But in many cases, letting daemon threads die suddenly is probably good enough.

KeyboardInterrupt and signals are only seen by the process (ie the main thread)... Have a look at Ctrl-c i.e. KeyboardInterrupt to kill threads in python

I think it's best to call join() on your threads when you expect them to die. I've taken the liberty to make the change your loops to end (you can add whatever cleanup needs are required to there as well). The variable die is checked on each pass and when it's True, the program exits.
import threading
import time
class MyThread (threading.Thread):
die = False
def __init__(self, name):
threading.Thread.__init__(self)
self.name = name
def run (self):
while not self.die:
time.sleep(1)
print (self.name)
def join(self):
self.die = True
super().join()
if __name__ == '__main__':
f = MyThread('first')
f.start()
s = MyThread('second')
s.start()
try:
while True:
time.sleep(2)
except KeyboardInterrupt:
f.join()
s.join()

An improved version of #Thomas K's answer:
Defining an assistant function is_any_thread_alive() according to this gist, which can terminates the main() automatically.
Example codes:
import threading
def job1():
...
def job2():
...
def is_any_thread_alive(threads):
return True in [t.is_alive() for t in threads]
if __name__ == "__main__":
...
t1 = threading.Thread(target=job1,daemon=True)
t2 = threading.Thread(target=job2,daemon=True)
t1.start()
t2.start()
while is_any_thread_alive([t1,t2]):
time.sleep(0)

One simple 'gotcha' to beware of, are you sure CAPS LOCK isn't on?
I was running a Python script in the Thonny IDE on a Pi4. With CAPS LOCK on, Ctrl+Shift+C is passed to the keyboard buffer, not Ctrl+C.

Interrupting a thread in Python with a KeyboardException in the main thread

I have a few classes that look more or less like this:
import threading
import time
class Foo():
def __init__(self, interval, callbacks):
self.thread = threading.Thread(target=self.loop)
self.interval = interval
self.thread_stop = threading.Event()
self.callbacks = callbacks
def loop():
while not self.thread_stop.is_set():
#do some stuff...
for callback in self.callbacks():
callback()
time.sleep(self.interval)
def start(self):
self.thread.start()
def kill(self):
self.thread_stop.set()
Which I am using from my main thread like this:
interval = someinterval
callbacks = [some callbacks]
f = Foo(interval, callbacks)
try:
f.start()
except KeyboardInterrupt:
f.kill()
raise
I would like a KeyboardInterrupt to kill the thread after all the callbacks have been completed, but before the loop repeats. Currently they are ignored and I have to resort to killing the terminal process that the program is running in.
I saw the idea of using threading.Event from this post, but it appears like I'm doing it incorrectly, and it's making working on this project a pretty large hassle.
I don't know if it may be relevant, but the callbacks I'm passing access data from the Internet and make heavy use of the retrying decorator to deal with unreliable connections.
EDIT
After everyone's help, the loop now looks like this inside Foo:
def thread_loop(self):
while not self.thread_stop.is_set():
# do some stuff
# call the callbacks
self.thread_stop.wait(self.interval)
This is kind of a solution, although it isn't ideal. This code runs on PythonAnywhere and the price of the account is by CPU time. I'll have to see how much this uses over the course of a day with the constant waking and sleeping of threads, but it at least solves the main issue

I think your problem is that you have a try-except-block around f.start(), but that returns immediately, so you aren't going to catch KeyboardInterrupts after the thread was started.
You could try adding a while-loop at the bottom of your program like this:
f.start()
try:
while True:
time.sleep(0.1)
except KeyboardInterrupt:
f.kill()
raise
This isn't exactly the most elegant solution, but it should work.

Thanks to #shx2 and #jazzpi for putting together the two separate pieces of the puzzle.
so the final code is
import threading
import time
class Foo():
def __init__(self, interval, callbacks):
self.thread = threading.Thread(target=self.loop)
self.interval = interval
self.thread_stop = threading.Event()
self.callbacks = callbacks
def loop():
while not self.thread_stop.is_set():
#do some stuff...
for callback in self.callbacks():
callback()
self.thread_stop.wait(self.interval)
def start(self):
self.thread.start()
def kill(self):
self.thread_stop.set()
And then in main
interval = someinterval
callbacks = [some, callbacks]
f = Foo(interval, callbacks)
f.start()
try:
while True:
time.sleep(0.1)
except KeyboardInterrupt:
f.kill()
raise

#jazzpi's answer correctly addresses the issue you're having in the main thread.
As to the sleep in thread's loop, you can simply replace the call to sleep with a call to self.thread_stop.wait(self.interval).
This way, your thread wakes up as soon as the stop event is set, or after waiting (i.e. sleeping) for self.interval seconds. (Event docs)

python asyncronous thread exception handling

I'm trying to implement a timeout functionality in Python.
It works by wrapping functions with a function decorator that calls the function as a thread but also calls a 'watchdog' thread that will raise an exception in the function thread after a specified period has elapsed.
It currently works for threads that don't sleep. During the do_rand call, I suspect the 'asynchronous' exception is actually being called after the time.sleep call and after the execution has moved beyond the try/except block, as this would explain the Unhandled exception in thread started by error. Additionally, the error from the do_rand call is generated 7 seconds after the call (the duration of time.sleep).
How would I go about 'waking' a thread up (using ctypes?) to get it to respond to an asynchronous exception ?
Or possibly a different approach altogether ?
Code:
# Import System libraries
import ctypes
import random
import sys
import threading
import time
class TimeoutException(Exception):
pass
def terminate_thread(thread, exc_type = SystemExit):
"""Terminates a python thread from another thread.
:param thread: a threading.Thread instance
"""
if not thread.isAlive():
return
exc = ctypes.py_object(exc_type)
res = ctypes.pythonapi.PyThreadState_SetAsyncExc(ctypes.c_long(thread.ident), exc)
if res == 0:
raise ValueError("nonexistent thread id")
elif res > 1:
# """if it returns a number greater than one, you're in trouble,
# and you should call it again with exc=NULL to revert the effect"""
ctypes.pythonapi.PyThreadState_SetAsyncExc(thread.ident, None)
raise SystemError("PyThreadState_SetAsyncExc failed")
class timeout_thread(threading.Thread):
def __init__(self, interval, target_thread):
super(timeout_thread, self).__init__()
self.interval = interval
self.target_thread = target_thread
self.done_event = threading.Event()
self.done_event.clear()
def run(self):
timeout = not self.done_event.wait(self.interval)
if timeout:
terminate_thread(self.target_thread, TimeoutException)
class timeout_wrapper(object):
def __init__(self, interval = 300):
self.interval = interval
def __call__(self, f):
def wrap_func(*args, **kwargs):
thread = threading.Thread(target = f, args = args, kwargs = kwargs)
thread.setDaemon(True)
timeout_ticker = timeout_thread(self.interval, thread)
timeout_ticker.setDaemon(True)
timeout_ticker.start()
thread.start()
thread.join()
timeout_ticker.done_event.set()
return wrap_func
#timeout_wrapper(2)
def print_guvnah():
try:
while True:
print "guvnah"
except TimeoutException:
print "blimey"
def print_hello():
try:
while True:
print "hello"
except TimeoutException:
print "Whoops, looks like I timed out"
def do_rand(*args):
try:
rand_num = 7 #random.randint(0, 10)
rand_pause = 7 #random.randint(0, 5)
print "Got rand: %d" % rand_num
print "Waiting for %d seconds" % rand_pause
time.sleep(rand_pause)
except TimeoutException:
print "Waited too long"
print_guvnah()
timeout_wrapper(3)(print_hello)()
timeout_wrapper(2)(do_rand)()

The problem is that time.sleep blocks. And it blocks really hard, so the only thing that can actually interrupt it is process signals. But the code with it gets really messy and in some cases even signals don't work ( when for example you are doing blocking socket.recv(), see this: recv() is not interrupted by a signal in multithreaded environment ).
So generally interrupting a thread (without killing entire process) cannot be done (not to mention that someone can simply override your signal handling from a thread).
But in this particular case instead of using time.sleep you can use Event class from threading module:
Thread 1
from threading import Event
ev = Event()
ev.clear()
state = ev.wait(rand_pause) # this blocks until timeout or .set() call
Thread 2 (make sure it has access to the same ev instance)
ev.set() # this will unlock .wait above
Note that state will be the internal state of the event. Thus state == True will mean that it was unlocked with .set() while state == False will mean that timeout occured.
Read more about events here:
http://docs.python.org/2/library/threading.html#event-objects

You'd need to use something other than sleep, or you'd need to send a signal to the other thread in order to make it wake up.
One option I've used is to set up a pair of file descriptors and use select or poll instead of sleep, this lets you write something to the file descriptor to wake up the other thread. Alternatively you just wear waiting until the sleep finishes if all you need is for the operation to error out because it took too long and nothing else is depending on it.

How to stop a looping thread in Python?

What's the proper way to tell a looping thread to stop looping?
I have a fairly simple program that pings a specified host in a separate threading.Thread class. In this class it sleeps 60 seconds, the runs again until the application quits.
I'd like to implement a 'Stop' button in my wx.Frame to ask the looping thread to stop. It doesn't need to end the thread right away, it can just stop looping once it wakes up.
Here is my threading class (note: I haven't implemented looping yet, but it would likely fall under the run method in PingAssets)
class PingAssets(threading.Thread):
def __init__(self, threadNum, asset, window):
threading.Thread.__init__(self)
self.threadNum = threadNum
self.window = window
self.asset = asset
def run(self):
config = controller.getConfig()
fmt = config['timefmt']
start_time = datetime.now().strftime(fmt)
try:
if onlinecheck.check_status(self.asset):
status = "online"
else:
status = "offline"
except socket.gaierror:
status = "an invalid asset tag."
msg =("{}: {} is {}. \n".format(start_time, self.asset, status))
wx.CallAfter(self.window.Logger, msg)
And in my wxPyhton Frame I have this function called from a Start button:
def CheckAsset(self, asset):
self.count += 1
thread = PingAssets(self.count, asset, self)
self.threads.append(thread)
thread.start()

Threaded stoppable function
Instead of subclassing threading.Thread, one can modify the function to allow
stopping by a flag.
We need an object, accessible to running function, to which we set the flag to stop running.
We can use threading.currentThread() object.
import threading
import time
def doit(arg):
t = threading.currentThread()
while getattr(t, "do_run", True):
print ("working on %s" % arg)
time.sleep(1)
print("Stopping as you wish.")
def main():
t = threading.Thread(target=doit, args=("task",))
t.start()
time.sleep(5)
t.do_run = False
if __name__ == "__main__":
main()
The trick is, that the running thread can have attached additional properties. The solution builds
on assumptions:
the thread has a property "do_run" with default value True
driving parent process can assign to started thread the property "do_run" to False.
Running the code, we get following output:
$ python stopthread.py
working on task
working on task
working on task
working on task
working on task
Stopping as you wish.
Pill to kill - using Event
Other alternative is to use threading.Event as function argument. It is by
default False, but external process can "set it" (to True) and function can
learn about it using wait(timeout) function.
We can wait with zero timeout, but we can also use it as the sleeping timer (used below).
def doit(stop_event, arg):
while not stop_event.wait(1):
print ("working on %s" % arg)
print("Stopping as you wish.")
def main():
pill2kill = threading.Event()
t = threading.Thread(target=doit, args=(pill2kill, "task"))
t.start()
time.sleep(5)
pill2kill.set()
t.join()
Edit: I tried this in Python 3.6. stop_event.wait() blocks the event (and so the while loop) until release. It does not return a boolean value. Using stop_event.is_set() works instead.
Stopping multiple threads with one pill
Advantage of pill to kill is better seen, if we have to stop multiple threads
at once, as one pill will work for all.
The doit will not change at all, only the main handles the threads a bit differently.
def main():
pill2kill = threading.Event()
tasks = ["task ONE", "task TWO", "task THREE"]
def thread_gen(pill2kill, tasks):
for task in tasks:
t = threading.Thread(target=doit, args=(pill2kill, task))
yield t
threads = list(thread_gen(pill2kill, tasks))
for thread in threads:
thread.start()
time.sleep(5)
pill2kill.set()
for thread in threads:
thread.join()

This has been asked before on Stack. See the following links:
Is there any way to kill a Thread in Python?
Stopping a thread after a certain amount of time
Basically you just need to set up the thread with a stop function that sets a sentinel value that the thread will check. In your case, you'll have the something in your loop check the sentinel value to see if it's changed and if it has, the loop can break and the thread can die.

I read the other questions on Stack but I was still a little confused on communicating across classes. Here is how I approached it:
I use a list to hold all my threads in the __init__ method of my wxFrame class: self.threads = []
As recommended in How to stop a looping thread in Python? I use a signal in my thread class which is set to True when initializing the threading class.
class PingAssets(threading.Thread):
def __init__(self, threadNum, asset, window):
threading.Thread.__init__(self)
self.threadNum = threadNum
self.window = window
self.asset = asset
self.signal = True
def run(self):
while self.signal:
do_stuff()
sleep()
and I can stop these threads by iterating over my threads:
def OnStop(self, e):
for t in self.threads:
t.signal = False

I had a different approach. I've sub-classed a Thread class and in the constructor I've created an Event object. Then I've written custom join() method, which first sets this event and then calls a parent's version of itself.
Here is my class, I'm using for serial port communication in wxPython app:
import wx, threading, serial, Events, Queue
class PumpThread(threading.Thread):
def __init__ (self, port, queue, parent):
super(PumpThread, self).__init__()
self.port = port
self.queue = queue
self.parent = parent
self.serial = serial.Serial()
self.serial.port = self.port
self.serial.timeout = 0.5
self.serial.baudrate = 9600
self.serial.parity = 'N'
self.stopRequest = threading.Event()
def run (self):
try:
self.serial.open()
except Exception, ex:
print ("[ERROR]\tUnable to open port {}".format(self.port))
print ("[ERROR]\t{}\n\n{}".format(ex.message, ex.traceback))
self.stopRequest.set()
else:
print ("[INFO]\tListening port {}".format(self.port))
self.serial.write("FLOW?\r")
while not self.stopRequest.isSet():
msg = ''
if not self.queue.empty():
try:
command = self.queue.get()
self.serial.write(command)
except Queue.Empty:
continue
while self.serial.inWaiting():
char = self.serial.read(1)
if '\r' in char and len(msg) > 1:
char = ''
#~ print('[DATA]\t{}'.format(msg))
event = Events.PumpDataEvent(Events.SERIALRX, wx.ID_ANY, msg)
wx.PostEvent(self.parent, event)
msg = ''
break
msg += char
self.serial.close()
def join (self, timeout=None):
self.stopRequest.set()
super(PumpThread, self).join(timeout)
def SetPort (self, serial):
self.serial = serial
def Write (self, msg):
if self.serial.is_open:
self.queue.put(msg)
else:
print("[ERROR]\tPort {} is not open!".format(self.port))
def Stop(self):
if self.isAlive():
self.join()
The Queue is used for sending messages to the port and main loop takes responses back. I've used no serial.readline() method, because of different end-line char, and I have found the usage of io classes to be too much fuss.

Depends on what you run in that thread.
If that's your code, then you can implement a stop condition (see other answers).
However, if what you want is to run someone else's code, then you should fork and start a process. Like this:
import multiprocessing
proc = multiprocessing.Process(target=your_proc_function, args=())
proc.start()
now, whenever you want to stop that process, send it a SIGTERM like this:
proc.terminate()
proc.join()
And it's not slow: fractions of a second.
Enjoy :)

My solution is:
import threading, time
def a():
t = threading.currentThread()
while getattr(t, "do_run", True):
print('Do something')
time.sleep(1)
def getThreadByName(name):
threads = threading.enumerate() #Threads list
for thread in threads:
if thread.name == name:
return thread
threading.Thread(target=a, name='228').start() #Init thread
t = getThreadByName('228') #Get thread by name
time.sleep(5)
t.do_run = False #Signal to stop thread
t.join()

I find it useful to have a class, derived from threading.Thread, to encapsulate my thread functionality. You simply provide your own main loop in an overridden version of run() in this class. Calling start() arranges for the object’s run() method to be invoked in a separate thread.
Inside the main loop, periodically check whether a threading.Event has been set. Such an event is thread-safe.
Inside this class, you have your own join() method that sets the stop event object before calling the join() method of the base class. It can optionally take a time value to pass to the base class's join() method to ensure your thread is terminated in a short amount of time.
import threading
import time
class MyThread(threading.Thread):
def __init__(self, sleep_time=0.1):
self._stop_event = threading.Event()
self._sleep_time = sleep_time
"""call base class constructor"""
super().__init__()
def run(self):
"""main control loop"""
while not self._stop_event.isSet():
#do work
print("hi")
self._stop_event.wait(self._sleep_time)
def join(self, timeout=None):
"""set stop event and join within a given time period"""
self._stop_event.set()
super().join(timeout)
if __name__ == "__main__":
t = MyThread()
t.start()
time.sleep(5)
t.join(1) #wait 1s max
Having a small sleep inside the main loop before checking the threading.Event is less CPU intensive than looping continuously. You can have a default sleep time (e.g. 0.1s), but you can also pass the value in the constructor.

Sometimes you don't have control over the running target. In those cases you can use signal.pthread_kill to send a stop signal.
from signal import pthread_kill, SIGTSTP
from threading import Thread
from itertools import count
from time import sleep
def target():
for num in count():
print(num)
sleep(1)
thread = Thread(target=target)
thread.start()
sleep(5)
pthread_kill(thread.ident, SIGTSTP)
result
0
1
2
3
4
[14]+ Stopped