Develop Reference

Python : call a function and wait for its thread(s) to finish

I am using Python to script a gdb session. My goal is to run a function whenever a breakpoint is hit, using gdb.events.stop.connect.
Here is my code in essence, which I call from gdb by running source main.py :
RESULT = None
def breakpoint_handler(event):
global RESULT
RESULT = do_something(event) # boolean
sleep(1)
def main():
global RESULT
gdb.events.stop.connect(breakpoint_handler)
command = "some gdb command"
gdb.execute(command)
sleep(1)
if RESULT:
# do something
pass
else:
# something else
pass
main()
This code works, but only thanks to the sleep calls, that tries to make sure the gdb.execute(command)call actually finish executing the gdb command before continuing. There is only little documentation but my guess is that this function is threaded, which explains the effectiveness of the waitas a fix.
Is there a way to make this cleaner by waiting for the thread created by gdb.execute to finish processing, without having access to the body of this method ?

You can set an event inside your callback and wait for it in your main thread. Here is an example, with the callback being called after 5 seconds by timer:
import threading
def some_callback():
event.set()
event = threading.Event()
threading.Timer(5, some_callback).start()
event.wait()

Python threading - blocking operation - terminate execution

I have a python program like this:
from threading import Thread
def foo():
while True:
blocking_function() #Actually waiting for a message on a socket
def run():
Thread(target=foo).start()
run()
This program does not terminate with KeyboardInterrupt, due to the main Thread exiting before a Thread running foo() has a chance to terminate. I tried keeping the main thread alive with just running while True loop after calling run() but that also doesn't exit the program (blocking_function() just blocks the thread from running I guess, waits for the message). Also tried catching KeyboardInterrupt exception in main thread and call sys.exit(0) - same outcome (I would actually expect it to kill the thread running foo(), but apparently it doesn't)
Now, I could simply timeout the execution of blocking_function() but that's no fun. Can I unblock it on KeyboardInterrupt or anything similar?
Main goal: Terminate the program with blocked thread on Ctrl+C

Maybe a little bit of a workaround, but you could use thread instead of threading. This is not really advised, but if it suits you and your program, why not.
You will need to keep your program running, otherwise the thread exits right after run()
import thread, time
def foo():
while True:
blocking_function() #Actually waiting for a message on a socket
def run():
thread.start_new_thread(foo, ())
run()
while True:
#Keep the main thread alive
time.sleep(1)

Python: Timer, how to stop thread when program ends?

I have a function I'm calling every 5 seconds like such:
def check_buzz(super_buzz_words):
print 'Checking buzz'
t = Timer(5.0, check_buzz, args=(super_buzz_words,))
t.dameon = True
t.start()
buzz_word = get_buzz_word()
if buzz_word is not 'fail':
super_buzz_words.put(buzz_word)
main()
check_buzz()
I'm exiting the script by either catching a KeyboardInterrupt or by catching a System exit and calling this:
sys.exit('\nShutting Down\n')
I'm also restarting the program every so often by calling:
execv(sys.executable, [sys.executable] + sys.argv)
My question is, how do I get that timer thread to shut off? If I keyboard interrupt, the timer keeps going.

I think you just spelled daemon wrong, it should have been:
t.daemon = True
Then sys.exit() should work

Expanding on the answer from notorious.no, and the comment asking:
How can I call t.cancel() if I have no access to t oustide the
function?
Give the Timer thread a distinct name when you first create it:
import threading
def check_buzz(super_buzz_words):
print 'Checking buzz'
t = Timer(5.0, check_buzz, args=(super_buzz_words,))
t.daemon = True
t.name = "check_buzz_daemon"
t.start()
Although the local variable t soon goes out of scope, the Timer thread that t pointed to still exists and still retains the name assigned to it.
Your atexit-registered method can then identify this thread by its name and cancel it:
from atexit import register
def all_done():
for thr in threading._enumerate():
if thr.name == "check_buzz_daemon":
if thr.is_alive():
thr.cancel()
thr.join()
register(all_done)
Calling join() after calling cancel()is based on a StackOverflow answer by Cédric Julien.
HOWEVER, your thread is set to be a Daemon. According to this StackOverflow post, daemon threads do not need to be explicitly terminated.

from atexit import register
def all_done():
if t.is_alive():
# do something that will close your thread gracefully
register(all_done)
Basically when your code is about to exit, it will fire one last function and this is where you will check if your thread is still running. If it is, do something that will either cancel the transaction or otherwise exit gracefully. In general, it's best to let threads finish by themselves, but if it's not doing anything important (please note the emphasis) than you can just do t.cancel(). Design your code so that threads will finish on their own if possible.

Another way would be to use the Queue() module to send and recieve info from a thread using the .put() outside the thread and the .get() inside the thread.
What you can also do is create a txt file and make program write to it when you exit And put an if statement in the thread function to check it after each iteration (this is not a really good solution but it also works)
I would have put a code exemple but i am writing from mobile sorry

Have the script achieved it was supposed to please check i am a beginner

i am just a beginner in python.What i try'ed to achieve is making two threads and calling different functions in different thread.I made the function in thread 1 to execute a function for 60 seconds and thread 2 to execute simultaneously and wait the main thread to wait for 70 second.When thread one exits it should also exit the second thread and finally control should come to main thread and again the call to thread one and thread two should go and same procedure repeat.
I try'ed achieving it using the below thread but i thing i was not able to
I have made a script in which i have started two thread named thread 1 and thread 2.
In thread 1 one function will run named func1 and in thread 2 function 2 will run named func 2.
Thread 1 will execute a command and wait for 60 seconds.
Thread 2 will run only till thread 1 is running .
Again after that the same process continues in while after a break of 80 Seconds.
I am a beginner in python.
Please suggest what all i have done wrong and how to correct it.
#!/usr/bin/python
import threading
import time
import subprocess
import datetime
import os
import thread
thread.start_new_thread( print_time, (None, None))
thread.start_new_thread( print_time1, (None, None))
command= "strace -o /root/Desktop/a.txt -c ./server"
final_dir = "/root/Desktop"
exitflag = 0
# Define a function for the thread
def print_time(*args):
os.chdir(final_dir)
print "IN first thread"
proc = subprocess.Popen(command,shell=True,stdout=subprocess.PIPE, stderr=subprocess.PIPE)
proc.wait(70)
exitflag=1
def print_time1(*args):
print "In second thread"
global exitflag
while exitflag:
thread.exit()
#proc = subprocess.Popen(command1,shell=True,stdout=subprocess.PIPE, sterr=subprocess.PIPE)
# Create two threads as follows
try:
while (1):
t1=threading.Thread(target=print_time)
t1.start()
t2=threading.Thread(target=print_time1)
t2=start()
time.sleep(80)
z = t1.isAlive()
z1 = t2.isAlive()
if z:
z.exit()
if z1:
z1.exit()
threading.Thread(target=print_time1).start()
threading.Thread(target=print_time1).start()
print "In try"
except:
print "Error: unable to start thread"

I can't get the example to run, I need to change the function definitons to
def print_time(*args)
and the thread call to
thread.start_new_thread( print_time, (None, None))
then you have a number of problems
you are currently not waiting for the exitflag to be set in the second thread, it justs runs to completion.
to share variables between thread you need to declare them global in the thread, otherwise you get a local variable.
thread.exit() in the print_time1 function generates an error
Your timings in the problem description and in the code does not match
So, to solve issue 1-3 for print_time1 declare it like (removing exit from the end)
def print_time1(*args):
global exitflag
while exitflag == 0: # wait for print_time
next
# Do stuff when thread is finalizing
But, check the doc for the thread module (https://docs.python.org/2/library/thread.html), "[...] however, you should consider using the high-level threading module instead."
import threading
...
while(1):
threading.Thread(target=print_time).start()
threading.Thread(target=print_time1).start()
time.sleep(80)
One final tought about the code is that you should check that the threads are actually finalized before starting new ones. Right now two new threads are started every 80 sec, this is regardless of whether the old threads have run to completion or not. Unless this is the wanted behaviour I would add a check for that in the while loop. Also while you are at it, move the try clause to be as close as possible to where the exception might be raised, i.e. where the threads are created. The way you have it now with the try encapsulating a while loop is not very common and imo not very pythonic (increases complexity of code)

Executing a function in new process with Python

Is there a way to do this? I was thinking maybe use subprocess or multiprocessing but I am not sure how to do this?
I don't have any code for an example because it is just a general question.

http://docs.python.org/library/subprocess.html
EDIT: Probably, I missed, what you want. Well, It all I can imagine about your question.
subprocess.call(["ls","-lAd"]) # executes external program. Like system()
# Capture output. Like popen, as I remember.
subprocess.check_output(["echo", "Hello World!"])
os.fork() # Binding to fork()
class MyThread(threading.thread):
def run():
print("Hello from thread")
MyThread().start()

yes there is
python provides 2 different ways of doing this threading and multiprocessing witch one you should use depend on the operation your performing.
the main difference is that Threading executes the function in the same interpreter while multiprocessing starts a new interpreter and runs the function in that interpreter. this means that multiprocessing is genraly used when your performing cpu bound operations like adding a lot of numbers and Thread is used for iobound operations like inputs or waiting for something to happen.
threading Example:
from threading import Thread
import time
def fun(a):
global myVar
myVar = "post start" # as you can see myVar is updated and can be read by the main Thread
time.sleep(1)
f = input(a)
print(f"inputed {f}")
myVar = "preThread"
t = Thread(target=fun,
args=("plz input your message ",))
t.start() # start the thread
print("this whil run after the thread started", myVar)
t.join() # wait for thread to finisch executing
print("this whil run after the thread ended", myVar)
outputs
this whil run after the thread started post start
plz input your message k
inputed k
this whil run after the thread ended post start
if you use the multiprocessing lib it starts a new python interpreter and all values are copied into it, and print and inputs wont work
from multiprocessing import Process
import time
def fun(a):
global myVar
myVar = "post start" # as you can see myVar is updated and can be read by the main Thread
time.sleep(1)
f = input(a)
print(f"inputed {f}")
myVar = "preThread"
t = Process(target=fun,
args=("plz input your message ",))
t.start() # start the thread
print("this whill run after the thread started", myVar)
t.join() # wait for thread to finisch executing
print("this whill run after the thread ended", myVar)
outputs:
this whill run after the thread started preThread
this whill run after the thread ended preThread
if you want to know more plz read
https://docs.python.org/3/library/threading.html for thread
https://docs.python.org/3/library/multiprocessing.html for multiprocessing\