Develop Reference

Python : call a function and wait for its thread(s) to finish

I am using Python to script a gdb session. My goal is to run a function whenever a breakpoint is hit, using gdb.events.stop.connect.
Here is my code in essence, which I call from gdb by running source main.py :
RESULT = None
def breakpoint_handler(event):
global RESULT
RESULT = do_something(event) # boolean
sleep(1)
def main():
global RESULT
gdb.events.stop.connect(breakpoint_handler)
command = "some gdb command"
gdb.execute(command)
sleep(1)
if RESULT:
# do something
pass
else:
# something else
pass
main()
This code works, but only thanks to the sleep calls, that tries to make sure the gdb.execute(command)call actually finish executing the gdb command before continuing. There is only little documentation but my guess is that this function is threaded, which explains the effectiveness of the waitas a fix.
Is there a way to make this cleaner by waiting for the thread created by gdb.execute to finish processing, without having access to the body of this method ?

You can set an event inside your callback and wait for it in your main thread. Here is an example, with the callback being called after 5 seconds by timer:
import threading
def some_callback():
event.set()
event = threading.Event()
threading.Timer(5, some_callback).start()
event.wait()

How to continue with the main program when threading

I'm trying to run a thread in python and am fairly new at it. While going through the basics, when I start a thread, the program doesn't continue with the main program and is stuck in the thread. i.e. it prints only "hello world" and never prints "hi there".
from threading import Thread
import time
def hello_world():
while True:
print("hello world")
time.sleep(5)
t = Thread(target = hello_world())
t.start()
print("hi there")
I'm using spyder IDE.
I searched online for some basic programs in threading but for those, the code works.
How should I proceed?

Your problem is in the line t = Thread(target = hello_world()).
You are trying to create a Thread with the target argument. According to the order of evaluation, Python first needs to know what to assign to target, so it evaluates the RHS. In your case the RHS is hello_world(). So the function is already being called in that exact moment!
So the function executes and enters the infinite loop and the Thread is never even created and your program is stuck.
What you would want to do is pass to target a mere reference to the function, so change said line to:
t = Thread(target = hello_world)
And now the RHS is evaluated as a reference to the given function, and behind the scenes the Thread will be created, making the call to that function, and your main Thread will keep running as expected.

Python main thread interruption

Can anyone explain how the interrupt_main() method works in Python?
I've got this piece of Python code :
import time, thread
def f():
time.sleep(5)
thread.interrupt_main()
def g():
thread.start_new_thread(f, ())
time.sleep(10)
print time.time()
try:
g()
except KeyboardInterrupt:
print time.time()
And when I try to run it, it gives me the following output :
1380542215.5
# ... 10 seconds break...
1380542225.51
However, if I interrupt the program manually (CTRL-C), the thread is interrupted correctly :
1380542357.58
^C1380542361.49
Why does the thread interruption only occur after 10 seconds (and not 5) in the first example?
I found an ancient thread n Python mailing list, but it explains nearly nothing.

raise KeyboardInterrupt does not interrupt a time.sleep(). The former is handled entirely inside the python interpreter, the latter invokes an operating system function.
So, in your case, the keyboard interrupt was handled, but only when time.sleep() completed its system call.
Try this instead:
def g():
thread.start_new_thread(f, ())
for _ in range(10):
time.sleep(1)

Python: run one function until another function finishes

I have two functions, draw_ascii_spinner and findCluster(companyid).
I would like to:
Run findCluster(companyid) in the backround and while its processing....
Run draw_ascii_spinner until findCluster(companyid) finishes
How do I begin to try to solve for this (Python 2.7)?

Use threads:
import threading, time
def wrapper(func, args, res):
res.append(func(*args))
res = []
t = threading.Thread(target=wrapper, args=(findcluster, (companyid,), res))
t.start()
while t.is_alive():
# print next iteration of ASCII spinner
t.join(0.2)
print res[0]

You can use multiprocessing. Or, if findCluster(companyid) has sensible stopping points, you can turn it into a generator along with draw_ascii_spinner, to do something like this:
for tick in findCluster(companyid):
ascii_spinner.next()

Generally, you will use Threads. Here is a simplistic approach which assumes, that there are only two threads: 1) the main thread executing a task, 2) the spinner thread:
#!/usr/bin/env python
import time
import thread
def spinner():
while True:
print '.'
time.sleep(1)
def task():
time.sleep(5)
if __name__ == '__main__':
thread.start_new_thread(spinner, ())
# as soon as task finishes (and so the program)
# spinner will be gone as well
task()

This can be done with threads. FindCluster runs in a separate thread and when done, it can simply signal another thread that is polling for a reply.

You'll want to do some research on threading, the general form is going to be this
Create a new thread for findCluster and create some way for the program to know the method is running - simplest in Python is just a global boolean
Run draw_ascii_spinner in a while loop conditioned on whether it is still running, you'll probably want to have this thread sleep for a short period of time between iterations
Here's a short tutorial in Python - http://linuxgazette.net/107/pai.html

Run findCluster() in a thread (the Threading module makes this very easy), and then draw_ascii_spinner until some condition is met.
Instead of using sleep() to set the pace of the spinner, you can wait on the thread's wait() with a timeout.

It is possible to have a working example? I am new in Python. I have 6 tasks to run in one python program. These 6 tasks should work in coordinations, meaning that one should start when another finishes. I saw the answers , but I couldn't adopted the codes you shared to my program.
I used "time.sleep" but I know that it is not good because I cannot know how much time it takes each time.
# Sending commands
for i in range(0,len(cmdList)): # port Sending commands
cmd = cmdList[i]
cmdFull = convert(cmd)
port.write(cmd.encode('ascii'))
# s = port.read(10)
print(cmd)
# Terminate the command + close serial port
port.write(cmdFull.encode('ascii'))
print('Termination')
port.close()
# time.sleep(1*60)

How to execute a function asynchronously every 60 seconds in Python?

I want to execute a function every 60 seconds on Python but I don't want to be blocked meanwhile.
How can I do it asynchronously?
import threading
import time
def f():
print("hello world")
threading.Timer(3, f).start()
if __name__ == '__main__':
f()
time.sleep(20)
With this code, the function f is executed every 3 seconds within the 20 seconds time.time.
At the end it gives an error and I think that it is because the threading.timer has not been canceled.
How can I cancel it?

You could try the threading.Timer class: http://docs.python.org/library/threading.html#timer-objects.
import threading
def f(f_stop):
# do something here ...
if not f_stop.is_set():
# call f() again in 60 seconds
threading.Timer(60, f, [f_stop]).start()
f_stop = threading.Event()
# start calling f now and every 60 sec thereafter
f(f_stop)
# stop the thread when needed
#f_stop.set()

The simplest way is to create a background thread that runs something every 60 seconds. A trivial implementation is:
import time
from threading import Thread
class BackgroundTimer(Thread):
def run(self):
while 1:
time.sleep(60)
# do something
# ... SNIP ...
# Inside your main thread
# ... SNIP ...
timer = BackgroundTimer()
timer.start()
Obviously, if the "do something" takes a long time, then you'll need to accommodate for it in your sleep statement. But, 60 seconds serves as a good approximation.

I googled around and found the Python circuits Framework, which makes it possible to wait
for a particular event.
The .callEvent(self, event, *channels) method of circuits contains a fire and suspend-until-response functionality, the documentation says:
Fire the given event to the specified channels and suspend execution
until it has been dispatched. This method may only be invoked as
argument to a yield on the top execution level of a handler (e.g.
"yield self.callEvent(event)"). It effectively creates and returns
a generator that will be invoked by the main loop until the event has
been dispatched (see :func:circuits.core.handlers.handler).
I hope you find it as useful as I do :)
./regards

It depends on what you actually want to do in the mean time. Threads are the most general and least preferred way of doing it; you should be aware of the issues with threading when you use it: not all (non-Python) code allows access from multiple threads simultaneously, communication between threads should be done using thread-safe datastructures like Queue.Queue, you won't be able to interrupt the thread from outside it, and terminating the program while the thread is still running can lead to a hung interpreter or spurious tracebacks.
Often there's an easier way. If you're doing this in a GUI program, use the GUI library's timer or event functionality. All GUIs have this. Likewise, if you're using another event system, like Twisted or another server-process model, you should be able to hook into the main event loop to cause it to call your function regularly. The non-threading approaches do cause your program to be blocked while the function is pending, but not between functioncalls.

Why dont you create a dedicated thread, in which you put a simple sleeping loop:
#!/usr/bin/env python
import time
while True:
# Your code here
time.sleep(60)

I think the right way to run a thread repeatedly is the next:
import threading
import time
def f():
print("hello world") # your code here
myThread.run()
if __name__ == '__main__':
myThread = threading.Timer(3, f) # timer is set to 3 seconds
myThread.start()
time.sleep(10) # it can be loop or other time consuming code here
if myThread.is_alive():
myThread.cancel()
With this code, the function f is executed every 3 seconds within the 10 seconds time.sleep(10). At the end running of thread is canceled.

If you want to invoke the method "on the clock" (e.g. every hour on the hour), you can integrate the following idea with whichever threading mechanism you choose:
import time
def wait(n):
'''Wait until the next increment of n seconds'''
x = time.time()
time.sleep(n-(x%n))
print(time.asctime())

[snip. removed non async version]
To use asyncing you would use trio. I recommend trio to everyone who asks about async python. It is much easier to work with especially sockets. With sockets I have a nursery with 1 read and 1 write function and the write function writes data from an deque where it is placed by the read function; and waiting to be sent. The following app works by using trio.run(function,parameters) and then opening an nursery where the program functions in loops with an await trio.sleep(60) between each loop to give the rest of the app a chance to run. This will run the program in a single processes but your machine can handle 1500 TCP connections insead of just 255 with the non async method.
I have not yet mastered the cancellation statements but I put at move_on_after(70) which is means the code will wait 10 seconds longer than to execute a 60 second sleep before moving on to the next loop.
import trio
async def execTimer():
'''This function gets executed in a nursery simultaneously with the rest of the program'''
while True:
trio.move_on_after(70):
await trio.sleep(60)
print('60 Second Loop')
async def OneTime_OneMinute():
'''This functions gets run by trio.run to start the entire program'''
with trio.open_nursery() as nursery:
nursery.start_soon(execTimer)
nursery.start_soon(print,'do the rest of the program simultaneously')
def start():
'''You many have only one trio.run in the entire application'''
trio.run(OneTime_OneMinute)
if __name__ == '__main__':
start()
This will run any number of functions simultaneously in the nursery. You can use any of the cancellable statements for checkpoints where the rest of the program gets to continue running. All trio statements are checkpoints so use them a lot. I did not test this app; so if there are any questions just ask.
As you can see trio is the champion of easy-to-use functionality. It is based on using functions instead of objects but you can use objects if you wish.
Read more at:
[1]: https://trio.readthedocs.io/en/stable/reference-core.html