import os
import sys
from multiprocessing import Process, Queue
import threading
class Test:
def __init__(self):
print '__init__ is called'
def say_hello_again_and_again(self):
print 'Hello :D'
threading.Timer(1, self.say_hello_again_and_again).start()
test = Test()
#test.say_hello_again_and_again()
process = Process(target=test.say_hello_again_and_again)
process.start()
this is test code.
the result:
pi#raspberrypi:~/Plant2 $ python test2.py
__init__ is called
Hello :D
If I use test.say_hello_again_and_again() , "Hello :D" is printed repeatedly.
But, process is not working as I expected. Why is "Hello :D" not being printed in my process?
What's happening in my process?
There are two problems with your code:
First: You start a process with start(). This is doing a fork, that means now you have two processes, the parent and the child running side by side. Now, the parent process immediately exits, because after start() it's the end of the program. To wait until the child has finished (which in your case is never), you have to add process.join().
I did test your suggestion, but it not works
Indeed. There is a second issue: You start a new thread with threading.Timer(1, ...).start() but then immediately end the process. Now, you don't wait until your thread started because the underlying process immediately dies. You'd need to also wait until the thread has stopped with join().
Now that's how your program would look like:
from multiprocessing import Process
import threading
class Test:
def __init__(self):
print '__init__ is called'
def say_hello_again_and_again(self):
print 'Hello :D'
timer = threading.Timer(1, self.say_hello_again_and_again)
timer.start()
timer.join()
test = Test()
process = Process(target=test.say_hello_again_and_again)
process.start()
process.join()
But this is suboptimal at best because you mix multiprocessing (which is using fork to start independent processes) and threading (which starts a thread within the process). While this is not really a problem it makes debugging a lot harder (one problem e.g. with the code above is that you can't stop it with ctrl-c because of some reason your spawned process is inherited by the OS and kept running). Why don't you just do this?
from multiprocessing import Process, Queue
import time
class Test:
def __init__(self):
print '__init__ is called'
def say_hello_again_and_again(self):
while True:
print 'Hello :D'
time.sleep(1)
test = Test()
process = Process(target=test.say_hello_again_and_again)
process.start()
process.join()
Related
I just start a new thread:
self.thread = ThreadedFunc()
self.thread.start()
after something happens I want to exit my program so I'm calling os._exit():
os._exit(1)
The program still works. Everything is functional and it just looks like the os._exit() didn't execute.
Is there a different way to exit a whole program from different thread? How to fix this?
EDIT: Added more complete code sample.
self.thread = DownloadThread()
self.thread.data_downloaded.connect(self.on_data_ready)
self.thread.data_progress.connect(self.on_progress_ready)
self.progress_initialized = False
self.thread.start()
class DownloadThread(QtCore.QThread):
# downloading stuff etc.
sleep(1)
subprocess.call(os.getcwd() + "\\another_process.exe")
sleep(2)
os._exit(1)
EDIT 2: SOLVED! There is a quit(), terminate() or exit() function which just stops the thread. It was that easy. Just look at the docs.
Calling os._exit(1) works for me.
You should use the standard lib threading.
I guess you are using multiprocessing, which is a process-based “threading” interface, which uses similar API to threading, but creates child process instead of child thread. so os._exit(1) only exits child process, not affecting the main process
Also you should ensure you have called join() function in the main thread. Otherwise, it is possible that the operating system schedules to run the main thread to the end before starting to do anything in child thread.
sys.exit() does not work because it is the same as raising a SystemExit exception. Raising an exception in thread only exits that thread, rather than the entire process.
Sample code. Tested under ubuntu by python3 thread.py; echo $?.
Return code is 1 as expected
import os
import sys
import time
import threading
# Python Threading Example for Beginners
# First Method
def greet_them(people):
for person in people:
print("Hello Dear " + person + ". How are you?")
os._exit(1)
time.sleep(0.5)
# Second Method
def assign_id(people):
i = 1
for person in people:
print("Hey! {}, your id is {}.".format(person, i))
i += 1
time.sleep(0.5)
people = ['Richard', 'Dinesh', 'Elrich', 'Gilfoyle', 'Gevin']
t = time.time()
#Created the Threads
t1 = threading.Thread(target=greet_them, args=(people,))
t2 = threading.Thread(target=assign_id, args=(people,))
#Started the threads
t1.start()
t2.start()
#Joined the threads
t1.join() # Cannot remove this join() for this example
t2.join()
# Possible to reach here if join() removed
print("I took " + str(time.time() - t))
Credit: Sample code is copied and modified from https://www.simplifiedpython.net/python-threading-example/
I am testing Python threading with the following script:
import threading
class FirstThread (threading.Thread):
def run (self):
while True:
print 'first'
class SecondThread (threading.Thread):
def run (self):
while True:
print 'second'
FirstThread().start()
SecondThread().start()
This is running in Python 2.7 on Kubuntu 11.10. Ctrl+C will not kill it. I also tried adding a handler for system signals, but that did not help:
import signal
import sys
def signal_handler(signal, frame):
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
To kill the process I am killing it by PID after sending the program to the background with Ctrl+Z, which isn't being ignored. Why is Ctrl+C being ignored so persistently? How can I resolve this?
Ctrl+C terminates the main thread, but because your threads aren't in daemon mode, they keep running, and that keeps the process alive. We can make them daemons:
f = FirstThread()
f.daemon = True
f.start()
s = SecondThread()
s.daemon = True
s.start()
But then there's another problem - once the main thread has started your threads, there's nothing else for it to do. So it exits, and the threads are destroyed instantly. So let's keep the main thread alive:
import time
while True:
time.sleep(1)
Now it will keep print 'first' and 'second' until you hit Ctrl+C.
Edit: as commenters have pointed out, the daemon threads may not get a chance to clean up things like temporary files. If you need that, then catch the KeyboardInterrupt on the main thread and have it co-ordinate cleanup and shutdown. But in many cases, letting daemon threads die suddenly is probably good enough.
KeyboardInterrupt and signals are only seen by the process (ie the main thread)... Have a look at Ctrl-c i.e. KeyboardInterrupt to kill threads in python
I think it's best to call join() on your threads when you expect them to die. I've taken the liberty to make the change your loops to end (you can add whatever cleanup needs are required to there as well). The variable die is checked on each pass and when it's True, the program exits.
import threading
import time
class MyThread (threading.Thread):
die = False
def __init__(self, name):
threading.Thread.__init__(self)
self.name = name
def run (self):
while not self.die:
time.sleep(1)
print (self.name)
def join(self):
self.die = True
super().join()
if __name__ == '__main__':
f = MyThread('first')
f.start()
s = MyThread('second')
s.start()
try:
while True:
time.sleep(2)
except KeyboardInterrupt:
f.join()
s.join()
An improved version of #Thomas K's answer:
Defining an assistant function is_any_thread_alive() according to this gist, which can terminates the main() automatically.
Example codes:
import threading
def job1():
...
def job2():
...
def is_any_thread_alive(threads):
return True in [t.is_alive() for t in threads]
if __name__ == "__main__":
...
t1 = threading.Thread(target=job1,daemon=True)
t2 = threading.Thread(target=job2,daemon=True)
t1.start()
t2.start()
while is_any_thread_alive([t1,t2]):
time.sleep(0)
One simple 'gotcha' to beware of, are you sure CAPS LOCK isn't on?
I was running a Python script in the Thonny IDE on a Pi4. With CAPS LOCK on, Ctrl+Shift+C is passed to the keyboard buffer, not Ctrl+C.
I'm going through an example from a book about threading, and this is the example they give:
## To use threads you need import Thread using the following code:
from threading import Thread
##Also we use the sleep function to make the thread "sleep"
from time import sleep
## To create a thread in Python you'll want to make your class work as a thread.
## For this, you should subclass your class from the Thread class
class CookBook(Thread):
def __init__(self):
Thread.__init__(self)
self.message = "Hello Parallel Python CookBook!!\n"
##this method thod prints only the message
def print_message(self):
print (self.message)
##The run method prints ten times the message
def run(self):
print ("Thread Starting\n")
x=0
while (x < 10):
self.print_message()
sleep(2)
x += 1
print ("Thread Ended\n")
#start the main process
print ("Process Started")
# create an instance of the HelloWorld class
hello_Python = CookBook()
# print the message...starting the thread
hello_Python.start()
#end the main process
print ("Process Ended")
#create an instance of the HelloWorld class
hello_Python = CookBook()
#print the message...starting the thread
hello_Python.start()
#end the main process
print ("Process Ended")
It's the first example in the first chapter, and at the end of the chapter the author says to make sure that you don't have any threads running in the background, that it's bad programming.
Question:
Given my example, how do you properly verify no threads are running in the background?
There are two ways depending on what you need:
Use join(), as comment suggested.
Set your thread daemon thread, which means calling Thread.__init__(self, daemon=True). When your main thread exits, other threads you create automatically exit too, thus you don't have to worry about them running in background.
I am using python 2.7 and Python thread doesn't kill its process after the main program exits. (checking this with the ps -ax command on ubuntu machine)
I have the below thread class,
import os
import threading
class captureLogs(threading.Thread):
'''
initialize the constructor
'''
def __init__(self, deviceIp, fileTag):
threading.Thread.__init__(self)
super(captureLogs, self).__init__()
self._stop = threading.Event()
self.deviceIp = deviceIp
self.fileTag = fileTag
def stop(self):
self._stop.set()
def stopped(self):
return self._stop.isSet()
'''
define the run method
'''
def run(self):
'''
Make the thread capture logs
'''
cmdTorun = "adb logcat > " + self.deviceIp +'_'+self.fileTag+'.log'
os.system(cmdTorun)
And I am creating a thread in another file sample.py,
import logCapture
import os
import time
c = logCapture.captureLogs('100.21.143.168','somefile')
c.setDaemon(True)
c.start()
print "Started the log capture. now sleeping. is this a dameon?", c.isDaemon()
time.sleep(5)
print "Sleep tiime is over"
c.stop()
print "Calling stop was successful:", c.stopped()
print "Thread is now completed and main program exiting"
I get the below output from the command line:
Started the log capture. now sleeping. is this a dameon? True
Sleep tiime is over
Calling stop was successful: True
Thread is now completed and main program exiting
And the sample.py exits.
But when I use below command on a terminal,
ps -ax | grep "adb"
I still see the process running. (I am killing them manually now using the kill -9 17681 17682)
Not sure what I am missing here.
My question is,
1) why is the process still alive when I already killed it in my program?
2) Will it create any problem if I don't bother about it?
3) is there any other better way to capture logs using a thread and monitor the logs?
EDIT: As suggested by #bug Killer, I added the below method in my thread class,
def getProcessID(self):
return os.getpid()
and used os.kill(c.getProcessID(), SIGTERM) in my sample.py . The program doesn't exit at all.
It is likely because you are using os.system in your thread. The spawned process from os.system will stay alive even after the thread is killed. Actually, it will stay alive forever unless you explicitly terminate it in your code or by hand (which it sounds like you are doing ultimately) or the spawned process exits on its own. You can do this instead:
import atexit
import subprocess
deviceIp = '100.21.143.168'
fileTag = 'somefile'
# this is spawned in the background, so no threading code is needed
cmdTorun = "adb logcat > " + deviceIp +'_'+fileTag+'.log'
proc = subprocess.Popen(cmdTorun, shell=True)
# or register proc.kill if you feel like living on the edge
atexit.register(proc.terminate)
# Here is where all the other awesome code goes
Since all you are doing is spawning a process, creating a thread to do it is overkill and only complicates your program logic. Just spawn the process in the background as shown above and then let atexit terminate it when your program exits. And/or call proc.terminate explicitly; it should be fine to call repeatedly (much like close on a file object) so having atexit call it again later shouldn't hurt anything.
I'm making a library that uses gevent to do some work asynchronously. I'd like to guarantee that the work is completed, even if the main module finishes execution.
class separate_library(object):
def __init__(self):
import gevent.monkey; gevent.monkey.patch_all()
def do_work(self):
from gevent import spawn
spawn(self._do)
def _do(self):
from gevent import sleep
sleep(1)
print 'Done!'
if __name__ == '__main__':
lib = separate_library()
lib.do_work()
If you run this, you'll notice the program ends immediately, and Done! doesn't get printed.
Now, the main module doesn't know, or care, how separate_library actually accomplishes the work (or even that gevent is being used), so it's unreasonable to require joining there.
Is there any way separate_library can detect certain types of program exits, and stall until the work is done? Keyboard interrupts, SIGINTs, and sys.exit() should end the program immediately, as that is probably the expected behaviour.
Thanks!
Try using a new thread that is not a daemon thread that spawns your gevent threads. Your program will not exit due to this non daemon thread.
import gevent
import threading
class separate_library(object):
def __init__(self):
import gevent.monkey; gevent.monkey.patch_all()
def do_work(self):
t = threading.Thread(target=self.spawn_gthreads)
t.setDaemon(False)
t.start()
def spawn_gthreads(self):
from gevent import spawn
gthreads = [spawn(self._do,x) for x in range(10)]
gevent.joinall(gthreads)
def _do(self,sec):
from gevent import sleep
sleep(sec)
print 'Done!'
if __name__ == '__main__':
lib = separate_library()
lib.do_work()