What difference makes it when I set a python thread as a daemon, using thread.setDaemon(True)?
A daemon thread will not prevent the application from exiting. The program ends when all non-daemon threads (main thread included) are complete.
So generally, if you're doing something in the background, you might want to set the thread as daemon so you don't have to explicitly have that thread's function return before the app can exit.
For example, if you are writing a GUI application and the user closes the main window, the program should quit. But if you have non-daemon threads hanging around, it won't.
From the docs: http://docs.python.org/library/threading.html#threading.Thread.daemon
Its initial value is inherited from
the creating thread; the main thread
is not a daemon thread and therefore
all threads created in the main thread
default to daemon = False.
The entire Python program exits when
no alive non-daemon threads are left.
Related
I run a separate process for some logging tasks in parallel to my main process. They share some resources and I run into issues terminating the logging process before the main process finishes.
Are there any drawbacks from finishing the main Python program and keeping the subprocess? Can I be sure that it will be terminated on exiting the main program? Or would it be better to call Process.terminate() as my last call in the main script?
As long as the processes you're launching are daemons, the main process will terminate them automatically before it exits:
daemon
The process’s daemon flag, a Boolean value. This must be set
before start() is called.
The initial value is inherited from the creating process.
When a process exits, it attempts to terminate all of its daemonic
child processes.
Note that a daemonic process is not allowed to create child processes.
Otherwise a daemonic process would leave its children orphaned if it
gets terminated when its parent process exits. Additionally, these are
not Unix daemons or services, they are normal processes that will be
terminated (and not joined) if non-daemonic processes have exited.
This flag is automatically set for processes created by a multiprocessing.Pool, but defaults to false for Process objects. The parent process will try to call join on all non-daemon children, so if you have any of those running, they will prevent the parent from exiting until they exit themselves.
Will the resources of non-daemon thread get released back to OS once the threads completes? ie If the main thread is not calling join() on these non-daemon threads, will the python GC call join on them and release the resources once held by the thread?
If you spawn a thread that runs a function, and then that function completes before the end of the program, then yes, the thread will get garbage collected once it is (a) no longer running, and (b) no longer referenced by anything else.
"
A thread can be flagged as a “daemon thread”. The significance of this
flag is that the entire Python program exits when only daemon threads
are left. The initial value is inherited from the creating thread. The
flag can be set through the daemon property.
Note Daemon threads are abruptly stopped at shutdown. Their resources
(such as open files, database transactions, etc.) may not be released
properly. If you want your threads to stop gracefully, make them
non-daemonic and use a suitable signalling mechanism such as an Event.
" -- Python Thread Docs
Daemons are cleaned up by Python, Non-daemonic threads are not - you have to signal them to stop. This is useful in executing some complicated parallel code though :)
This does mean though you can have random dummy / useless threads sitting around for you to manually cleanup if you use non-daemonic threads.
tl;dr Non-daemonic threads never 'finish' you have to signal them to finish via your own mechanism or one of the SIGS e.g. SIGTERM.
What is the difference between thread.start_new_thread and threading.Thread.start in python?
I have noticed that when start_new_thread is called, the new thread terminates as soon as the calling thread terminates. threading.Thread.start is the opposite: the calling thread waits for other threads to terminate.
The thread module is the low-level threading API of Python. Its direct usage isn't recommended, unless you really need to. The threading module is a high-level API, built on top of thread. The Thread.start method is actually implemented using thread.start_new_thread.
The daemon attribute of Thread must be set before calling start, specifying whether the thread should be a daemon. The entire Python program exits when no alive non-daemon threads are left. By default, daemon is False, so the thread is not a daemon, and hence the process will wait for all its non-daemon thread to exit, which is the behavior you're observing.
P.S. start_new_thread really is very low-level. It's just a thin wrapper around the Python core thread launcher, which itself calls the OS thread spawning function.
See the threading.Thread.daemon flag - basically whenever no non-daemon threads are running, the interpreter terminates.
I'm a little confused about what setting a thread to be a daemon means.
The documentation says this:
A thread can be flagged as a “daemon
thread”. The significance of this flag
is that the entire Python program
exits when only daemon threads are
left. The initial value is inherited
from the creating thread. The flag can
be set through the daemon property.
I'm not sure what makes this different from a normal thread.
Is this saying that this program won't ever finish?
def threadfunc():
while True:
time.sleep(1)
threading.Thread(target=threadfunc).start()
Even though the main thread finishes it's execution. While will finish immediately?
def threadfunc():
while True:
time.sleep(1)
th = threading.Thread(target=threadfunc)
th.daemon = True
th.start()
I ask because I have a situation where in my main thread I'm calling sys.exit(), and the process just hangs and my other threads are running as I can see the log.
Does this have anything to do with sys.exit() being called with threads alive?
Is this saying that this program won't ever finish?
Yes, that program won't finish, just try it out.
I ask because I have a situation where
in my main thread I'm calling
sys.exit(), and the process just hangs
and my other threads are running as I
can see the log. Does this have
anything to do with sys.exit() being
called with threads alive?
Yes, even exit won't stop other threads, it simply raises SystemExit in the main thread. So while the main thread will stop (just like it does on any other unhandled Exception), all other non-daemonic threads will continue to work.
Setting thread.daemon = True will allow the main program to exit. Apps normally wait till all child threads are finished before completing.
th.daemon = True #set this thread as a Daemon Thread
You can think in a Daemon thread as a service this means that it will be running in the background of your computer doing differents task, like indexing files, parsing xml, retrieving news etc, anything that is a long running process.
Your Main thread will finish and your daemon will still be running in the background, that is the reason why your program aka Main thread finish, if you want just put an infinite loop and you will see your thread still running.
An example for a daemon thread is the garbage collection.
In a python script, I started a bunch of threads, each of which pulls some resource at an interval using time.sleep(interval). I have another thread running, which uses the cmd module to monitor user inputs. When the user enters 'q', I call
sys.exit(0)
However, when the script is running and I enter 'q', the thread user input monitoring thread is quit, but the sleeping threads are still alive. (meaning the program does not exit)
I'm wondering if I'm doing it the right way?
sys.exit will only stop the thread it executes from. If you have other non-daemon thread in your program they will continue to execute. Section 17.2.1 of the Python library docs contains:
A thread can be flagged as a “daemon
thread”. The significance of this flag
is that the entire Python program
exits when only daemon threads are
left. The initial value is inherited
from the creating thread. The flag can
be set through the daemon property.
See also Why does sys.exit() not exit when called inside a thread in Python?.