Develop Reference

Run atexit() when python process is killed

I have a python process which runs in background, and I would like it to generate some output only when the script is terminated.
def handle_exit():
print('\nAll files saved in ' + directory)
generate_output()
atexit.register(handle_exit)
Calling raising a KeyboardInterupt exception and sys.exit() calls handle_exit() properly, but if I were to do kill {PID} from the terminal it terminates the script without calling handle_exit().
Is there a way to terminate the process that is running in the background, and still have it run handle_exit() before terminating?

Try signal.signal. It allows to catch any system signal:
import signal
def handle_exit():
print('\nAll files saved in ' + directory)
generate_output()
atexit.register(handle_exit)
signal.signal(signal.SIGTERM, handle_exit)
signal.signal(signal.SIGINT, handle_exit)
Now you can kill {pid} and handle_exit will be executed.

To enable signals when debugging PyCharm on Windows:
Within PyCharm hit Ctrl + Shift + A to bring up the "Find Actions..." menu
Search for "Registry" and hit enter
Find the key kill.windows.processes.softly and enable it (you can start typing "kill" and it will search for the key)
Restart PyCharm

To check your system and see which signal is being called:
import signal
import time
def handle_signal(sig_id, frame):
sig = {x.value: x for x in signal.valid_signals()}.get(sig_id)
print(f'{sig.name}, {sig_id=}, {frame=}')
exit(-1)
for sig in signal.valid_signals():
print(f'{sig.value}: signal.{sig.name},')
signal.signal(sig, handle_signal)
time.sleep(30)

python daemon thread exits but process still run in the background

I am using python 2.7 and Python thread doesn't kill its process after the main program exits. (checking this with the ps -ax command on ubuntu machine)
I have the below thread class,
import os
import threading
class captureLogs(threading.Thread):
'''
initialize the constructor
'''
def __init__(self, deviceIp, fileTag):
threading.Thread.__init__(self)
super(captureLogs, self).__init__()
self._stop = threading.Event()
self.deviceIp = deviceIp
self.fileTag = fileTag
def stop(self):
self._stop.set()
def stopped(self):
return self._stop.isSet()
'''
define the run method
'''
def run(self):
'''
Make the thread capture logs
'''
cmdTorun = "adb logcat > " + self.deviceIp +'_'+self.fileTag+'.log'
os.system(cmdTorun)
And I am creating a thread in another file sample.py,
import logCapture
import os
import time
c = logCapture.captureLogs('100.21.143.168','somefile')
c.setDaemon(True)
c.start()
print "Started the log capture. now sleeping. is this a dameon?", c.isDaemon()
time.sleep(5)
print "Sleep tiime is over"
c.stop()
print "Calling stop was successful:", c.stopped()
print "Thread is now completed and main program exiting"
I get the below output from the command line:
Started the log capture. now sleeping. is this a dameon? True
Sleep tiime is over
Calling stop was successful: True
Thread is now completed and main program exiting
And the sample.py exits.
But when I use below command on a terminal,
ps -ax | grep "adb"
I still see the process running. (I am killing them manually now using the kill -9 17681 17682)
Not sure what I am missing here.
My question is,
1) why is the process still alive when I already killed it in my program?
2) Will it create any problem if I don't bother about it?
3) is there any other better way to capture logs using a thread and monitor the logs?
EDIT: As suggested by #bug Killer, I added the below method in my thread class,
def getProcessID(self):
return os.getpid()
and used os.kill(c.getProcessID(), SIGTERM) in my sample.py . The program doesn't exit at all.

It is likely because you are using os.system in your thread. The spawned process from os.system will stay alive even after the thread is killed. Actually, it will stay alive forever unless you explicitly terminate it in your code or by hand (which it sounds like you are doing ultimately) or the spawned process exits on its own. You can do this instead:
import atexit
import subprocess
deviceIp = '100.21.143.168'
fileTag = 'somefile'
# this is spawned in the background, so no threading code is needed
cmdTorun = "adb logcat > " + deviceIp +'_'+fileTag+'.log'
proc = subprocess.Popen(cmdTorun, shell=True)
# or register proc.kill if you feel like living on the edge
atexit.register(proc.terminate)
# Here is where all the other awesome code goes
Since all you are doing is spawning a process, creating a thread to do it is overkill and only complicates your program logic. Just spawn the process in the background as shown above and then let atexit terminate it when your program exits. And/or call proc.terminate explicitly; it should be fine to call repeatedly (much like close on a file object) so having atexit call it again later shouldn't hurt anything.

delegate SIGINT signal to child process and then cleanup and terminate the parent

I have a main python(testmain.py) script that executes another python script(test.py) using subprocess.Popen command. When I press Ctrl-C , I want the child to exit with exit code 2 and then the parent to display that exit code and then terminate .
I have signal handlers in both parent and child scripts.
testmain.py
def signal_handler(signal, frame):
print "outer signal handler"
exit(2)
signal.signal(signal.SIGINT, signal_handler)
def execute()
proc=subprocess.Popen("python test.py",shell=True)
streamdata=proc.communicate()[0]
rc=proc.returncode
print "return code:",rc
execute()
test.py
def signal_handler(signal, frame):
print "exiting: inner function"
exit(2)
signal.signal(signal.SIGINT, signal_handler)
I checked Delegate signal handling to a child process in python that is kind of similar to my question but in that case, the parent is continuing it's execution, which I don't want.
I want to: 1.exit test.py with exit(2) 2.print that exit code in testmain.py 3.exit test.py with exit(2)
could someone please provide suggestions to do this?
Thanks.
UPDATE : Handling the signal only in the child (test.py) and checking the return code in parent(testmain.py) will do what I want .
if rc==2:
print "child was terminated"
exit(2)
but I was wondering if there is a clean way to do this using signal handling.

Your child process shouldn't care what the parent does i.e., if you want the child to exit with specific status on Ctrl+C then just do that:
import sys
try:
main()
except KeyboardInterrupt: # use default SIGINT handler
sys.exit(2)
Or you could define the signal handler explicitly:
import os
import signal
def signal_handler(signal, frame):
os.write(1, b"outer signal handler\n")
os._exit(2)
signal.signal(signal.SIGINT, signal_handler)
main()
There might be a difference in behavior if there are atexit handlers and/or multiple threads.
Unrelated: depending on what your main() function does, there could be a significant delay before a signal is handled in Python. Some blocking methods on Python 2 may ignore the signal completely: use Python 3 or apply a custom workaround for a specific case e.g., using a timeout parameter for some calls.
You could handle SIGINT in a similar way in the parent:
for cmd in commands:
process = Popen(cmd)
try:
process.wait()
except KeyboardInterrupt:
# child process may still be alive here
for _ in range(5): # wait a while
if process.poll() is not None:
break # the process is dead
time.sleep(.1)
else: # no break, kill the process explicitly
try:
process.kill()
except OSError:
pass
sys.exit("Child exited with %d" % process.wait())
Python 2 doesn't restore signals for child processes e.g., if you SIG_IGN the SIGINT signal in the parent, you could reset the necessary hanlders explicitly using preexec_fn parameter.

Trapping and handling taskkill in Windows Python

I am using Python 2.6.6 for Windows (on Windows XP SP3) with pywin32-218.
In my Python application, I have a second thread (apart from the main thread) which spawns a subprocess to run another Windows executable.
My problem is that when the main process (python.exe) is killed (e.g. using taskkill), I want to terminate the subprocess (calc.exe) and perform some cleaning up.
I tried various methods (atexit, signal and win32api.handleConsoleCtrl), but none seem to be able to trap the taskkill signal.
My code as follows (test.py):
import sys
import os
import signal
import win32api
import atexit
import time
import threading
import subprocess
class SecondThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.secondProcess = None
def run(self):
secondCommand = ['C:\WINDOWS\system32\calc.exe']
self.secondProcess = subprocess.Popen(secondCommand)
print 'calc.exe running'
self.secondProcess.wait()
print 'calc.exe stopped'
# do cleanup here
def stop(self):
if self.secondProcess and self.secondProcess.returncode == None:
self.secondProcess.kill()
secondThread = SecondThread()
def main():
secondThread.start()
def cleanup():
print 'cleaning up'
secondThread.stop()
print 'cleaned up'
atexit.register(cleanup)
def handleSignal(signalNum, frame):
print 'handleSignal'
cleanup()
sys.exit(0)
for signalNum in (signal.SIGINT, signal.SIGILL, signal.SIGABRT, signal.SIGFPE, signal.SIGSEGV, signal.SIGTERM):
signal.signal(signalNum, handleSignal)
def handleConsoleCtrl(signalNum):
print ('handleConsoleCtrl')
cleanup()
win32api.SetConsoleCtrlHandler(handleConsoleCtrl, True)
main()
The application is launched using
python.exe test.py
The console then prints "calc.exe running", and the Calculator application runs, and using Process Explorer, I can see calc.exe as a sub-process of python.exe
Then I kill the main process using
taskkill /pid XXXX /f
(where XXXX is the PID for python.exe)
What happens after this is that the command prompt returns without further output (i.e. none of "cleaning up", "handleSignal" or "handleConsoleCtrl" gets printed), the Calculator application continues running, and from Process Explorer, python.exe is no longer running but calc.exe has re-parented itself.

Taskkill (normally) sends WM_CLOSE. If your application is console only and has no window, while you can get CTRL_CLOSE_EVENT via a handler set by SetConsoleCtrlHandler (which happens if your controlling terminal window is closed) you can't receive a bare WM_CLOSE message.
If you have to stick with taskkill (rather than using a different program to send a Ctrl-C) one solution is to set the aforementioned handler and ensure your application has its own terminal window (e.g. by usingstart.exe "" <yourprog> to invoke it). See https://stackoverflow.com/a/23197789/4513656 for details an alternatives.

signal.SIGINT in windows cause process exit directly

I wrote those test python code as following:
import signal
import time
import os
def handler(signum, frame):
print "do whatever, like call thread.interrupt_main()"
return
signal.signal(signal.SIGINT, handler)
while 1:
try:
time.sleep(10)
except:
os.kill(int(os.getpid()), signal.SIGINT)
pass
when i excute this test code on windows, the process print "do whatever, like call thread.interrupt_main()", then exit;
on linux, it works correctly.
why on windows it not work?

http://docs.python.org/2/library/os.html#os.kill
Windows: The signal.CTRL_C_EVENT and signal.CTRL_BREAK_EVENT signals
are special signals which can only be sent to console processes which
share a common console window, e.g., some subprocesses. Any other
value for sig will cause the process to be unconditionally killed by
the TerminateProcess API, and the exit code will be set to sig.
Try following code:
import time
while 1:
try:
time.sleep(1)
except KeyboardInterrupt:
print "do whatever, like call thread.interrupt_main()"