Develop Reference

__exit__ method not being called when SIGTERM [duplicate]

I have a python process which runs in background, and I would like it to generate some output only when the script is terminated.
def handle_exit():
print('\nAll files saved in ' + directory)
generate_output()
atexit.register(handle_exit)
Calling raising a KeyboardInterupt exception and sys.exit() calls handle_exit() properly, but if I were to do kill {PID} from the terminal it terminates the script without calling handle_exit().
Is there a way to terminate the process that is running in the background, and still have it run handle_exit() before terminating?

Try signal.signal. It allows to catch any system signal:
import signal
def handle_exit():
print('\nAll files saved in ' + directory)
generate_output()
atexit.register(handle_exit)
signal.signal(signal.SIGTERM, handle_exit)
signal.signal(signal.SIGINT, handle_exit)
Now you can kill {pid} and handle_exit will be executed.

To enable signals when debugging PyCharm on Windows:
Within PyCharm hit Ctrl + Shift + A to bring up the "Find Actions..." menu
Search for "Registry" and hit enter
Find the key kill.windows.processes.softly and enable it (you can start typing "kill" and it will search for the key)
Restart PyCharm

To check your system and see which signal is being called:
import signal
import time
def handle_signal(sig_id, frame):
sig = {x.value: x for x in signal.valid_signals()}.get(sig_id)
print(f'{sig.name}, {sig_id=}, {frame=}')
exit(-1)
for sig in signal.valid_signals():
print(f'{sig.value}: signal.{sig.name},')
signal.signal(sig, handle_signal)
time.sleep(30)

How do I make a command to run in background using pexpect.spawn?

I have a code snippet in which I run a command in background:
from sys import stdout,exit
import pexpect
try:
child=pexpect.spawn("./gmapPlayerCore.r &")
except:
print "Replaytool Execution Failed!!"
exit(1)
child.timeout=Timeout
child.logfile=stdout
status=child.expect(["[nN]o [sS]uch [fF]ile","",pexpect.TIMEOUT,pexpect.EOF])
if status==0:
print "gmapPlayerCore file is missing"
exit(1)
elif status==1:
print "Starting ReplayTool!!!"
else:
print "Timed out!!"
Here, after the script exits the process in the spawn also gets killed even though it is run in background
How to achieve this?

You are asking the spawned child to be synchronous so that you can execute child.expect(…) and asynchronous &. These don't agree with each other.
You probably want:
child=pexpect.spawn("./gmapPlayerCore.r") # no &
status=child.expect(["[nN]o [sS]uch [fF]ile","",pexpect.TIMEOUT,pexpect.EOF])
child.interact()
where interact is defined as:
This gives control of the child process to the interactive user (the human at the keyboard). …

python daemon thread exits but process still run in the background

I am using python 2.7 and Python thread doesn't kill its process after the main program exits. (checking this with the ps -ax command on ubuntu machine)
I have the below thread class,
import os
import threading
class captureLogs(threading.Thread):
'''
initialize the constructor
'''
def __init__(self, deviceIp, fileTag):
threading.Thread.__init__(self)
super(captureLogs, self).__init__()
self._stop = threading.Event()
self.deviceIp = deviceIp
self.fileTag = fileTag
def stop(self):
self._stop.set()
def stopped(self):
return self._stop.isSet()
'''
define the run method
'''
def run(self):
'''
Make the thread capture logs
'''
cmdTorun = "adb logcat > " + self.deviceIp +'_'+self.fileTag+'.log'
os.system(cmdTorun)
And I am creating a thread in another file sample.py,
import logCapture
import os
import time
c = logCapture.captureLogs('100.21.143.168','somefile')
c.setDaemon(True)
c.start()
print "Started the log capture. now sleeping. is this a dameon?", c.isDaemon()
time.sleep(5)
print "Sleep tiime is over"
c.stop()
print "Calling stop was successful:", c.stopped()
print "Thread is now completed and main program exiting"
I get the below output from the command line:
Started the log capture. now sleeping. is this a dameon? True
Sleep tiime is over
Calling stop was successful: True
Thread is now completed and main program exiting
And the sample.py exits.
But when I use below command on a terminal,
ps -ax | grep "adb"
I still see the process running. (I am killing them manually now using the kill -9 17681 17682)
Not sure what I am missing here.
My question is,
1) why is the process still alive when I already killed it in my program?
2) Will it create any problem if I don't bother about it?
3) is there any other better way to capture logs using a thread and monitor the logs?
EDIT: As suggested by #bug Killer, I added the below method in my thread class,
def getProcessID(self):
return os.getpid()
and used os.kill(c.getProcessID(), SIGTERM) in my sample.py . The program doesn't exit at all.

It is likely because you are using os.system in your thread. The spawned process from os.system will stay alive even after the thread is killed. Actually, it will stay alive forever unless you explicitly terminate it in your code or by hand (which it sounds like you are doing ultimately) or the spawned process exits on its own. You can do this instead:
import atexit
import subprocess
deviceIp = '100.21.143.168'
fileTag = 'somefile'
# this is spawned in the background, so no threading code is needed
cmdTorun = "adb logcat > " + deviceIp +'_'+fileTag+'.log'
proc = subprocess.Popen(cmdTorun, shell=True)
# or register proc.kill if you feel like living on the edge
atexit.register(proc.terminate)
# Here is where all the other awesome code goes
Since all you are doing is spawning a process, creating a thread to do it is overkill and only complicates your program logic. Just spawn the process in the background as shown above and then let atexit terminate it when your program exits. And/or call proc.terminate explicitly; it should be fine to call repeatedly (much like close on a file object) so having atexit call it again later shouldn't hurt anything.

Trapping and handling taskkill in Windows Python

I am using Python 2.6.6 for Windows (on Windows XP SP3) with pywin32-218.
In my Python application, I have a second thread (apart from the main thread) which spawns a subprocess to run another Windows executable.
My problem is that when the main process (python.exe) is killed (e.g. using taskkill), I want to terminate the subprocess (calc.exe) and perform some cleaning up.
I tried various methods (atexit, signal and win32api.handleConsoleCtrl), but none seem to be able to trap the taskkill signal.
My code as follows (test.py):
import sys
import os
import signal
import win32api
import atexit
import time
import threading
import subprocess
class SecondThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.secondProcess = None
def run(self):
secondCommand = ['C:\WINDOWS\system32\calc.exe']
self.secondProcess = subprocess.Popen(secondCommand)
print 'calc.exe running'
self.secondProcess.wait()
print 'calc.exe stopped'
# do cleanup here
def stop(self):
if self.secondProcess and self.secondProcess.returncode == None:
self.secondProcess.kill()
secondThread = SecondThread()
def main():
secondThread.start()
def cleanup():
print 'cleaning up'
secondThread.stop()
print 'cleaned up'
atexit.register(cleanup)
def handleSignal(signalNum, frame):
print 'handleSignal'
cleanup()
sys.exit(0)
for signalNum in (signal.SIGINT, signal.SIGILL, signal.SIGABRT, signal.SIGFPE, signal.SIGSEGV, signal.SIGTERM):
signal.signal(signalNum, handleSignal)
def handleConsoleCtrl(signalNum):
print ('handleConsoleCtrl')
cleanup()
win32api.SetConsoleCtrlHandler(handleConsoleCtrl, True)
main()
The application is launched using
python.exe test.py
The console then prints "calc.exe running", and the Calculator application runs, and using Process Explorer, I can see calc.exe as a sub-process of python.exe
Then I kill the main process using
taskkill /pid XXXX /f
(where XXXX is the PID for python.exe)
What happens after this is that the command prompt returns without further output (i.e. none of "cleaning up", "handleSignal" or "handleConsoleCtrl" gets printed), the Calculator application continues running, and from Process Explorer, python.exe is no longer running but calc.exe has re-parented itself.

Taskkill (normally) sends WM_CLOSE. If your application is console only and has no window, while you can get CTRL_CLOSE_EVENT via a handler set by SetConsoleCtrlHandler (which happens if your controlling terminal window is closed) you can't receive a bare WM_CLOSE message.
If you have to stick with taskkill (rather than using a different program to send a Ctrl-C) one solution is to set the aforementioned handler and ensure your application has its own terminal window (e.g. by usingstart.exe "" <yourprog> to invoke it). See https://stackoverflow.com/a/23197789/4513656 for details an alternatives.

signal.SIGINT in windows cause process exit directly

I wrote those test python code as following:
import signal
import time
import os
def handler(signum, frame):
print "do whatever, like call thread.interrupt_main()"
return
signal.signal(signal.SIGINT, handler)
while 1:
try:
time.sleep(10)
except:
os.kill(int(os.getpid()), signal.SIGINT)
pass
when i excute this test code on windows, the process print "do whatever, like call thread.interrupt_main()", then exit;
on linux, it works correctly.
why on windows it not work?

http://docs.python.org/2/library/os.html#os.kill
Windows: The signal.CTRL_C_EVENT and signal.CTRL_BREAK_EVENT signals
are special signals which can only be sent to console processes which
share a common console window, e.g., some subprocesses. Any other
value for sig will cause the process to be unconditionally killed by
the TerminateProcess API, and the exit code will be set to sig.
Try following code:
import time
while 1:
try:
time.sleep(1)
except KeyboardInterrupt:
print "do whatever, like call thread.interrupt_main()"