I need to be able to kill a python process from another process. Here is an example of how I'm doing it now:
In the 'main' process:
# Write the ProcessID to tmp file
with open('/tmp/%s' % self.query_identifier, 'w') as f:
f.write(str(os.getpid()))
try:
cursor.execute('''very long query''')
except Exception:
do_some_other_stuff()
raise ConnectionError("There was an error completing this process")
And in the other process which 'kills' that process, I have:
pid = int(open('/tmp/%s' % self.query_identifier).read())
os.kill(pid, signal.SIGKILL)
This works great. However, this entirely terminates the python process, and so it doesn't ever get to the except block of code. What would be a better way to do the above? For example, so that I can do the "kill" operation from another separate process without terminating the python program.
The worker program:
import signal
# Define and register a signal handler
def handler(signum, frame):
raise IOError("Quitting on {}".format(signum))
signal.signal(signal.SIGINT, handler)
try:
while(True): # Imitate a long and winding road
pass
except IOError:
print("I've been killed!")
The supervisor program:
import os, signal
os.kill(pid, signal.SIGINT)
Related
Is there any argument or options to setup a timeout for Python's subprocess.Popen method?
Something like this:
subprocess.Popen(['..'], ..., timeout=20) ?
I would advise taking a look at the Timer class in the threading module. I used it to implement a timeout for a Popen.
First, create a callback:
def timeout( p ):
if p.poll() is None:
print 'Error: process taking too long to complete--terminating'
p.kill()
Then open the process:
proc = Popen( ... )
Then create a timer that will call the callback, passing the process to it.
t = threading.Timer( 10.0, timeout, [proc] )
t.start()
t.join()
Somewhere later in the program, you may want to add the line:
t.cancel()
Otherwise, the python program will keep running until the timer has finished running.
EDIT: I was advised that there is a race condition that the subprocess p may terminate between the p.poll() and p.kill() calls. I believe the following code can fix that:
import errno
def timeout( p ):
if p.poll() is None:
try:
p.kill()
print 'Error: process taking too long to complete--terminating'
except OSError as e:
if e.errno != errno.ESRCH:
raise
Though you may want to clean the exception handling to specifically handle just the particular exception that occurs when the subprocess has already terminated normally.
subprocess.Popen doesn't block so you can do something like this:
import time
p = subprocess.Popen(['...'])
time.sleep(20)
if p.poll() is None:
p.kill()
print 'timed out'
else:
print p.communicate()
It has a drawback in that you must always wait at least 20 seconds for it to finish.
import subprocess, threading
class Command(object):
def __init__(self, cmd):
self.cmd = cmd
self.process = None
def run(self, timeout):
def target():
print 'Thread started'
self.process = subprocess.Popen(self.cmd, shell=True)
self.process.communicate()
print 'Thread finished'
thread = threading.Thread(target=target)
thread.start()
thread.join(timeout)
if thread.is_alive():
print 'Terminating process'
self.process.terminate()
thread.join()
print self.process.returncode
command = Command("echo 'Process started'; sleep 2; echo 'Process finished'")
command.run(timeout=3)
command.run(timeout=1)
The output of this should be:
Thread started
Process started
Process finished
Thread finished
0
Thread started
Process started
Terminating process
Thread finished
-15
where it can be seen that, in the first execution, the process finished correctly (return code 0), while the in the second one the process was terminated (return code -15).
I haven't tested in windows; but, aside from updating the example command, I think it should work since I haven't found in the documentation anything that says that thread.join or process.terminate is not supported.
You could do
from twisted.internet import reactor, protocol, error, defer
class DyingProcessProtocol(protocol.ProcessProtocol):
def __init__(self, timeout):
self.timeout = timeout
def connectionMade(self):
#defer.inlineCallbacks
def killIfAlive():
try:
yield self.transport.signalProcess('KILL')
except error.ProcessExitedAlready:
pass
d = reactor.callLater(self.timeout, killIfAlive)
reactor.spawnProcess(DyingProcessProtocol(20), ...)
using Twisted's asynchronous process API.
A python subprocess auto-timeout is not built in, so you're going to have to build your own.
This works for me on Ubuntu 12.10 running python 2.7.3
Put this in a file called test.py
#!/usr/bin/python
import subprocess
import threading
class RunMyCmd(threading.Thread):
def __init__(self, cmd, timeout):
threading.Thread.__init__(self)
self.cmd = cmd
self.timeout = timeout
def run(self):
self.p = subprocess.Popen(self.cmd)
self.p.wait()
def run_the_process(self):
self.start()
self.join(self.timeout)
if self.is_alive():
self.p.terminate() #if your process needs a kill -9 to make
#it go away, use self.p.kill() here instead.
self.join()
RunMyCmd(["sleep", "20"], 3).run_the_process()
Save it, and run it:
python test.py
The sleep 20 command takes 20 seconds to complete. If it doesn't terminate in 3 seconds (it won't) then the process is terminated.
el#apollo:~$ python test.py
el#apollo:~$
There is three seconds between when the process is run, and it is terminated.
As of Python 3.3, there is also a timeout argument to the blocking helper functions in the subprocess module.
https://docs.python.org/3/library/subprocess.html
Unfortunately, there isn't such a solution. I managed to do this using a threaded timer that would launch along with the process that would kill it after the timeout but I did run into some stale file descriptor issues because of zombie processes or some such.
No there is no time out. I guess, what you are looking for is to kill the sub process after some time. Since you are able to signal the subprocess, you should be able to kill it too.
generic approach to sending a signal to subprocess:
proc = subprocess.Popen([command])
time.sleep(1)
print 'signaling child'
sys.stdout.flush()
os.kill(proc.pid, signal.SIGUSR1)
You could use this mechanism to terminate after a time out period.
Yes, https://pypi.python.org/pypi/python-subprocess2 will extend the Popen module with two additional functions,
Popen.waitUpTo(timeout=seconds)
This will wait up to acertain number of seconds for the process to complete, otherwise return None
also,
Popen.waitOrTerminate
This will wait up to a point, and then call .terminate(), then .kill(), one orthe other or some combination of both, see docs for full details:
http://htmlpreview.github.io/?https://github.com/kata198/python-subprocess2/blob/master/doc/subprocess2.html
For Linux, you can use a signal. This is platform dependent so another solution is required for Windows. It may work with Mac though.
def launch_cmd(cmd, timeout=0):
'''Launch an external command
It launchs the program redirecting the program's STDIO
to a communication pipe, and appends those responses to
a list. Waits for the program to exit, then returns the
ouput lines.
Args:
cmd: command Line of the external program to launch
time: time to wait for the command to complete, 0 for indefinitely
Returns:
A list of the response lines from the program
'''
import subprocess
import signal
class Alarm(Exception):
pass
def alarm_handler(signum, frame):
raise Alarm
lines = []
if not launch_cmd.init:
launch_cmd.init = True
signal.signal(signal.SIGALRM, alarm_handler)
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
signal.alarm(timeout) # timeout sec
try:
for line in p.stdout:
lines.append(line.rstrip())
p.wait()
signal.alarm(0) # disable alarm
except:
print "launch_cmd taking too long!"
p.kill()
return lines
launch_cmd.init = False
I have a main python(testmain.py) script that executes another python script(test.py) using subprocess.Popen command. When I press Ctrl-C , I want the child to exit with exit code 2 and then the parent to display that exit code and then terminate .
I have signal handlers in both parent and child scripts.
testmain.py
def signal_handler(signal, frame):
print "outer signal handler"
exit(2)
signal.signal(signal.SIGINT, signal_handler)
def execute()
proc=subprocess.Popen("python test.py",shell=True)
streamdata=proc.communicate()[0]
rc=proc.returncode
print "return code:",rc
execute()
test.py
def signal_handler(signal, frame):
print "exiting: inner function"
exit(2)
signal.signal(signal.SIGINT, signal_handler)
I checked Delegate signal handling to a child process in python that is kind of similar to my question but in that case, the parent is continuing it's execution, which I don't want.
I want to: 1.exit test.py with exit(2) 2.print that exit code in testmain.py 3.exit test.py with exit(2)
could someone please provide suggestions to do this?
Thanks.
UPDATE : Handling the signal only in the child (test.py) and checking the return code in parent(testmain.py) will do what I want .
if rc==2:
print "child was terminated"
exit(2)
but I was wondering if there is a clean way to do this using signal handling.
Your child process shouldn't care what the parent does i.e., if you want the child to exit with specific status on Ctrl+C then just do that:
import sys
try:
main()
except KeyboardInterrupt: # use default SIGINT handler
sys.exit(2)
Or you could define the signal handler explicitly:
import os
import signal
def signal_handler(signal, frame):
os.write(1, b"outer signal handler\n")
os._exit(2)
signal.signal(signal.SIGINT, signal_handler)
main()
There might be a difference in behavior if there are atexit handlers and/or multiple threads.
Unrelated: depending on what your main() function does, there could be a significant delay before a signal is handled in Python. Some blocking methods on Python 2 may ignore the signal completely: use Python 3 or apply a custom workaround for a specific case e.g., using a timeout parameter for some calls.
You could handle SIGINT in a similar way in the parent:
for cmd in commands:
process = Popen(cmd)
try:
process.wait()
except KeyboardInterrupt:
# child process may still be alive here
for _ in range(5): # wait a while
if process.poll() is not None:
break # the process is dead
time.sleep(.1)
else: # no break, kill the process explicitly
try:
process.kill()
except OSError:
pass
sys.exit("Child exited with %d" % process.wait())
Python 2 doesn't restore signals for child processes e.g., if you SIG_IGN the SIGINT signal in the parent, you could reset the necessary hanlders explicitly using preexec_fn parameter.
I have some threads running, and one of those threads contains an object that will be spawning subprocesses. I want one such subprocess to be able to kill the entire application. The aforementioned object will need to save some state when it receives this signal. Unfortunately I can't get the signal to be handled in the thread that causes the kill.
Here is some example code that attempts to replicate the situation.
parent.py: starts a thread. that thread runs some subprocesses, one of which will try to kill the parent process.
#!/usr/local/bin/python3
import subprocess, time, threading, random
def killer_func():
possible_cmds = [['echo', 'hello'],
['echo', 'world'],
['/work/turbulencetoo/tmp/killer.py']
]
random.shuffle(possible_cmds)
for cmd in possible_cmds:
try:
time.sleep(2)
subprocess.check_call(cmd)
time.sleep(2)
except KeyboardInterrupt:
print("Kill -2 caught properly!!")
print("Here I could properly save my state")
break
except Exception as e:
print("Unhandled Exception: {}".format(e))
else:
print("No Exception")
killer_thread = threading.Thread(target=killer_func)
killer_thread.start()
try:
while True:
killer_thread.join(4)
if not killer_thread.is_alive():
print("The killer thread has died")
break
else:
print("Killer thread still alive, try to join again.")
except KeyboardInterrupt:
print("Caught the kill -2 in the main thread :(")
print("Main program shutting down")
killer.py, a simple program that tries to kill its parent process with SIGINT:
#!/usr/local/bin/python3
import time, os, subprocess, sys
ppid = os.getppid()
# -2 specifies SIGINT, python handles this as a KeyboardInterrupt exception
cmd = ["kill", "-2", "{}".format(ppid)]
subprocess.check_call(cmd)
time.sleep(3)
sys.exit(0)
Here is some sample output from running the parent program:
$ ./parent.py
hello
Killer thread still alive, try to join again.
No Exception
Killer thread still alive, try to join again.
Caught the kill -2 in the main thread :(
Main program shutting down
No Exception
world
No Exception
I've tried using signal.signal() inside killer_func, but it doesn't work in a sub thread.
Is there a way to force the signal or exception to be handled by the function without the main thread being aware?
The main thread of your program will always be the one that receives the signal. The signal module documentation states this:
Some care must be taken if both signals and threads are used in the
same program. The fundamental thing to remember in using signals and
threads simultaneously is: always perform signal() operations in the
main thread of execution. Any thread can perform an alarm(),
getsignal(), pause(), setitimer() or getitimer(); only the main thread
can set a new signal handler, and the main thread will be the only one
to receive signals (this is enforced by the Python signal module, even
if the underlying thread implementation supports sending signals to
individual threads). This means that signals can’t be used as a means
of inter-thread communication. Use locks instead.
You'll need to refactor your program such that the main thread receiving the signal doesn't prevent you from saving state. The easiest way is use something like threading.Event() to tell the background thread that the program has been aborted, and let it clean up when it sees the event has been set:
import subprocess
import threading
import random
def killer_func(event):
possible_cmds = [['echo', 'hello'],
['echo', 'world'],
['/home/cycdev/killer.py']
]
random.shuffle(possible_cmds)
for cmd in possible_cmds:
subprocess.check_call(cmd)
event.wait(4)
if event.is_set():
print("Main thread got a signal. Time to clean up")
# save state here.
return
event = threading.Event()
killer_thread = threading.Thread(target=killer_func, args=(event,))
killer_thread.start()
try:
killer_thread.join()
except KeyboardInterrupt:
print("Caught the kill -2 in the main thread :)")
event.set()
killer_thread.join()
print("Main program shutting down")
Signals are always handled in the main thread. When you receive a signal, you don't know where it comes from. You can't say "handle it in the thread that spawned the signal-sending-process" because you don't know what signal-sending-process is.
The way to solve this is to use Condition Variables to notify all threads that a signal was received and that they have to shut down.
import threading
got_interrupt = False # global variable
def killer_func(cv):
...
with cv:
cv.wait(2)
interupted = got_interrupt # Read got_interrupt while holding the lock
if interrupted:
cleanup()
...
lock = threading.Lock()
notifier_cv = threading.Condition(lock)
killer_thread = threading.Thread(target=killer_func, args=(notifier_cv,))
killer_thread.start()
try:
...
except KeyboardInterrupt:
with cv:
got_interrupt = True
cv.notify_all()
I wrote those test python code as following:
import signal
import time
import os
def handler(signum, frame):
print "do whatever, like call thread.interrupt_main()"
return
signal.signal(signal.SIGINT, handler)
while 1:
try:
time.sleep(10)
except:
os.kill(int(os.getpid()), signal.SIGINT)
pass
when i excute this test code on windows, the process print "do whatever, like call thread.interrupt_main()", then exit;
on linux, it works correctly.
why on windows it not work?
http://docs.python.org/2/library/os.html#os.kill
Windows: The signal.CTRL_C_EVENT and signal.CTRL_BREAK_EVENT signals
are special signals which can only be sent to console processes which
share a common console window, e.g., some subprocesses. Any other
value for sig will cause the process to be unconditionally killed by
the TerminateProcess API, and the exit code will be set to sig.
Try following code:
import time
while 1:
try:
time.sleep(1)
except KeyboardInterrupt:
print "do whatever, like call thread.interrupt_main()"
I have my main application thread that spawns 2 threads and I catch SIGINT in my main thread to quit them nicely.
On linux, I'm using signal.pause() and it works perfectly.
What is the best way to implement signal.pause() on Windows?
My ugly solution is:
my_queue.get(True, averylongtime)
And put something in my_queue in my signal handler. Note that if I don't specify a timeout, SIGINT is not caught. But I wonder if there's a better solution.
Thank you
I use this:
#another:
while not self.quit:
# your code
# main
try:
# your code
except KeyboardInterrupt:
another.quit = True
time.sleep(5) # or wait for threading.enumerate() or similar
If I want it more robust, say, exit in presence of bugs too:
except KeyboardInterrupt:
another.quit = True
signal.alarm(5)
time.sleep(6)
A side effect to this is that every block where you except: or except Exception, e: (which is not something you should do anyway/much) you have to prepend except KeyboardInterrupt: raise so that the exception is not "eaten".
I use this for catching a ctrl-c on windows. In case I'm writing to a pipe or file or what have you.. I want to exit gracefully. Below is a toy example
import signal
import sys
def signal_handler(signal, frame):
print('Process Interrupted!\n\a')
sys.exit(0)
signal.signal(signal.SIGINT,signal_handler)
#Rest of your code