I want to be able to start a process and then be able to kill it afterwards
Here's a little python script that starts a process, checks if it is running, waits a while, kills it, waits for it to terminate, then checks again. It uses the 'kill' command. Version 2.6 of python subprocess has a kill function. This was written on 2.5.
import subprocess
import time
proc = subprocess.Popen(["sleep", "60"], shell=False)
print 'poll =', proc.poll(), '("None" means process not terminated yet)'
time.sleep(3)
subprocess.call(["kill", "-9", "%d" % proc.pid])
proc.wait()
print 'poll =', proc.poll()
The timed output shows that it was terminated after about 3 seconds, and not 60 as the call to sleep suggests.
$ time python prockill.py
poll = None ("None" means process not terminated yet)
poll = -9
real 0m3.082s
user 0m0.055s
sys 0m0.029s
Have a look at the subprocess module.
You can also use low-level primitives like fork() via the os module.
http://docs.python.org/library/os.html#process-management
A simple function that uses subprocess module:
def CMD(cmd) :
p = subprocess.Popen(cmd, shell=True,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
close_fds=False)
return (p.stdin, p.stdout, p.stderr)
see docs for primitive fork() and modules subprocess, multiprocessing, multithreading
If you need to interact with the sub process at all, I recommend the pexpect module (link text). You can send input to the process, receive (or "expect") output in return, and you can close the process (with force=True to send SIGKILL).
Related
I need to save some image files from my simulation at different times. So my idea was to open a subprocess save some image files and close it .
import subprocess
cmd = "rosrun pcl_ros pointcloud_to_pcd input:=camera/depth/points"
proc = subprocess.Popen(cmd, shell=True)
When it comes to closing I tried different things:
import os
import signal
import subprocess
cmd = "rosrun pcl_ros pointcloud_to_pcd input:=camera/depth/points"
pro = subprocess.Popen(cmd, stdout=subprocess.PIPE,
shell=True, preexec_fn=os.setsid)
os.killpg(os.getpgid(pro.pid), signal.SIGTERM)
command did not execute , so it doesn't work for me. I also tried a solution with psutil and it didn't work neither...
you probably don't need shell=True here, which is the cause of your problems. I suspect that when you kill the process group in your second snippet, the shell process is killed before the process you want to run has a chance to start...
Try to pass the parameters as a list of strings (so you don't need shell=True), wait a bit, and use terminate on the Popen object. You don't need process group, or psutil to kill the process & its children, just plain old terminate() on the process object does the trick.
cmd = ["rosrun","pcl_ros","pointcloud_to_pcd","input:=camera/depth/points"]
proc = subprocess.Popen(cmd)
time.sleep(1) # maybe needed to wait the process to do something useful
proc.terminate()
Note that proc.terminate() tries to exit gracefully where proc.kill() would have just killed the process (there's a difference under Un*x systems, not under Windows)
Another plea for "do not use shell=True unless forced at gunpoint".
I want to use a timeout on a subprocess
from subprocess32 import check_output
output = check_output("sleep 30", shell=True, timeout=1)
Unfortunately, whilst this raises a timeout error, it does so after 30 seconds. It seems that check_output cannot interrupt the shell command.
What can I do on on the Python side to stop this?
I suspect that subprocess32 fails to kill the timed out process.
check_output() with timeout is essentially:
with Popen(*popenargs, stdout=PIPE, **kwargs) as process:
try:
output, unused_err = process.communicate(inputdata, timeout=timeout)
except TimeoutExpired:
process.kill()
output, unused_err = process.communicate()
raise TimeoutExpired(process.args, timeout, output=output)
There are two issues:
[the second] .communicate() may wait for descendant processes, not just for the immediate child, see Python subprocess .check_call vs
.check_output
process.kill() might not kill the whole process tree, see How to terminate a python subprocess launched with shell=True
It leads to the behaviour that you observed: the TimeoutExpired happens in a second, the shell is killed, but check_output() returns only in 30 seconds after the grandchild sleep process exits.
To workaround the issues, kill the whole process tree (all subprocesses that belong to the same group):
#!/usr/bin/env python3
import os
import signal
from subprocess import Popen, PIPE, TimeoutExpired
from time import monotonic as timer
start = timer()
with Popen('sleep 30', shell=True, stdout=PIPE, preexec_fn=os.setsid) as process:
try:
output = process.communicate(timeout=1)[0]
except TimeoutExpired:
os.killpg(process.pid, signal.SIGINT) # send signal to the process group
output = process.communicate()[0]
print('Elapsed seconds: {:.2f}'.format(timer() - start))
Output
Elapsed seconds: 1.00
Update for Python 3.6.
This is still happening but I have tested a lot of combinations of check_output, communicate and run methods and now I have a clear knowledge about where is the bug and how to avoid it in a easy way on Python 3.5 and Python 3.6.
My conclusion: It happens when you mix the use shell=True and any PIPE on stdout, stderr or stdin parameters (used in Popen and run methods).
Be careful: check_output uses PIPE inside.
If you look at the code inside on Python 3.6 it is basically a call to run with stdout=PIPE: https://github.com/python/cpython/blob/ae011e00189d9083dd84c357718264e24fe77314/Lib/subprocess.py#L335
So, to solve #innisfree problem on Python 3.5 or 3.6 just do this:
check_output(['sleep', '30'], timeout=1)
And for other cases, just avoid mixing shell=True and PIPE, keeping in mind that check_output uses PIPE.
I try to run simple script in windows in the same shell.
When I run
subprocess.call(["python.exe", "a.py"], shell=False)
It works fine.
But when I run
subprocess.Popen(["python.exe", "a.py"], shell=False)
It opens new shell and the shell=false has no affect.
a.py just print message to the screen.
First calling Popen with shell=False doesn't mean that the underlying python won't try to open a window/console. It's just that the current python instance executes python.exe directly and not in a system shell (cmd or sh).
Second, Popen returns a handle on the process, and you have to perform a wait() on this handle for it to end properly or you could generate a defunct process (depending on the platform you're running on). I suggest that you try
p = subprocess.Popen(["python.exe", "a.py"], shell=False)
return_code = p.wait()
to wait for process termination and get return code.
Note that Popen is a very bad way to run processes in background. The best way would be to use a separate thread
import subprocess
import threading
def run_it():
subprocess.call(["python.exe", "a.py"], shell=False)
t = threading.Thread(target=run_it)
t.start()
# do your stuff
# in the end
t.join()
Using subprocess and the command 'gnome-terminal -e bash' I can open up a gnome-terminal as desired (and have it stick around). This is done with either
p=subprocess.Popen(['gnome-terminal', '-e', 'bash'])
or
p=subprocess.Popen(['gnome-terminal -e bash'], shell=True)
but I cannot close the terminal using p.terminate() or p.kill(). From what I understand, this is a little trickier when using shell=True but I did not expect to run into problems otherwise.
To terminate a terminal and its children (in the same process group):
#!/usr/bin/env python
import os
import signal
import subprocess
p = subprocess.Popen(['gnome-terminal', '--disable-factory', '-e', 'bash'],
preexec_fn=os.setpgrp)
# do something here...
os.killpg(p.pid, signal.SIGINT)
--disable-factory is used to avoid re-using an active terminal so that we can kill newly created terminal via the subprocess handle
os.setpgrp puts gnome-terminal in its own process group so that os.killpg() could be used to send signal to this group
You should be able to do this workaround:
get the process id
kill the process
Working Solution: Close gnome-terminal-server
As suggested by #j-f-sebastian in the comment, gnome-terminal
just sends the request (to gnome-terminal-server) to start a new terminal and exits immediately -- there is nothing to kill the process is already dead (and newly created processes are not descendants: the new bash process is a child of gnome-terminal-server, not gnome-terminal).
import subprocess
import os, signal
import time
p=subprocess.Popen(['gnome-terminal -e bash'], stdout=subprocess.PIPE, shell=True, preexec_fn=os.setsid)
print "this is going to be closed in 3 sec"
time.sleep(3)
# this line returns the list of bash instances pid as string
bash_pids = subprocess.check_output(["pidof", "bash"])
# I get the last instance opened
pid_to_kill = bash_pids.split(" ")[0]
os.kill(int(pid_to_kill), signal.SIGTERM)
My solution is following this logic:
run gnome-terminal
get the latest bash instance opened process id
kill this process id
Broken solutions
These solutions might work in simpler cases:
Solution 1
import subprocess
import os, signal
p=subprocess.Popen(['gnome-terminal -e bash'], shell=True)
p_pid = p.pid # get the process id
os.kill(p_pid, signal.SIGKILL)
In order to choose the appropriate method of signal to pass instead of SIGKILL you can refer the signal documentation. E.g.
On Windows, signal() can only be called with SIGABRT, SIGFPE, SIGILL, SIGINT, SIGSEGV, or SIGTERM
For Unix you have a quite extensive list of method to call.
To have a better overview about os.kill, you can refer its documentation.
Solution 2
An alternative method useful for Unix could be:
import subprocess
import os, signal
p=subprocess.Popen(['gnome-terminal -e bash'], stdout=subprocess.PIPE, shell=True, preexec_fn=os.setsid)
os.killpg(os.getpgid(p.pid), signal.SIGTERM)
It seems that your process is opening child process that prevent the parent to be close. Adding a session id to your parent process, you should be able to fix it.
Solution 3
import subprocess, psutil
def kill(p_pid):
process = psutil.Process(p_pid)
for proc in process.get_children(recursive=True):
proc.kill()
process.kill()
p = subprocess.Popen(['gnome-terminal -e bash'], shell=True)
try:
p.wait(timeout=3)
except subprocess.TimeoutExpired:
kill(p.pid)
This solution requires psutil.
Solution 4
According to askubuntu, it seems that the best way to close a gnome terminal instance would be to execute a bash command like:
killall -s {signal} gnome-terminal
where {signal} simulates Alt + F4.
You can try to do it using [pexpect]:
p = pexpect.spawn(your_cmd_here)
p.send('^F4')
I wanted to add this snippet for anyone who is running on Linux Ubuntu and trying to open a subprocess, run a script, and terminate it after a time.wait().
I found a litany of solutions that would open a window, but not close it. Or a solution would open a window, and close it, but wouldn't run the script inside the terminal.
There was no exact answer so I had to hack together several solutions, as I am a novice when it comes t subprocess/shell.
This snippet was able to open a subprocess, run the script, and when 10 seconds had passed the subprocess was terminated. Again, this was built ofn the shoulders of giants. I hope this saves someone time; cheers.
import os
import signal
import subprocess
import time
command = 'python3 Rmonitor.py'
p = subprocess.Popen(['gnome-terminal','--disable-factory', '--', 'bash', '-c', command],preexec_fn=os.setpgrp)
time.sleep(10)
os.killpg(p.pid, signal.SIGINT)
I have tried both terminate() and kill() but both have failed to stop a subprocess I start in my python code.
Is there any other way?
On Windows with Python 2.7
I have also tried the following with no results...
os.kill(p.pid, signal.SIGTERM)
and
import ctypes
PROCESS_TERMINATE = 1
handle = ctypes.windll.kernel32.OpenProcess(PROCESS_TERMINATE, False, theprocess.pid)
ctypes.windll.kernel32.TerminateProcess(handle, -1)
ctypes.windll.kernel32.CloseHandle(handle)
You could use the os.system('taskkill') here:http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/taskkill.mspx?mfr=true
Try using psutil, or another way to remotely kill the process itself (psutil is cross platform so the code is nicer):
p = psutil.Process(pid)
p.terminate() #or p.kill()
Code taken from How to terminate process from Python using pid?.
Note that if using shell=True, the PID is of the shell and not any process it spawned. To kill a subprocess with shell=True you may look at How to terminate a python subprocess launched with shell=True