Develop Reference

Python 3 Stopping subprocess by sending Ctrl C

I have some GPU test software i'm trying to automate using python3, The test would normally be run for 3 minutes then cancelled by a user using ctrl+c generating the following output
After exiting with ctrl+c the test can then be run again with no issue
When trying to automate this with subprocess popen and sending SIGINT or SIGTERM i'm not getting the same as if keyboard entry was used. The script exits abruptly and on subsequent runs cant find the gpus (assume its not unloading the driver properly)
from subprocess import Popen, PIPE
from signal import SIGINT
from time import time
def check_subproc_alive(subproc):
return subproc.poll() is None
def print_subproc(subproc, timer=True):
start_time = time()
while check_subproc_alive(subproc):
line = subproc.stdout.readline().decode('utf-8')
print(line, end="")
if timer and (time() - start_time) > 10:
break
subproc = Popen(['./gpu_test.sh', '-t', '1'], stdin=PIPE, stdout=PIPE, stderr=PIPE, shell=False)
print_subproc(subproc)
subproc.send_signal(SIGINT)
print_subproc(subproc, False)
How can I send ctrl+c to a subprocess as if a user typed it?
**UPDATE
import subprocess
def start(executable_file):
return subprocess.Popen(
executable_file,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
)
def read(process):
return process.stdout.readline().decode("utf-8").strip()
def write(process):
process.stdin.write('\x03'.encode())
process.stdin.flush()
def terminate(process):
process.stdin.close()
process.terminate()
process.wait(timeout=0.2)
process = start("./test.sh")
write(process)
for x in range(100):
print(read(process))
terminate(process)
Tried the above code and can get characters to register with dummy sh script however sending the \x03 command just sends an empty char and doesn't end script

I think you can probably use something like this:
import signal
try:
p=subprocess...
except KeyboardInterrupt:
p.send_signal(signal.SIGINT)

The following solution is the only one I could find that works for windows and is the closest resemblance to sending a Ctrl+C event.
import signal
os.kill(self.p.pid, signal.CTRL_C_EVENT)

Python kill a subprocess(that starts another process) and start it again

I'm trying to make a python script that starts the program livestreamer (that starts the program mplayer) and after 10 seconds it should kill the program, or the subprocess. here is my current code that doesn't work, I think I know why but I don't know how to solve it.
I think the problem is that the subprocess starts livestreamer and then the program livestreamer starts the program mplayer. Python doesn't know about mplayer and can't close it. How would I be able to kill both livestreamer and mplayer after 10 second and then start them again as a loop?
I'm using Ubuntu 14.04 (Linux) and Python 2.7.6
import subprocess
import time
import os
import sys
import signal
url = "http://new.livestream.com/accounts/398160/events/3155348"
home = os.environ['HOME']
if not os.geteuid() == 0:
if not os.path.exists('/%s/.config/livestreamer' % home):
os.makedirs('/%s/.config/livestreamer' % home)
lscfg = open('%s/.config/livestreamer/config' % home, 'w+')
lscfg.write("player=mplayer -geometry 0%:0% -nomouseinput -loop 100 -noborder -fixed-vo")
lscfg.close()
cmd = "livestreamer %s best --player-continuous-http --player-no-close" % url
while True:
proc1 = subprocess.Popen(cmd.split(), shell=False)
time.sleep(10)
proc1.kill()
Solution:
import subprocess
import time
import os
import sys
import signal
url = "http://new.livestream.com/accounts/398160/events/3155348"
home = os.environ['HOME']
if not os.geteuid() == 0:
if not os.path.exists('/%s/.config/livestreamer' % home):
os.makedirs('/%s/.config/livestreamer' % home)
lscfg = open('%s/.config/livestreamer/config' % home, 'w+')
lscfg.write("player=mplayer -geometry 0%:0% -nomouseinput -loop 100 -noborder -fixed-vo")
lscfg.close()
cmd = "livestreamer %s best --player-continuous-http --player-no-close" % url
#restarting the player every 10th minute to catch up on possible delay
while True:
proc1 = subprocess.Popen(cmd.split(), shell=False)
time.sleep(600)
os.system("killall -9 mplayer")
proc1.kill()
As you can see os.system("killall -9 mplayer") was the command to kill the process mplayer.

In your code you kill livestreamer but not mplayer so mplayer will continue running.
By using kill on your subprocess you send a signal SIGKILL and unless the subprocess do handle the signal interruption it will simply close itself fast and without killing his own childs so mplayer will live (and may become a zombie process).
You have no reference to your subprocess child 'mplayer' but if you can get his PID you can kill it with os.kill(...)
os.kill(process_pid, signal.SIGTERM)

Using os.system("killall -9 mplayer") was the easy way to solve this. Mind using this option will kill all process of mplayer though this is not a problem in my case but may be a problem for other cases.
while True:
proc1 = subprocess.Popen(cmd.split(), shell=False)
time.sleep(600)
os.system("killall -9 mplayer")
proc1.kill()

Daemon thread launching software won't die

I'm trying to write a small script which will use plink.exe (from the same folder) to create a ssh tunnel (on windows).
I'm basically using os.system to launch the the command:
import time
import threading
from os.path import join, dirname, realpath
pc_tunnel_command = '-ssh -batch -pw xxxx -N -L 1234:host1:5678 user#host2'
if __name__ == '__main__':
t = threading.Thread(target = os.system, \
args = (join(dirname(realpath(__file__)), 'plink.exe ') + \
pc_tunnel_command,))
t.daemon = True
t.start()
#without this line it will die. I guess that plink doesn't have enough time to start.
time.sleep(5)
print 'Should die now'
However, it seems that the thread (and plink.exe) keep running. Why is this happening? Any way to force the thread to close? Better way to launch plink?
I want plink.exe to die when my program ends. Using a daemon thread was my plan of having the tunnel run in the background, and then dying when my main code exits.
BTW - same thing happens with subprocess.call.

You can use the atexit and signal modules to register calls back that will explicitly kill the process when your program exits normally or receives SIGTERM, respectively:
import sys
import time
import atexit
import signal
import subprocess
from functools import partial
from os.path import join, dirname, realpath
pc_tunnel_command = '-ssh -batch -pw xxxx -N -L 1234:host1:5678 user#host2'
def handle_exit(p, *args):
print("killing it")
p.terminate()
sys.exit(0)
if __name__ == '__main__':
p = subprocess.Popen(join(dirname(realpath(__file__)), 'plink.exe ') + pc_tunnel_command, shell=True)
func = partial(handle_exit, p)
signal.signal(signal.SIGTERM, func)
atexit.register(func)
print 'Should die now'
The one thing that is odd about the behavior your desrcibed is that I would have expected your program to exit after your sleep call, but leave plink running in the background, rather than having your program hang until the os.system call completes. That's the behavior I see on Linux, at least. In any case, explicitly terminating the child process should solve the issue for you.

os.system does not return until the child process exits. The same is true for subprocess.call. That's why your thread is sitting there, waiting for plink to finish. You can probably use subprocess.Popen to launch the process asynchronously and then exit. In any case, the additional thread you are creating is unnecessary.

Process execution check and getting PID in Python

I need to run a bash command in the background but then need to kill(os.kill()) it later. I also want to make Sure the command runs I have this to make sure the command runs.
if subprocess.Popen("tcpdump -i eth0 -XX -w /tmp/tmp.cap &", shell=True).wait() == 0:
I'm not sure on how to change this so i can use Popen.pid to get the pid while still being able to check if execution was successful.
Any help would be appreciated.
Thanks.

To start a subprocess, wait some time and kill it, and check that its exit status is zero:
import shlex
from subprocess import Popen
from threading import Timer
def kill(process):
try:
process.kill()
except OSError:
pass # ignore
p = Popen(shlex.split("tcpdump -i eth0 -XX -w /tmp/tmp.cat"))
t = Timer(10, kill, [p]) # run kill in 10 seconds
t.start()
returncode = p.wait()
t.cancel()
if returncode != 0:
# ...
Or you could implement the timeout yourself:
import shlex
from subprocess import Popen
from time import sleep, time as timer # use time.monotonic instead
p = Popen(shlex.split("tcpdump -i eth0 -XX -w /tmp/tmp.cat"))
deadline = timer() + 10 # kill in 10 seconds if not complete
while timer() < deadline:
if p.poll() is not None: # process has finished
break
sleep(1) # sleep a second
else: # timeout happened
try:
p.kill()
except OSError:
pass
if p.wait() != 0:
# ...
It assumes sleep uses similar clock as timer.
threading.Timer variant allows your code to continue as soon as the subprocess exits.

Use the Popen.poll() method. You can get the Popen.returncode as well to determine if the processes completed successfully.
import subprocess
tasks = [subprocess.Popen('ping www.stackoverflow.com -n 5 && exit 0', shell=True),
subprocess.Popen('ping www.stackoverflow.com -n 5 && exit 1', shell=True)]
for task in tasks:
while task.poll() is None:
# the task has not finished
pass
print task
print task.pid
print task.returncode

How to limit program's execution time when using subprocess?

I want to use subprocess to run a program and I need to limit the execution time. For example, I want to kill it if it runs for more than 2 seconds.
For common programs, kill() works well. But if I try to run /usr/bin/time something, kill() can’t really kill the program.
My code below seems doesn’t work well. The program is still running.
import subprocess
import time
exec_proc = subprocess.Popen("/usr/bin/time -f \"%e\\n%M\" ./son > /dev/null", stdout = subprocess.PIPE, stderr = subprocess.STDOUT, shell = True)
max_time = 1
cur_time = 0.0
return_code = 0
while cur_time <= max_time:
if exec_proc.poll() != None:
return_code = exec_proc.poll()
break
time.sleep(0.1)
cur_time += 0.1
if cur_time > max_time:
exec_proc.kill()

If you're using Python 2.6 or later, you can use the multiprocessing module.
from multiprocessing import Process
def f():
# Stuff to run your process here
p = Process(target=f)
p.start()
p.join(timeout)
if p.is_alive():
p.terminate()
Actually, multiprocessing is the wrong module for this task since it is just a way to control how long a thread runs. You have no control over any children the thread may run. As singularity suggests, using signal.alarm is the normal approach.
import signal
import subprocess
def handle_alarm(signum, frame):
# If the alarm is triggered, we're still in the exec_proc.communicate()
# call, so use exec_proc.kill() to end the process.
frame.f_locals['self'].kill()
max_time = ...
stdout = stderr = None
signal.signal(signal.SIGALRM, handle_alarm)
exec_proc = subprocess.Popen(['time', 'ping', '-c', '5', 'google.com'],
stdin=None, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
signal.alarm(max_time)
try:
(stdout, stderr) = exec_proc.communicate()
except IOError:
# process was killed due to exceeding the alarm
finally:
signal.alarm(0)
# do stuff with stdout/stderr if they're not None

do it like so in your command line:
perl -e 'alarm shift #ARGV; exec #ARGV' <timeout> <your_command>
this will run the command <your_command> and terminate it in <timeout> second.
a dummy example :
# set time out to 5, so that the command will be killed after 5 second
command = ['perl', '-e', "'alarm shift #ARGV; exec #ARGV'", "5"]
command += ["ping", "www.google.com"]
exec_proc = subprocess.Popen(command)
or you can use the signal.alarm() if you want it with python but it's the same.

I use os.kill() but am not sure if it works on all OSes.
Pseudo code follows, and see Doug Hellman's page.
proc = subprocess.Popen(['google-chrome'])
os.kill(proc.pid, signal.SIGUSR1)</code>