I have two python files:
a.py:
import subprocess, time, os, signal
myprocess = subprocess.Popen("b.py", shell=True)
time.sleep(2)
os.kill(myprocess.pid, signal.SIGTERM)
b.py:
import atexit
def cleanup():
print "Cleaning up things before the program exits..."
atexit.register(cleanup)
print "Hello world!"
while True:
pass
a.py is spawning b.py and after 2 seconds it is killing the process. The problem is that I want the cleanup function to call in b.py before it gets killed but I can't get it to work.
I also tried SIGKILL and SIGINT in the os.kill function but neither worked for me.
Current output (a.py):
Hello, World!
(2 seconds later, program ends)
Expected output (a.py):
Hello, World!
(2 seconds later)
Cleaning up things before the program exits...
(program ends)
Use a different signal for Windows platform: signal.CTRL_C_EVENT
Put some more sleep into a.py, otherwise the child process does not get a chance to clean up before the parent process exits:
import subprocess, time, os, signal
myprocess = subprocess.Popen("b.py", shell=True)
time.sleep(2)
os.kill(myprocess.pid, signal.CTRL_C_EVENT)
time.sleep(2)
I also want to discourage you from using the shell, if you don't actually need the shell features:
import subprocess, time, os, signal, sys
myprocess = subprocess.Popen([sys.executable, "b.py"])
Linux/macOS users: signal.CTRL_C_EVENT doesn't exist, you want signal.SIGINT.
Related
I'm trying to capture the stdout of a Popen object while it's running and display this data on a gui and log it. However whenever I try and read from the stdout attribute my program freezes. Minimal working code below. 'here' prints, then the process string representation, but then it hangs when it tries to read the first byte of stdout. Why is this the case?
Main script
import subprocess
import os
from threading import Thread
def print_to_terminal(process):
print(process)
print(process.stdout.read(1), flush=True)
sys.stdout.flush()
runner = subprocess.Popen(['python', 'print_and_wait.py'], env=os.environ, stdout=subprocess.PIPE)
print('here')
t = Thread(target=print_to_terminal, args=[runner]).run()
print('there')
runner.wait()
script Popen is calling
from time import sleep
for _ in range(10):
print('hello')
sleep(1)
After comments: This did work if I added a flush to the print in the print_and_wait function. See below
from time import sleep
for _ in range(10):
print('hello', flush=True)
sleep(1)
While I try to kill a python process, the child process started via os.system won't be terminated at the same time.
Killing child process when parent crashes in python and
Python Process won't call atexit
(atexit looks like not work with signal)
Does that mean I need to handle this situation by myself? If so, what is the preferred way to do so?
> python main.py
> ps
4792 ttys002 0:00.03 python run.py
4793 ttys002 0:00.03 python loop.py
> kill -15 4792
> ps
4793 ttys002 0:00.03 python loop.py
Sample Code:
main.py
import os
os.system('python loop.py')
loop.py
import time
while True:
time.sleep(1000)
UPDATE1
I did some experiment, and find out a workable version but still confuse about the logic.
import os
import sys
import signal
import subprocess
def sigterm_handler(_signo, _stack_frame):
# it raises SystemExit(0):
print 'go die'
sys.exit(0)
signal.signal(signal.SIGTERM, sigterm_handler)
try:
# os.system('python loop.py')
# use os.system won't work, it will even ignore the SIGTERM entirely for some reason
subprocess.call(['python', 'loop.py'])
except:
os.killpg(0, signal.SIGKILL)
kill -15 4792 sends SIGTERM to run.py in your example -- it sends nothing to loop.py (or its parent shell). SIGTERM is not propagated to other processes in the process tree by default.
os.system('python loop.py') starts at least two processes the shell and python process. You don't need it; use subprocess.check_call(), to run a single child process without the implicit shell. btw, if your subprocess is a Python script; consider importing it and running corresponding functions instead.
os.killpg(0, SIGKILL) sends SIGKILL signal to the current process group. A shell creates a new process group (a job) for each pipeline and therefore the os.killpg() in the parent has no effect on the child (see the update). See How to terminate a python subprocess launched with shell=True.
#!/usr/bin/env python
import subprocess
import sys
try:
p = subprocess.Popen([executable, 'loop'])
except EnvironmentError as e: #
sys.exit('failed to start %r, reason: %s' % (executable, e))
else:
try: # wait for the child process to finish
p.wait()
except KeyboardInterrupt: # on Ctrl+C (SIGINT)
#NOTE: the shell sends SIGINT (on CtrL+C) to the executable itself if
# the child process is in the same foreground process group as its parent
sys.exit("interrupted")
Update
It seems os.system(cmd) doesn't create a new process group for cmd:
>>> import os
>>> os.getpgrp()
16180
>>> import sys
>>> cmd = sys.executable + ' -c "import os; print(os.getpgrp())"'
>>> os.system(cmd) #!!! same process group
16180
0
>>> import subprocess
>>> import shlex
>>> subprocess.check_call(shlex.split(cmd))
16180
0
>>> subprocess.check_call(cmd, shell=True)
16180
0
>>> subprocess.check_call(cmd, shell=True, preexec_fn=os.setpgrp) #!!! new
18644
0
and therefore os.system(cmd) in your example should be killed by the os.killpg() call.
Though if I run it in bash; it does create a new process group for each pipeline:
$ python -c "import os; print(os.getpgrp())"
25225
$ python -c "import os; print(os.getpgrp())"
25248
I'm trying to write a small script which will use plink.exe (from the same folder) to create a ssh tunnel (on windows).
I'm basically using os.system to launch the the command:
import time
import threading
from os.path import join, dirname, realpath
pc_tunnel_command = '-ssh -batch -pw xxxx -N -L 1234:host1:5678 user#host2'
if __name__ == '__main__':
t = threading.Thread(target = os.system, \
args = (join(dirname(realpath(__file__)), 'plink.exe ') + \
pc_tunnel_command,))
t.daemon = True
t.start()
#without this line it will die. I guess that plink doesn't have enough time to start.
time.sleep(5)
print 'Should die now'
However, it seems that the thread (and plink.exe) keep running. Why is this happening? Any way to force the thread to close? Better way to launch plink?
I want plink.exe to die when my program ends. Using a daemon thread was my plan of having the tunnel run in the background, and then dying when my main code exits.
BTW - same thing happens with subprocess.call.
You can use the atexit and signal modules to register calls back that will explicitly kill the process when your program exits normally or receives SIGTERM, respectively:
import sys
import time
import atexit
import signal
import subprocess
from functools import partial
from os.path import join, dirname, realpath
pc_tunnel_command = '-ssh -batch -pw xxxx -N -L 1234:host1:5678 user#host2'
def handle_exit(p, *args):
print("killing it")
p.terminate()
sys.exit(0)
if __name__ == '__main__':
p = subprocess.Popen(join(dirname(realpath(__file__)), 'plink.exe ') + pc_tunnel_command, shell=True)
func = partial(handle_exit, p)
signal.signal(signal.SIGTERM, func)
atexit.register(func)
print 'Should die now'
The one thing that is odd about the behavior your desrcibed is that I would have expected your program to exit after your sleep call, but leave plink running in the background, rather than having your program hang until the os.system call completes. That's the behavior I see on Linux, at least. In any case, explicitly terminating the child process should solve the issue for you.
os.system does not return until the child process exits. The same is true for subprocess.call. That's why your thread is sitting there, waiting for plink to finish. You can probably use subprocess.Popen to launch the process asynchronously and then exit. In any case, the additional thread you are creating is unnecessary.
I'm struggling with some processes I started with Popen and which start subprocesses. When I start these processes manually in a terminal every process terminates as expected if I send CTRL+C. But running inside a python program using subprocess.Popen any attempt to terminate the process only gets rid of the parent but not of its children.
I tried .terminate() ..kill() as well as ..send_signal() with signal.SIGBREAK, signal.SIGTERM, but in every case I just terminate the parent process.
With this parent process I can reproduce the misbehavior:
#!/usr/bin/python
import time
import sys
import os
import subprocess
import signal
if __name__ == "__main__":
print os.getpid(), "MAIN: start a process.."
p = subprocess.Popen([sys.executable, 'process_to_shutdown.py'])
print os.getpid(), "MAIN: started process", p.pid
time.sleep(2)
print os.getpid(), "MAIN: kill the process"
# these just terminate the parent:
#p.terminate()
#p.kill()
#os.kill(p.pid, signal.SIGINT)
#os.kill(p.pid, signal.SIGTERM)
os.kill(p.pid, signal.SIGABRT)
p.wait()
print os.getpid(), "MAIN: job done - ciao"
The real life child process is manage.py from Django which spawns a few subprocesses and waits for CRTL-C. But the following example seems to work, too:
#!/usr/bin/python
import time
import sys
import os
import subprocess
if __name__ == "__main__":
timeout = int(sys.argv[1]) if len(sys.argv) >= 2 else 0
if timeout == 0:
p = subprocess.Popen([sys.executable, '-u', __file__, '13'])
print os.getpid(), "just waiting..."
p.wait()
else:
for i in range(timeout):
time.sleep(1)
print os.getpid(), i, "alive!"
sys.stdout.flush()
print os.getpid(), "ciao"
So my question in short: how do I kill the process in the first example and get rid of the child processes as well? On windows os.kill(p.pid, signal.CTRL_C_EVENT) seems to work in some cases, but what's the right way to do it? And how does a Terminal do it?
Like Henri Korhonen mentioned in a comment, grouping processes should help. Additionally, if you are on Windows and this is Cygwin Python that starts Windows applications, it appears Cygwin Python can not kill the children. For those cases you would need to run TASKKILL. TASKKILL also takes a group parameter.
can I use Popen from python subprocess to close started process? For example, from popen I run some application. In some part of my code I have to close that ran app.
For example, from console in Linux I do:
./some_bin
... It works and logs stdout here ...
Ctrl + C and it breaks
I need something like Ctrl + C but in my program code.
from subprocess import Popen
process = Popen(['slow', 'running', 'program'])
while process.poll():
if raw_input() == 'Kill':
if process.poll(): process.kill()
kill() will kill a process. See more here: Python subprocess module
Use the subprocess module.
import subprocess
# all arguments must be passed one at a time inside a list
# they must all be string elements
arguments = ["sleep", "3600"] # first argument is the program's name
process = subprocess.Popen(arguments)
# do whatever you want
process.terminate()
Some time ago I needed a 'gentle' shutdown for a process by sending CTRL+C in Windows console.
Here's what I have:
import win32api
import win32con
import subprocess
import time
import shlex
cmdline = 'cmd.exe /k "timeout 60"'
args = shlex.split(cmdline)
myprocess = subprocess.Popen(args)
pid = myprocess.pid
print(myprocess, pid)
time.sleep(5)
win32api.GenerateConsoleCtrlEvent(win32con.CTRL_C_EVENT, pid)
# ^^^^^^^^^^^^^^^^^^^^ instead of myprocess.terminate()