Develop Reference

Subprocess Popen problem with terminate shell script

I have a little problem with understand what is going on with my code.
I have a lot of lines in my code so i decided to simulate my problem with shorter version.
Im using raspberry pi 4, flirone on rapsbian.
import sys
import os
import time
import subprocess
global shellscript
#global pid
def subprocess_settings():
shellscript = subprocess.Popen(["/home/pi/Desktop/init_flir.sh"], close_fds=True)
#shellscript = subprocess.Popen(["/home/pi/Desktop/init_flir.sh", "&> /dev/null"], stdin=None, stdout=None, close_fds=True)
#pid = shellscript.pid
try:
x = 1
while True:
if x == 0:
sys.exit()
elif x == 1:
print("Yas, x=1")
time.sleep(2)
subprocess_settings()
time.sleep(5)
print("Driver test is in background mode")
time.sleep(2)
print("Close process")
subprocess.Popen.kill(shellscript)
x=0
except KeyboardInterrupt:
subprocess.Popen.kill(shellscript)
and my .sh file:
#!/bin/bash
cd /home/pi/github/flirone-v4l2
sudo modprobe v4l2loopback exclusive_caps=0,0 video_nr=1,2,3
sudo ./flirone ~/github/flirone-v4l2/palettes/Iron2.raw
I want to run my .sh file. It has to work in background. I need to terminate this with 2 ways, first one from keyboard and second inside my elif.
At the end i need to hide terminal output from .sh file.
What im doing wrong??
Is this is a problem with global variables?
How to fix this to work properly?
Thanks for any help.

This solution works properly, as i wanted. In flirone driver i runned ./flirone and opened 1 process from subprocess.Popen and 2 additional processes in flirone code. I checked this running command sudo ./flirone ~/github/flirone-v4l2/palettes/Iron2.raw and ps aux | grep flirone . I used pidof flirone command to check main ID. When im killing this process everything works as should be and other processes ends too.
import sys
import os
import signal
import time
import subprocess
def subprocess_kill():
pidof = subprocess.check_output("pidof flirone", shell=True)
time.sleep(1)
subprocess.check_output("sudo kill -9 %s " % (pidof), shell=True)
try:
x = 1
while True:
if x == 0:
sys.exit()
elif x == 1:
print("x=1")
time.sleep(2)
os.chdir("/home/pi/github/flirone-v4l2")
shellscript = subprocess.Popen("sudo modprobe v4l2loopback exclusive_caps=0,0 video_nr=1,2,3", shell=True)
shellscript2 = subprocess.Popen("sudo ./flirone ~/github/flirone-v4l2/palettes/Iron2.raw", shell=True)
time.sleep(5)
print("Driver test is in background mode")
time.sleep(2)
print("Close process")
subprocess_kill()
x=0
except KeyboardInterrupt:
print("hey")
When i deleted killing method from except KeyboardInterrupt processes ends too.
If you want to hide output of subprocess u can use:
FNULL = open(os.devnull, "w")
shellscript2 = subprocess.Popen("sudo ./flirone ~/github/flirone-v4l2/palettes/Iron2.raw", shell=True, stdout=FNULL)
More: How to hide output of subprocess in Python 2.7
Subprocess python 2.7 lib

Unable to prevent program to print a Error

I took a piece of code from here:
https://stackoverflow.com/a/10457565/13882705
which is
# I have used os comands for a while
# this program will try to close a firefox window every ten secounds
import os
import time
# creating a forever loop
while 1 :
os.system("TASKKILL /F /IM firefox.exe")
time.sleep(10)
It will terminate a process if it is running using OS module
But if the program did not find the app we mentioned then it prints
ERROR: The process "firefox.exe" not found.
Is there a way to make the program just print application not found once and wait until the program is rerunned?
It is fine even if it just prints "Application Not found"

Use subprocess.run instead of os.system so you have more control:
import subprocess
import time
while True:
proc = subprocess.run(["TASKKILL", "/F", "/IM", "firefox.exe"], stderr=subprocess.PIPE)
if proc.returncode != 0:
print("Application not found.")
break # since application isn't here we just exit
time.sleep(10)

Python 3 Stopping subprocess by sending Ctrl C

I have some GPU test software i'm trying to automate using python3, The test would normally be run for 3 minutes then cancelled by a user using ctrl+c generating the following output
After exiting with ctrl+c the test can then be run again with no issue
When trying to automate this with subprocess popen and sending SIGINT or SIGTERM i'm not getting the same as if keyboard entry was used. The script exits abruptly and on subsequent runs cant find the gpus (assume its not unloading the driver properly)
from subprocess import Popen, PIPE
from signal import SIGINT
from time import time
def check_subproc_alive(subproc):
return subproc.poll() is None
def print_subproc(subproc, timer=True):
start_time = time()
while check_subproc_alive(subproc):
line = subproc.stdout.readline().decode('utf-8')
print(line, end="")
if timer and (time() - start_time) > 10:
break
subproc = Popen(['./gpu_test.sh', '-t', '1'], stdin=PIPE, stdout=PIPE, stderr=PIPE, shell=False)
print_subproc(subproc)
subproc.send_signal(SIGINT)
print_subproc(subproc, False)
How can I send ctrl+c to a subprocess as if a user typed it?
**UPDATE
import subprocess
def start(executable_file):
return subprocess.Popen(
executable_file,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
)
def read(process):
return process.stdout.readline().decode("utf-8").strip()
def write(process):
process.stdin.write('\x03'.encode())
process.stdin.flush()
def terminate(process):
process.stdin.close()
process.terminate()
process.wait(timeout=0.2)
process = start("./test.sh")
write(process)
for x in range(100):
print(read(process))
terminate(process)
Tried the above code and can get characters to register with dummy sh script however sending the \x03 command just sends an empty char and doesn't end script

I think you can probably use something like this:
import signal
try:
p=subprocess...
except KeyboardInterrupt:
p.send_signal(signal.SIGINT)

The following solution is the only one I could find that works for windows and is the closest resemblance to sending a Ctrl+C event.
import signal
os.kill(self.p.pid, signal.CTRL_C_EVENT)

Python kill a subprocess(that starts another process) and start it again

I'm trying to make a python script that starts the program livestreamer (that starts the program mplayer) and after 10 seconds it should kill the program, or the subprocess. here is my current code that doesn't work, I think I know why but I don't know how to solve it.
I think the problem is that the subprocess starts livestreamer and then the program livestreamer starts the program mplayer. Python doesn't know about mplayer and can't close it. How would I be able to kill both livestreamer and mplayer after 10 second and then start them again as a loop?
I'm using Ubuntu 14.04 (Linux) and Python 2.7.6
import subprocess
import time
import os
import sys
import signal
url = "http://new.livestream.com/accounts/398160/events/3155348"
home = os.environ['HOME']
if not os.geteuid() == 0:
if not os.path.exists('/%s/.config/livestreamer' % home):
os.makedirs('/%s/.config/livestreamer' % home)
lscfg = open('%s/.config/livestreamer/config' % home, 'w+')
lscfg.write("player=mplayer -geometry 0%:0% -nomouseinput -loop 100 -noborder -fixed-vo")
lscfg.close()
cmd = "livestreamer %s best --player-continuous-http --player-no-close" % url
while True:
proc1 = subprocess.Popen(cmd.split(), shell=False)
time.sleep(10)
proc1.kill()
Solution:
import subprocess
import time
import os
import sys
import signal
url = "http://new.livestream.com/accounts/398160/events/3155348"
home = os.environ['HOME']
if not os.geteuid() == 0:
if not os.path.exists('/%s/.config/livestreamer' % home):
os.makedirs('/%s/.config/livestreamer' % home)
lscfg = open('%s/.config/livestreamer/config' % home, 'w+')
lscfg.write("player=mplayer -geometry 0%:0% -nomouseinput -loop 100 -noborder -fixed-vo")
lscfg.close()
cmd = "livestreamer %s best --player-continuous-http --player-no-close" % url
#restarting the player every 10th minute to catch up on possible delay
while True:
proc1 = subprocess.Popen(cmd.split(), shell=False)
time.sleep(600)
os.system("killall -9 mplayer")
proc1.kill()
As you can see os.system("killall -9 mplayer") was the command to kill the process mplayer.

In your code you kill livestreamer but not mplayer so mplayer will continue running.
By using kill on your subprocess you send a signal SIGKILL and unless the subprocess do handle the signal interruption it will simply close itself fast and without killing his own childs so mplayer will live (and may become a zombie process).
You have no reference to your subprocess child 'mplayer' but if you can get his PID you can kill it with os.kill(...)
os.kill(process_pid, signal.SIGTERM)

Using os.system("killall -9 mplayer") was the easy way to solve this. Mind using this option will kill all process of mplayer though this is not a problem in my case but may be a problem for other cases.
while True:
proc1 = subprocess.Popen(cmd.split(), shell=False)
time.sleep(600)
os.system("killall -9 mplayer")
proc1.kill()

terminate a process and its subprocesses started with subprocess.popen the right way (windows and linux)

I'm struggling with some processes I started with Popen and which start subprocesses. When I start these processes manually in a terminal every process terminates as expected if I send CTRL+C. But running inside a python program using subprocess.Popen any attempt to terminate the process only gets rid of the parent but not of its children.
I tried .terminate() ..kill() as well as ..send_signal() with signal.SIGBREAK, signal.SIGTERM, but in every case I just terminate the parent process.
With this parent process I can reproduce the misbehavior:
#!/usr/bin/python
import time
import sys
import os
import subprocess
import signal
if __name__ == "__main__":
print os.getpid(), "MAIN: start a process.."
p = subprocess.Popen([sys.executable, 'process_to_shutdown.py'])
print os.getpid(), "MAIN: started process", p.pid
time.sleep(2)
print os.getpid(), "MAIN: kill the process"
# these just terminate the parent:
#p.terminate()
#p.kill()
#os.kill(p.pid, signal.SIGINT)
#os.kill(p.pid, signal.SIGTERM)
os.kill(p.pid, signal.SIGABRT)
p.wait()
print os.getpid(), "MAIN: job done - ciao"
The real life child process is manage.py from Django which spawns a few subprocesses and waits for CRTL-C. But the following example seems to work, too:
#!/usr/bin/python
import time
import sys
import os
import subprocess
if __name__ == "__main__":
timeout = int(sys.argv[1]) if len(sys.argv) >= 2 else 0
if timeout == 0:
p = subprocess.Popen([sys.executable, '-u', __file__, '13'])
print os.getpid(), "just waiting..."
p.wait()
else:
for i in range(timeout):
time.sleep(1)
print os.getpid(), i, "alive!"
sys.stdout.flush()
print os.getpid(), "ciao"
So my question in short: how do I kill the process in the first example and get rid of the child processes as well? On windows os.kill(p.pid, signal.CTRL_C_EVENT) seems to work in some cases, but what's the right way to do it? And how does a Terminal do it?

Like Henri Korhonen mentioned in a comment, grouping processes should help. Additionally, if you are on Windows and this is Cygwin Python that starts Windows applications, it appears Cygwin Python can not kill the children. For those cases you would need to run TASKKILL. TASKKILL also takes a group parameter.