I'm working on a project to produce a shell in Python, and one important feature is the ability to pause and background a running subprocess. However the only methods I've found of pausing the subprocess appear to kill it instantly, so I can't resume it later.
Our group has tried excepting KeyboardInterrupt:
try:
process = subprocess.Popen(processName)
process.communicate()
except KeyboardInterrupt:
print "control character pressed"
and also using signals:
def signal_handler(signal,frame):
print 'control character pressed'
signal.signal(signal.SIGINT, signal_handler)
process.communicate()
Another issue is that both of these only work when Ctrl-C is pressed, nothing else has any effect (I imagine this is why the subprocesses are being killed).
The reason you have the process dying is because you are allowing the Ctrl+C to reach the subprocess. If you were to use the parameter preexec_fn = os.setpgrp, as part of the Popen call, then the the child is set to be in a different process group from the parent.
Ctrl+C sends a SIGINT to the complete process group, but since the child is in a different process group, it doesn't receive the SIGINT and thus doesn't die.
After that, the send_signal() function can be used to send a SIGSTOP to the child process whenever you want to pause it, and a SIGCONT to resume it.
Related
This is specifically for Windows, I don't have this issue on linux based systems.
So, I have a program that creates subprocesses when running it.
These subprocesses will terminate correctly if the program exits normally, or even with exceptions or ctrl+c event, by using try and KeyboardInterrupt and finally in if __name__ == '__main__':
However, if I kill the program in the middle, I'm talking about killing it in PyCharm, using the STOP button. Those subprocesses will not terminate. I'm not exactly sure what signal this STOP button sends on Windows.
I tried signal handling using signal.signal(signal.SIGTERM, handler). It doesn't work, I have tried SIGTERM, SIGINT, (SIGKILL, CTRL_C_EVENT, CTRL_BREAK_EVENT don't work in signal handler. ). None of them works. I have also read this post: How to handle the signal in python on windows machine
How can I gracefully exit in this scenario? This STOP button in PyCharm scenario.
How can I return a process id of a lengthy process started using Thread in Python before the thread completes its execution?
I'm using Tkinter GUI so I can't start a lengthy process on the main thread so instead I start one on a separate thread.
The thread in turn calls subprocess.popen. This process should run for like 5 -6 hours.
But When I press stopbutton I need this process to stop but I am unable to return the process id of the process created using subprocess.popen.
Is there any solution to this?
If you are using subprocess.Popen simply to spin off another process, there is no reason you need to do so from another thread. A sub-process created this way does not block your main thread. You can continue to do other things while the sub-process is running. You simply keep a reference to the Popen object returned.
The Popen object has all the facilities you need for monitoring / interacting with the sub-process. You can read and write to its standard input and output (via stdin and stdout members, if created with PIPE); you can monitor readability / writability of stdin and stdout (with select module); you can check whether the sub-process is still in existence with poll, reap its exit status with wait; you can stop it with terminate (or kill depending on how emphatic you wish to be).
There are certainly times when it might be advantageous to do this from another thread -- for example, if you need significant interaction with the sub-process and implementing that in the main thread would over-complicate your logic. In that case, it would be best to arrange a mechanism whereby you signal to your other "monitoring" thread that it's time to shutdown and allow the monitoring thread to execute terminate or kill on the sub-process.
I have a Python3 daemon process running on Linux. It is a normal single thread process running in the background and doing select.select() in the main loop and then handling I/O. Sometimes (approx. 1 or 2 times in a month) it stops responding. When it happens, I'd like to debug the problem.
I have tried the pyrasite, but was not succesfull, because stdin/stdout of the daemon is redirected to the /dev/null device and pyrasite uses this stdin/stdout, not the console where it was started from.
So I have added a SIGUSR1 signal handler which logs the stack trace. Works fine normally.
Today I got a freeze. ps shows, the daemon is in "S" (interruptible sleep) state. A busy loop is ruled out.
The server does not respond to SIGUSR or SIGINT (used for shutdown).
I'd like to have at least some hint what is going on there.
Under what conditions is a sleeping Python3 Linux process not handling interrupts that it is supposed to handle?
UPDATE:
I could reproduce the issue finally. After adding a lot of debug messages, I have found a race condition that I'm going to fix soon.
When the daemon is not responding, it is sleeping at os.read(p) where p is a read-end of a new pipe (see: os.pipe) where nobody writes to.
However, all my attempts to write a simple demonstration program failed. When I tried to read from an empty pipe, the program blocked as expected, but could be interrupted (killed from other terminal with SIGINT) as usual. The mystery remains unsolved.
UPDATE2:
Finally some code! I have deliberately chosen low level system calls.
import os
import time
import signal
import sys
def sighandler(*unused):
print("got signal", file=sys.stderr)
print("==========")
signal.signal(signal.SIGUSR1, sighandler)
pid = os.getpid()
rfd, wfd = os.pipe()
if os.fork():
os.close(wfd)
print("parent: read() start")
os.read(rfd, 4096)
print("parent: read() stop")
else:
os.close(rfd)
os.kill(pid, signal.SIGUSR1)
print("child: wait start")
time.sleep(3)
print("child: wait end")
If you run this many time, you'll get this:
parent: read() start
got signal
child: wait start
child: wait end
parent: read() stop
which is fine, but sometimes you'll see this:
parent: read() start
child: wait start
child: wait end
got signal
parent: read() stop
What is happening here:
parent starts a read from the pipe
child sends a signal to the parent. The parent must have received this signal, but it seems to be "somehow postponed"
child waits
child exits, pipe is closed automatically
parent's read operation ends with an EOF
the signal is handled now
Now, due to a bug in my program, the signal was received in step 2, but the EOF was not delivered, so the read did not finish and step 6 (signal handling) was never reached.
That is all information I am able to provide.
I am using python 2.7 version on ubuntu. I am curious regarding how different signals are handled in python program during its execution. Is there any priority based selection.? For eg: If there are two different signals generated at the same time, which one will be served first? In my program given below it waits for the user to press Ctrl-C key, if done so it will display "Process can't be killed with ctrl-c key!". Along with this it keep generating an SIGALRM signal every second and keeps generating "Got an alarm" message in output every second.
#!/usr/bin/env python
import signal
import time
def ctrlc_catcher(signum, frm):
print "Process can't be killed with ctrl-c!"
def alarm_catcher(signum,frame):
print "Got an alarm"
signal.signal(signal.SIGINT, ctrlc_catcher)
signal.signal(signal.SIGALRM, alarm_catcher)
while True:
signal.alarm(1)
time.sleep(1)
pass
Now when I execute the programme it produces following output indefinitely:
Got an alarm
Got an alarm
Got an alarm
Got an alarm
If during the execution I hit Ctrl-C key once then the output is interrupted and as shown below:
Got an alarm
Got an alarm
Got an alarm
Got an alarm
Process can't be killed with ctrl-c
Got an alarm
Everything is working as programmed and as expected.
My question is if I press the ctrl-c key continuously then why the output is as given below:
Process can't be killed with ctrl-c
Process can't be killed with ctrl-c
Process can't be killed with ctrl-c
Why isn't the output regarding the triggering of alarm also shows up in the above output as the alarm is being triggered every second?
Is the alarm signal (signal.ALARM) being ignored because of signal.SIGNIT? Or the continuous pressing of Ctrl-C key is pausing something?
Thanks
The behavior you see is due to the interaction of two factors:
(1) When you call signal.alarm, you clear any previous alarms; after the call, only the most recently requested alarm is scheduled.
(2) A caught signal terminates time.sleep and causes the sleep to be cut short; it does not resume after the signal handler returns.
Now, when you send SIGINT to your process, it usually arrives during the sleep, which it interrupts, and so after your handler ctlc_catcher returns the while loop immediately continues to the next iteration, scheduling a new alarm for one second from that point and clearing any old alarms. In other words, if SIGINT arrives during an iteration of the loop, that iteration will almost never end up sleeping for a full second, and so the next iteration of the loop will execute and clear the already scheduled alarm before it has a chance to be delivered.
It follows from this that if you press cntl-C more frequently than once per second, you won't see "Got an alarm." at all.
If you want to guarantee that an alarm is delivered once per second despite any interrupts, you'll have to do some extra work to decide, on each loop iteration, whether you should schedule an alarm.
Perhaps something like this?
#!/usr/local/cpython-3.3/bin/python
import subprocess
p = subprocess.Popen("./app",
stdin = subprocess.PIPE,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE,
shell = True)
p.stdin.write(bytes("3\n", "ascii"))
p.stdin.write(bytes("4\n", "ascii"))
print(p.stdout.read())
exit_code = p.wait()
print(exit_code)
I have a simple example:
from twisted.internet import utils,reactor
def test:
utils.getProcessOutput(executable="/bin/sleep",args=["10000"])
reactor.callWhenRunning(test)
reactor.run()
when I send signal "TERM" to program, "sleep" continues to be carried out, when I press Ctrl-C on keyboard "sleep" stopping. ( Ctrl-C is not equivalent signal TERM ?) Why ? How to kill "sleep" after send signal "TERM" to this program ?
Ctrl-C sends SIGINT to the entire foreground process group. That means it gets send to your Twisted program and to the sleep child process.
If you want to kill the sleep process whenever the Python process is going to exit, then you may want a before shutdown trigger:
def killSleep():
# Do it, somehow
reactor.addSystemEventTrigger('before', 'shutdown', killSleep)
As your example code is written, killSleep is difficult to implement. getProcessOutput doesn't give you something that easily allows the child to be killed (for example, you don't know its pid). If you use reactor.spawnProcess and a custom ProcessProtocol, this problem is solved though - the ProcessProtocol will be connected to a process transport which has a signalProcess method which you can use to send a SIGTERM (or whatever you like) to the child process.
You could also ignore SIGINT and this point and then manually deliver it to the whole process group:
import os, signal
def killGroup():
signal.signal(signal.SIGINT, signal.SIG_IGN)
os.kill(-os.getpgid(os.getpid()), signal.SIGINT)
reactor.addSystemEventTrigger('before', 'shutdown', killGroup)
Ignore SIGINT because the Twisted process is already shutting down and another signal won't do any good (and will probably confuse it or at least lead to spurious errors being reported). Sending a signal to -os.getpgid(os.getpid()) is how to send it to your entire process group.