Develop Reference

How do I run automatically run a command once my program ends?

I am currently coding on python3 and have a while True loop where it will run indefinitely so I have to force stop it for it to end. For further details, I am creating a system that automatically detects the humidity in a box, and starts a fan, once it reaches a high enough humidity. When I force stop my program, I would like it to run a command, specifically GPIO.output(21, GPIO.LOW), this will cause the fan to turn off. Any ideas? (I'm currently using a raspberry pi 3 to code this all on)

Here's an example of you can detect SIGINT (Ctrl-C):
from time import sleep
from signal import signal, SIGINT, SIG_DFL
INTERRUPTED = False
def handler(signum, _):
global INTERRUPTED
signal(signum, SIG_DFL)
INTERRUPTED = True
signal(SIGINT, handler)
while not INTERRUPTED:
sleep(1)
print('Interrupted')
So here we have a potentially infinite loop but having set a handler for SIGINT we can break out of the loop upon receipt of that signal and run some cleanup code or whatever else may be necessary

Capture an input to break out of your loop instead of manually closing the program, then put GPIO.output(21, GPIO.LOW) after the loop. Or if you are set on manually ending the program perhaps manually start a separate program right after that only runs that single command?

When is a Python program not responding to interrupts?

I have a Python3 daemon process running on Linux. It is a normal single thread process running in the background and doing select.select() in the main loop and then handling I/O. Sometimes (approx. 1 or 2 times in a month) it stops responding. When it happens, I'd like to debug the problem.
I have tried the pyrasite, but was not succesfull, because stdin/stdout of the daemon is redirected to the /dev/null device and pyrasite uses this stdin/stdout, not the console where it was started from.
So I have added a SIGUSR1 signal handler which logs the stack trace. Works fine normally.
Today I got a freeze. ps shows, the daemon is in "S" (interruptible sleep) state. A busy loop is ruled out.
The server does not respond to SIGUSR or SIGINT (used for shutdown).
I'd like to have at least some hint what is going on there.
Under what conditions is a sleeping Python3 Linux process not handling interrupts that it is supposed to handle?
UPDATE:
I could reproduce the issue finally. After adding a lot of debug messages, I have found a race condition that I'm going to fix soon.
When the daemon is not responding, it is sleeping at os.read(p) where p is a read-end of a new pipe (see: os.pipe) where nobody writes to.
However, all my attempts to write a simple demonstration program failed. When I tried to read from an empty pipe, the program blocked as expected, but could be interrupted (killed from other terminal with SIGINT) as usual. The mystery remains unsolved.
UPDATE2:
Finally some code! I have deliberately chosen low level system calls.
import os
import time
import signal
import sys
def sighandler(*unused):
print("got signal", file=sys.stderr)
print("==========")
signal.signal(signal.SIGUSR1, sighandler)
pid = os.getpid()
rfd, wfd = os.pipe()
if os.fork():
os.close(wfd)
print("parent: read() start")
os.read(rfd, 4096)
print("parent: read() stop")
else:
os.close(rfd)
os.kill(pid, signal.SIGUSR1)
print("child: wait start")
time.sleep(3)
print("child: wait end")
If you run this many time, you'll get this:
parent: read() start
got signal
child: wait start
child: wait end
parent: read() stop
which is fine, but sometimes you'll see this:
parent: read() start
child: wait start
child: wait end
got signal
parent: read() stop
What is happening here:
parent starts a read from the pipe
child sends a signal to the parent. The parent must have received this signal, but it seems to be "somehow postponed"
child waits
child exits, pipe is closed automatically
parent's read operation ends with an EOF
the signal is handled now
Now, due to a bug in my program, the signal was received in step 2, but the EOF was not delivered, so the read did not finish and step 6 (signal handling) was never reached.
That is all information I am able to provide.

Read parameters like eta from youtube-dl

Hi I would like to read the output from youtube dl on cmd and put in my wxpython program. This is the function I used.
def execute(self,command,textctrl):
process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
output = ''
# Poll process for new output until finished
for line in iter(process.stdout.readline, ""):
textctrl.AppendText(line)
output += line
process.wait()
exitCode = process.returncode
if (exitCode == 0):
return output
else:
raise Exception(command, exitCode, output)
The wxpython program started updating the textctrl and then froze. It didnt update the eta, size, speed etc

As long as you are blocked in this function and are not letting control return to the event loop, then there can be no events dispatched to handlers. With no events being sent and processed, there can be no repainting of the contents of the widgets, no interaction with with the mouse and keyboard, nothing. Basically the application is frozen because your execute function is not letting its heart beat and the brain is cut off from the rest of the body.
When programming GUIs or other implementations of event driven programming the key is to never do anything in an event handler or callback that would take more than a noticeable (by a human) amount of time before it returns to the event loop. If you have something that will take longer than that time, then you need to redesign it so the long running task is managed a different way.
One way would be to set things up in the event handler (such as starting the process) and then return from the event handler. Part of that setup would be to start a timer that comes back periodically and checks if there is output available. If so then read it (without blocking) and process it, and then return to the event loop again. Continue until the process is done and then stop the timer after the last chunk of data is processed.
Another approach is to use a thread to run the long running task. This is a common approach but you need to be careful to not manipulate any UI objects from the worker thread. So in your example the text you read from the process will need to be sent back to the GUI thread in order to have it appended to the text control. wx.CallAfter is an easy way to do that.
See https://wiki.wxpython.org/LongRunningTasks for more details and some examples.

Pausing Python subprocesses from keyboard input without killing the subprocess

I'm working on a project to produce a shell in Python, and one important feature is the ability to pause and background a running subprocess. However the only methods I've found of pausing the subprocess appear to kill it instantly, so I can't resume it later.
Our group has tried excepting KeyboardInterrupt:
try:
process = subprocess.Popen(processName)
process.communicate()
except KeyboardInterrupt:
print "control character pressed"
and also using signals:
def signal_handler(signal,frame):
print 'control character pressed'
signal.signal(signal.SIGINT, signal_handler)
process.communicate()
Another issue is that both of these only work when Ctrl-C is pressed, nothing else has any effect (I imagine this is why the subprocesses are being killed).

The reason you have the process dying is because you are allowing the Ctrl+C to reach the subprocess. If you were to use the parameter preexec_fn = os.setpgrp, as part of the Popen call, then the the child is set to be in a different process group from the parent.
Ctrl+C sends a SIGINT to the complete process group, but since the child is in a different process group, it doesn't receive the SIGINT and thus doesn't die.
After that, the send_signal() function can be used to send a SIGSTOP to the child process whenever you want to pause it, and a SIGCONT to resume it.

Twisted program and TERM signal

I have a simple example:
from twisted.internet import utils,reactor
def test:
utils.getProcessOutput(executable="/bin/sleep",args=["10000"])
reactor.callWhenRunning(test)
reactor.run()
when I send signal "TERM" to program, "sleep" continues to be carried out, when I press Ctrl-C on keyboard "sleep" stopping. ( Ctrl-C is not equivalent signal TERM ?) Why ? How to kill "sleep" after send signal "TERM" to this program ?

Ctrl-C sends SIGINT to the entire foreground process group. That means it gets send to your Twisted program and to the sleep child process.
If you want to kill the sleep process whenever the Python process is going to exit, then you may want a before shutdown trigger:
def killSleep():
# Do it, somehow
reactor.addSystemEventTrigger('before', 'shutdown', killSleep)
As your example code is written, killSleep is difficult to implement. getProcessOutput doesn't give you something that easily allows the child to be killed (for example, you don't know its pid). If you use reactor.spawnProcess and a custom ProcessProtocol, this problem is solved though - the ProcessProtocol will be connected to a process transport which has a signalProcess method which you can use to send a SIGTERM (or whatever you like) to the child process.
You could also ignore SIGINT and this point and then manually deliver it to the whole process group:
import os, signal
def killGroup():
signal.signal(signal.SIGINT, signal.SIG_IGN)
os.kill(-os.getpgid(os.getpid()), signal.SIGINT)
reactor.addSystemEventTrigger('before', 'shutdown', killGroup)
Ignore SIGINT because the Twisted process is already shutting down and another signal won't do any good (and will probably confuse it or at least lead to spurious errors being reported). Sending a signal to -os.getpgid(os.getpid()) is how to send it to your entire process group.