Develop Reference

How could I stop the script without having to wait for the time set for interval to pass?

In this script I was looking to launch a given program and monitor it as long as the program exists. Thus, I reached the point where I got to use the threading's module Timer method for controlling a loop that writes to a file and prints out to the console a specific stat of the launched process (for this case, mspaint).
The problem arises when I'm hitting CTRL + C in the console or when I close mspaint, with the script capturing any of the 2 events only after the time defined for the interval has completely ran out. These events make the script stop.
For example, if a 20 seconds time is set for the interval, once the script has started, if at second 5 I either hit CTRL + C or close mspaint, the script will stop only after the remaining 15 seconds will have passed.
I would like for the script to stop right away when I either hit CTRL + C or close mspaint (or any other process launched through this script).
The script can be used with the following command, according to the example:
python.exe mon_tool.py -p "C:\Windows\System32\mspaint.exe" -i 20
I'd really appreciate if you could come up with a working example.
I had used python 3.10.4 and psutil 5.9.0 .
This is the code:
# mon_tool.py
import psutil, sys, os, argparse
from subprocess import Popen
from threading import Timer
debug = False
def parse_args(args):
parser = argparse.ArgumentParser()
parser.add_argument("-p", "--path", type=str, required=True)
parser.add_argument("-i", "--interval", type=float, required=True)
return parser.parse_args(args)
def exceptionHandler(exception_type, exception, traceback, debug_hook=sys.excepthook):
'''Print user friendly error messages normally, full traceback if DEBUG on.
Adapted from http://stackoverflow.com/questions/27674602/hide-traceback-unless-a-debug-flag-is-set
'''
if debug:
print('\n*** Error:')
debug_hook(exception_type, exception, traceback)
else:
print("%s: %s" % (exception_type.__name__, exception))
sys.excepthook = exceptionHandler
def validate(data):
try:
if data.interval < 0:
raise ValueError
except ValueError:
raise ValueError(f"Time has a negative value: {data.interval}. Please use a positive value")
def main():
args = parse_args(sys.argv[1:])
validate(args)
# creates the "Process monitor data" folder in the "Documents" folder
# of the current Windows profile
default_path: str = f"{os.path.expanduser('~')}\\Documents\Process monitor data"
if not os.path.exists(default_path):
os.makedirs(default_path)
abs_path: str = f'{default_path}\data_test.txt'
print("data_test.txt can be found in: " + default_path)
# launches the provided process for the path argument, and
# it checks if the process was indeed launched
p: Popen[bytes] = Popen(args.path)
PID = p.pid
isProcess: bool = True
while isProcess:
for proc in psutil.process_iter():
if(proc.pid == PID):
isProcess = False
process_stats = psutil.Process(PID)
# creates the data_test.txt and it erases its content
with open(abs_path, 'w', newline='', encoding='utf-8') as testfile:
testfile.write("")
# loop for writing the handles count to data_test.txt, and
# for printing out the handles count to the console
def process_monitor_loop():
with open(abs_path, 'a', newline='', encoding='utf-8') as testfile:
testfile.write(f"{process_stats.num_handles()}\n")
print(process_stats.num_handles())
Timer(args.interval, process_monitor_loop).start()
process_monitor_loop()
if __name__ == '__main__':
main()
Thank you!

I think you could use python-worker (link) for the alternatives
import time
from datetime import datetime
from worker import worker, enableKeyboardInterrupt
# make sure to execute this before running the worker to enable keyboard interrupt
enableKeyboardInterrupt()
# your codes
...
# block lines with periodic check
def block_next_lines(duration):
t0 = time.time()
while time.time() - t0 <= duration:
time.sleep(0.05) # to reduce resource consumption
def main():
# your codes
...
#worker(keyboard_interrupt=True)
def process_monitor_loop():
while True:
print("hii", datetime.now().isoformat())
block_next_lines(3)
return process_monitor_loop()
if __name__ == '__main__':
main_worker = main()
main_worker.wait()
here your process_monitor_loop will be able to stop even if it's not exactly 20 sec of interval

You can try registering a signal handler for SIGINT, that way whenever the user presses Ctrl+C you can have a custom handler to clean all of your dependencies, like the interval, and exit gracefully.
See this for a simple implementation.

This is the solution for the second part of the problem, which checks if the launched process exists. If it doesn't exist, it stops the script.
This solution comes on top of the solution, for the first part of the problem, provided above by #danangjoyoo, which deals with stopping the script when CTRL + C is used.
Thank you very much once again, #danangjoyoo! :)
This is the code for the second part of the problem:
import time, psutil, sys, os
from datetime import datetime
from worker import worker, enableKeyboardInterrupt, abort_all_thread, ThreadWorkerManager
from threading import Timer
# make sure to execute this before running the worker to enable keyboard interrupt
enableKeyboardInterrupt()
# block lines with periodic check
def block_next_lines(duration):
t0 = time.time()
while time.time() - t0 <= duration:
time.sleep(0.05) # to reduce resource consumption
def main():
# launches mspaint, gets its PID and checks if it was indeed launched
path = f"C:\Windows\System32\mspaint.exe"
p = psutil.Popen(path)
PID = p.pid
isProcess: bool = True
while isProcess:
for proc in psutil.process_iter():
if(proc.pid == PID):
isProcess = False
interval = 5
global counter
counter = 0
#allows for sub_process to run only once
global run_sub_process_once
run_sub_process_once = 1
#worker(keyboard_interrupt=True)
def process_monitor_loop():
while True:
print("hii", datetime.now().isoformat())
def sub_proccess():
'''
Checks every second if the launched process still exists.
If the process doesn't exist anymore, the script will be stopped.
'''
print("Process online:", psutil.pid_exists(PID))
t = Timer(1, sub_proccess)
t.start()
global counter
counter += 1
print(counter)
# Checks if the worker thread is alive.
# If it is not alive, it will kill the thread spawned by sub_process
# hence, stopping the script.
for _, key in enumerate(ThreadWorkerManager.allWorkers):
w = ThreadWorkerManager.allWorkers[key]
if not w.is_alive:
t.cancel()
if not psutil.pid_exists(PID):
abort_all_thread()
t.cancel()
global run_sub_process_once
if run_sub_process_once:
run_sub_process_once = 0
sub_proccess()
block_next_lines(interval)
return process_monitor_loop()
if __name__ == '__main__':
main_worker = main()
main_worker.wait()
Also, I have to note that #danangjoyoo's solution comes as an alternative to signal.pause() for Windows. This only deals with CTRL + C problem part. signal.pause() works only for Unix systems. This is how it was supposed for its usage, for my case, in case it were a Unix system:
import signal, sys
from threading import Timer
def main():
def signal_handler(sig, frame):
print('\nYou pressed Ctrl+C!')
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
print('Press Ctrl+C')
def process_monitor_loop():
try:
print("hi")
except KeyboardInterrupt:
signal.pause()
Timer(10, process_monitor_loop).start()
process_monitor_loop()
if __name__ == '__main__':
main()
The code above is based on this.

Python script doesn't start on boot

I have a python script in my Raspberry Pi that is connected to a rain gauge. When the rain gauge detects rain, the script shows 0.2 and write it to file. This is the code:
#!/usr/bin/env python3
import time
import RPi.GPIO as GPIO
BUTTON_GPIO = 16
if __name__ == '__main__':
GPIO.setmode(GPIO.BCM)
GPIO.setup(BUTTON_GPIO, GPIO.IN, pull_up_down=GPIO.PUD_UP)
pressed = False
while True:
# button is pressed when pin is LOW
if not GPIO.input(BUTTON_GPIO):
if not pressed:
print("0.2")
pressed = True
# button not pressed (or released)
else:
pressed = False
time.sleep(0.1)
My idea is to use a code like that to save the total amount of rain. When the python script show 0.2 > write it to file.
python3 rain.py >> rain.txt
The code creates a file but doesn't write anything until execution is finished by Ctrl + C.
I need to execute it on boot. I have tried to add it to crontab and rc.local but doesn't work.
I tried to execute it with sudo and pi. The permissions are 755.
Thank you!

try this
import time
import RPi.GPIO as GPIO
BUTTON_GPIO = 16
if __name__ == '__main__':
outputfile=open("/var/log/rain.txt","a",0)
GPIO.setmode(GPIO.BCM)
GPIO.setup(BUTTON_GPIO, GPIO.IN, pull_up_down=GPIO.PUD_UP)
pressed = False
while True:
# button is pressed when pin is LOW
if not GPIO.input(BUTTON_GPIO):
if not pressed:
openputfile.write("0.2\n")
pressed = True
# button not pressed (or released)
else:
pressed = False
time.sleep(0.1)
open a file in append mode with non buffered write.
Then when an event occurs, write to that file.
Do not use shell redirect as it will (in this case) buffer all the program output until exit and then write to a file. Of course, the exit never happens as you have a "while True" with no break

Indeed, this construct command >> file takes the whole of stdout and flushes into the file. It's done only when command execution is over. You must write to the file as soon as your intermediate result is ready:
#!/usr/bin/env python3
import sys
import time
import RPi.GPIO as GPIO
BUTTON_GPIO = 16
if __name__ == '__main__':
GPIO.setmode(GPIO.BCM)
GPIO.setup(BUTTON_GPIO, GPIO.IN, pull_up_down=GPIO.PUD_UP)
pressed = False
# command line arguments
if len(sys.argv) > 1: ## file name was passed
fname = sys.argv[1]
else: ## standard output
fname = None
## this will clear the file with name `fname`
## exchange 'w' for 'a' to keep older data into it
outfile = open(fname, 'w')
outfile.close()
try:
while True:
# button is pressed when pin is LOW
if not GPIO.input(BUTTON_GPIO):
if not pressed:
if fname is None: ## default print
print("0.2")
else:
outfile = open(fname, 'a')
print("0.2", file=outfile)
outfile.close()
pressed = True
# button not pressed (or released)
else:
pressed = False
time.sleep(0.1)
except (Exception, KeyboardInterrupt):
outfile.close()
In this approach you should run python3 rain.py rain.txt and everything will be fine. The try except pattern ensures the file will be properly closed when execution is interrupted by errors or keyboard events.
Notice the file keyword argument in call to print. It selects an open file object to write printed stuff. It defaults to sys.stdout.

Trouble with starting a process out of another process

If I got it right, Python doesn't accept a process to be started out of a process?! For example:
def function1():
while True:
wait_for_condition
#then....
process2.start()
def function2():
does something
process2.join()
process1 = multiprocessing.Process(target=function1,))
process2 = multiprocessing.Process(target=function2,))
process1.start()
In my test python denied to open a process out of a process.
Is there a solution with another way to solve this?
If not - Id have another way to go, but this way would include a modification of the electronics (connect one output to one input and use this to let a process wait for an event and then start. ... but I think this is not a clean way. Its more kind of an workaround. And I'd have a little risk to cause a shortcut if Input and Output is not set correctly).
Edit:
The Task:
Having three processes parallel. These wait for an input at one attached sensor each.
If one of these processes get an input change they should reset a counter (LED_counter) and start another process (LED_process) in not already started. After that the process waits for an input change again.
Beside that...
The LED_process starts to active one output and counting down the LED_counter. If the LED_counter reaches zero, the process terminates. If the code starts again it must be able to restart from the top of the code.
Edit 2:
Latest try with threading (don't be confused by some german words). If I try this code -> the different threads mixes in some strange way together. But for now I can't find a mistake. Same code with multiprocessing works fine:
import RPi.GPIO as GPIO
import time
import threading
import sys
LED_time = 10 #LEDs active time
#Sensor Inputs
SGT = 25
SGA = 23
SHT = 12
GPIO.setmode(GPIO.BCM)
GPIO.setup(SGT, GPIO.IN, pull_up_down=GPIO.PUD_UP)
GPIO.setup(SGA, GPIO.IN, pull_up_down=GPIO.PUD_UP)
GPIO.setup(SHT, GPIO.IN, pull_up_down=GPIO.PUD_UP)
def Sens_check(Sensor,Name):
print("Thread_{} aktiv".format(Name))
while True:
GPIO.wait_for_edge(Sensor, GPIO.FALLING)
#LcGT.value = LED_time
print("{} Offen".format(Name))
time.sleep(0.1)
GPIO.wait_for_edge(SGT, GPIO.RISING)
print("{} Geschlossen".format(Name))
time.sleep(0.1)
SensGT_Thread = threading.Thread(
target=Sens_check,
args=(SGT,"Gartentor",))
SensGA_Thread = threading.Thread(
target=Sens_check,
args=(SGA,"Garage",))
SensHT_Thread = threading.Thread(
target=Sens_check,
args=(SHT,"Haustuere",))
try:
SensGT_Thread.start()
time.sleep(0.1)
SensGA_Thread.start()
time.sleep(0.1)
SensHT_Thread.start()
SensGT_Thread.join()
SensGA_Thread.join()
SensHT_Thread.join()
except:
print("FAILURE")
finally:
sys.exit(1)

Processes can only be started within the process they were created in. In the code provided, process2 was created in the main process, and yet tried to be started within another one (process1). Also, processes cannot be restarted, so they should be created each time .start is used.
Here's an example of starting processes within a process:
import multiprocessing
import time
def function1():
print("Starting more processes")
sub_procs = [multiprocessing.Process(target=function2) for _ in range(5)]
for proc in sub_procs:
proc.start()
for proc in sub_procs:
proc.join()
print("Done with more processes")
def function2():
print("Doing work")
time.sleep(1) # work
print("Done with work")
print("Starting one subprocess")
process1 = multiprocessing.Process(target=function1)
process1.start()
print("Moving on without joining")
"""Output of this:
Starting one subprocess
Moving on without joining
Starting more processes
Doing work
Doing work
Doing work
Doing work
Doing work
Done with work
Done with work
Done with work
Done with work
Done with work
Done with more processes
"""

Raspi python script keeps crashing

I own a 3d printer which is connectet to a raspberry. To controll (like shut on and off via relais) the printer remotely, I made a small python script.
One possibility to controll it is to use the Telegram Bot (telepot) which seems to work properly.
The other method to shut the printer on and off is a hardware switch. For secutity reasons, I want the printer to shut off only if the switch is pressed for 3 seconds.
The problem is, the script crashes sometimes (most at after a long uptime).
I assume that it's my while loop at the end, but I don't really understand why it only crashes after a variable amount of time and how to improve that.
That's the code so far:
#!/usr/bin/python
import time
import subprocess
import os
import datetime
import telepot
import urllib
import RPi.GPIO as GPIO
chat_id_auth = XXXXXXXXXXX
# Warnungen ausschalten
GPIO.setwarnings(False)
# Pin Nummern verwenden
GPIO.setmode(GPIO.BOARD)
# Pin 11 als Input
GPIO.setup(29, GPIO.IN)
GPIO.setup(11, GPIO.OUT)
GPIO.output(11, 0) #on
def handle(msg):
chat_id = msg['chat']['id']
command = msg['text']
print 'Got command: %s' % command
if chat_id == chat_id_auth:
if command == '/status':
bot.sendMessage(chat_id, 'online')
elif command == '/picture':
bashCommand = 'wget --output-document snapshot.jpg http://127.0.0.1:8080/?action=snapshot'
subprocess.check_output(bashCommand, shell=True)
print ('downloaded photo')
bot.sendPhoto(chat_id, open('snapshot.jpg','rb'), caption='Printer Status')
print ('sent photo')
bashCommand = 'rm snapshot.jpg'
subprocess.check_output(bashCommand, shell=True)
print ('removed photo')
elif command == '/ip':
bashCommand = 'ifconfig eth0 | grep \'inet addr:\' | cut -d: -f2 | awk \'{ print $1}\''
output = subprocess.check_output(bashCommand, shell=True)
bot.sendMessage(chat_id, output)
print(output)
elif command == '/on':
#bashCommand = 'echo \'0\' > /sys/class/gpio/gpio17/value'
#output = subprocess.check_output(bashCommand, shell=True)
GPIO.output(11, 0) #on
bot.sendMessage(chat_id, 'Drucker wird eingeschaltet..')
#print(output)
elif command == '/off':
#bashCommand = 'echo \'1\' > /sys/class/gpio/gpio17/value'
#output = subprocess.check_output(bashCommand, shell=True)
GPIO.output(11, 1) #off
bot.sendMessage(chat_id, 'Drucker wird ausgeschaltet..')
#print(output)
else:
bot.sendMessage(chat_id, 'You are not authorized.')
bot = telepot.Bot('XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX')
bot.message_loop(handle)
print 'I am listening ...'
global n
n=0
while True:
if GPIO.input(29):
if GPIO.input(11) == 0:
if n >= 300: #Taster 3s halten zum Ausschalten
GPIO.output(11, 1) #off
n=0
time.sleep(2)
elif GPIO.input(11) == 1:
GPIO.output(11, 0) #on
n=0
time.sleep(2)
time.sleep(0.01)
n+=1
else:
n=0
print n
GPIO.cleanup()

I cannot see a clear reason for a crash but the while loop at the end seems to be very busy waiting when nothing is done (no button pressed). From what I understand it then prints 0 and loops without any delay, so it will print zeros as fast as it can and keeping the Raspi busy as much as possible. Maybe that load is the indirect reason for any crash you experience (but that is just wild guessing).
Anyway, I propose to improve on that massively busy waiting. Even if it does not address the crashes, it would improve your code. To achieve this, simply add a time.sleep(0.1) in the else: branch to loop at most ten times per second (and not as fast as the CPU can).
In any case you should debug the situation. Find out why there is a crash.
Log your output into some file. (./script.py >& script.log)
try/except your exceptions and print debugging stuff in case you catch some.
Whoever starts it should monitor its exit code and log it.
Unix signals received result in an exit code of 128 + the signal number.
Type kill -l on the command line to see a list of all signals and their numbers.

How do I use a countdown thread properly, how do I stop it prematurely?

Threading is not working the way I expect it.
I have a working solution where I monitor when a fridge gets opened and closed with a Raspberry Pi and a reed switch (playback of a sound is unpaused and paused). I now wanted to add a timer to do something when the door is kept open too long. I figured starting a thread which would sleep for x seconds before the alerting action would be a good idea. I would kill the thread with a signal when the switch is closed again.
My approach is failing. The CountDown run thread is started but the terminate signal command is executed but has no effect. Additionally, the commands following c.terminate() are not executed. I looked at examples for threading but they seem to be for more complex situations. What am I missing?
The code:
#!/usr/bin/env python2.7
import threading, subprocess, sys, time, syslog
import RPi.GPIO as GPIO
sound = "/home/pi/sounds/fridge_link.mp3" # sound to play while switch is open
cmd = ['mplayer', '-nolirc', '-noconsolecontrols', '-slave', '-quiet', sound] # command to play sound
lim = 10 # seconds until warning
# thread for countdown (should be interruptable)
# based on http://chimera.labs.oreilly.com/books/1230000000393/ch12.html#_solution_197
class CountdownTask:
def __init__(self):
self._running = True
def terminate(self):
self._running = False
print("thread killed")
def run(self, n):
print("start timer")
time.sleep(n)
## action when timer isup
print("timer ended")
c = CountdownTask()
t = threading.Thread(target=c.run, args=(lim,))
t.daemon = True
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
p.stdin.write('\npausing_keep pause\n')
REED = 27 # data pin of reed sensor (in)
# GPIO setup
GPIO.setwarnings(False)
GPIO.setmode(GPIO.BCM)
GPIO.setup(REED,GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
def edge(channel):
if GPIO.input(REED):
print("detect close")
c.terminate()
p.stdin.write('\npause\n')
pass
else:
print("detect open")
t.start()
p.stdin.write('\npausing_toggle pause\n')
def main():
GPIO.add_event_detect(REED, GPIO.BOTH,callback=edge,bouncetime=1000)
while True:
time.sleep(0.2)
pass
#------------------------------------------------------------
if __name__ == "__main__": main()
New version:
#!/usr/bin/env python2.7
import threading, subprocess, sys, time, syslog
import RPi.GPIO as GPIO
sound = "/home/pi/sounds/fridge_link.mp3" # sound to play while switch is open
cmd = ['mplayer', '-nolirc', '-noconsolecontrols', '-slave', '-quiet', sound] # command to play sound
lim = 10 # seconds until warning
# thread for countdown (should be interruptable)
class CountdownTask:
global dooropen
def __init__(self):
self._running = True
def terminate(self):
self._running = False
print("thread killed")
def run(self, n):
while self._running and dooropen == False:
time.sleep(0.2)
pass
while self._running and dooropen:
print("start timer")
time.sleep(n)
## action when timer isup
print("timer ended")
c = CountdownTask()
t = threading.Thread(target=c.run, args=(lim,))
t.daemon = True
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
p.stdin.write('\npausing_keep pause\n')
REED = 27 # data pin of reed sensor (in)
# GPIO setup
GPIO.setwarnings(False)
GPIO.setmode(GPIO.BCM)
GPIO.setup(REED,GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
dooropen = False # assuming door's closed when starting
def edge(channel):
global dooropen
if GPIO.input(REED): # * no longer reached
if dooropen == False: # catch fridge compressor spike
print("false close alert")
return
p.stdin.write('\npause\n')
dooropen = False
pass
else:
print("detect open")
if dooropen == True:
print("false open alert")
return
p.stdin.write('\npausing_toggle pause\n')
dooropen = True
def main():
GPIO.add_event_detect(REED, GPIO.BOTH,callback=edge,bouncetime=1000)
t.start()
while True:
time.sleep(0.2)
pass
#------------------------------------------------------------
if __name__ == "__main__": main()
Adjusted section, working now:
def run(self, n):
while self._running and dooropen == False:
time.sleep(0.2)
pass
while self._running and dooropen:
time.sleep(n)
if dooropen:
## action when timer isup

Your programmed thread termination mechanism via self._running is not working, because you are not polling/checking the state of self._running in the run() method of the tread (that is actually done in the example you were referring to).
Regular polling adds complexity that is not necessary here. You should structure your logic in a different way, which is simple and reliable. Example code:
import threading
import time
dooropen = True
def warnafter(timeout):
time.sleep(timeout)
if dooropen:
print("Warning!")
t = threading.Thread(target=warnafter, args=(2,))
t.start()
time.sleep(1)
dooropen = False
t.join()
Change time.sleep(1) to time.sleep(3) and a warning is printed. Why does this work, and how does this translate to your use case?
First of all, let's give things names. You have your main thread and the "warn thread". These are the cornerstones of the architecture in my example code:
Have a shared state between the two threads indicating whether the door is open or not, translating into the fact whether a warning should be issued or not. Let's call this state dooropen, and it can be True or False. It is a variable accessible in both, the scope of your main thread as well as in the scope your warn thread has access to. That is, it lives in shared memory.
This is your convention: dooropen is only written from the main thread. The warn thread only reads it.
Spawn your warn thread whenever you think it is the right time. Make it sleep (the exact sleep time can be unreliable, especially on embedded systems).
The crucial part: right before raising an alarm in the warn thread, make it check the dooropen state. If not dooropen, simply do not raise the alarm!
Do you see the two different paradigms?
Your paradigm is to plant an armed bomb, programmed to explode after a given amount of time. This bomb does not talk back to you anymore. Your hope is that you are able to defuse/destroy the bomb before it can explode, if you do not need it to explode anymore.
The paradigm I am proposing ships a bomb that actually is not armed until it needs to be. At the point in time when your bomb would simply explode, this one asks if it really should do so, and only then arms itself and explodes.
Given the latter paradigm, if the warn thread is told to not perform its action, it silently quits, on its own. The concept of "terminating the thread from the outside" is not needed!
In practice, you would need a little more advanced concept, where a warn thread has its own active switch. That is, your main thread can deactivate single warn threads in a controlled fashion. See this example:
import threading
import time
class WarnThread(threading.Thread):
def __init__(self, timeout, name):
threading.Thread.__init__(self)
self._timeout = timeout
self.active = True
self.name = name
self.start()
def run(self):
self._warnafter()
def _warnafter(self):
time.sleep(self._timeout)
if self.active:
print("WarnThread %s: Warning after timeout" % self.name)
ws = [WarnThread(2, i) for i in range(5)]
# Simulate spending some time doing other things,
# such as responding to external events.
time.sleep(1)
# Selectively deactivate some of the warn threads.
ws[0].active = False
ws[2].active = False
for w in ws:
w.join()
Output:
WarnThread 4: Warning after timeout
WarnThread 1: Warning after timeout
WarnThread 3: Warning after timeout