I'm working on something that reads commands from a stream of data. I got stuck trying to work out making a re-usable non-blocking countdown in Python that works off a trigger. So I started a small program w/ just keyboard input and some basic threading to work out the logic. I found a few posts, and this post (How to create a trigger with threading.Timer?) and it was very helpful. But I need some help with another part.
Right now my logic is along the lines of: "Each time the value of command is 1 call start"
How do I update my logic to be:
"if value of command is 1 call start, do not call start again as long as the value of command remains 1.
So its more of a value change detection than a normal if/else, or I have to track a boolean somewhere. I'm just not sure how to approach it.
#! /usr/bin/env python
import time
import threading
import random
from random import randint
import logging
from sys import argv
logging.basicConfig(level=logging.DEBUG, format='[%(levelname)s] (%(threadName)-10s) %(message)s')
def countdown(pName,command):
print("{0} countdown - command{1} ".format(pName,command))
retry = 0
while True:
print("{0}:{1}".format(pName,retry))
retry += 1
if retry > randint(5,10):
break
time.sleep(1)
print("{0} ended".format(pName))
def start(pName,command):
print("starting countdown for: ",pName)
t = threading.Thread(target=countdown,args=(pName,command))
t.setName(pName)
t.setDaemon(True)
t.start()
if __name__ == "__main__":
while 1:
command = int(input("[1 or 2] >"))
if command == 1:
start("Salad",command)
elif command == 2:
start("Bingo",command)
This is pretty brute right now, but its just a first past to try and puzzle it out.
Thanks!
You want the function isAlive. First, you'll have to move your thread variable to the main function, so that main has the appropriate scope to call thread.isAlive().
if __name__ == "__main__":
tSalad = threading.Thread()
tBingo = threading.Thread()
while 1:
command = int(input("[1 or 2] >"))
if command == 1 and not tSalad.isAlive():
tSalad = threading.Thread(target = countdown, args=("Salad", 1))
start("Salad", tSalad)
elif command == 2 and not tBingo.isAlive():
tBingo = threading.Thread(target = countdown, args=("Bingo", 2))
start("Bingo", tBingo)
Then you modify your 'start' function to take a thread argument:
def start(pName, t):
print("starting countdown for: ",pName)
t.setName(pName)
t.setDaemon(True)
t.start()
That should do the trick for you.
Related
In this script I was looking to launch a given program and monitor it as long as the program exists. Thus, I reached the point where I got to use the threading's module Timer method for controlling a loop that writes to a file and prints out to the console a specific stat of the launched process (for this case, mspaint).
The problem arises when I'm hitting CTRL + C in the console or when I close mspaint, with the script capturing any of the 2 events only after the time defined for the interval has completely ran out. These events make the script stop.
For example, if a 20 seconds time is set for the interval, once the script has started, if at second 5 I either hit CTRL + C or close mspaint, the script will stop only after the remaining 15 seconds will have passed.
I would like for the script to stop right away when I either hit CTRL + C or close mspaint (or any other process launched through this script).
The script can be used with the following command, according to the example:
python.exe mon_tool.py -p "C:\Windows\System32\mspaint.exe" -i 20
I'd really appreciate if you could come up with a working example.
I had used python 3.10.4 and psutil 5.9.0 .
This is the code:
# mon_tool.py
import psutil, sys, os, argparse
from subprocess import Popen
from threading import Timer
debug = False
def parse_args(args):
parser = argparse.ArgumentParser()
parser.add_argument("-p", "--path", type=str, required=True)
parser.add_argument("-i", "--interval", type=float, required=True)
return parser.parse_args(args)
def exceptionHandler(exception_type, exception, traceback, debug_hook=sys.excepthook):
'''Print user friendly error messages normally, full traceback if DEBUG on.
Adapted from http://stackoverflow.com/questions/27674602/hide-traceback-unless-a-debug-flag-is-set
'''
if debug:
print('\n*** Error:')
debug_hook(exception_type, exception, traceback)
else:
print("%s: %s" % (exception_type.__name__, exception))
sys.excepthook = exceptionHandler
def validate(data):
try:
if data.interval < 0:
raise ValueError
except ValueError:
raise ValueError(f"Time has a negative value: {data.interval}. Please use a positive value")
def main():
args = parse_args(sys.argv[1:])
validate(args)
# creates the "Process monitor data" folder in the "Documents" folder
# of the current Windows profile
default_path: str = f"{os.path.expanduser('~')}\\Documents\Process monitor data"
if not os.path.exists(default_path):
os.makedirs(default_path)
abs_path: str = f'{default_path}\data_test.txt'
print("data_test.txt can be found in: " + default_path)
# launches the provided process for the path argument, and
# it checks if the process was indeed launched
p: Popen[bytes] = Popen(args.path)
PID = p.pid
isProcess: bool = True
while isProcess:
for proc in psutil.process_iter():
if(proc.pid == PID):
isProcess = False
process_stats = psutil.Process(PID)
# creates the data_test.txt and it erases its content
with open(abs_path, 'w', newline='', encoding='utf-8') as testfile:
testfile.write("")
# loop for writing the handles count to data_test.txt, and
# for printing out the handles count to the console
def process_monitor_loop():
with open(abs_path, 'a', newline='', encoding='utf-8') as testfile:
testfile.write(f"{process_stats.num_handles()}\n")
print(process_stats.num_handles())
Timer(args.interval, process_monitor_loop).start()
process_monitor_loop()
if __name__ == '__main__':
main()
Thank you!
I think you could use python-worker (link) for the alternatives
import time
from datetime import datetime
from worker import worker, enableKeyboardInterrupt
# make sure to execute this before running the worker to enable keyboard interrupt
enableKeyboardInterrupt()
# your codes
...
# block lines with periodic check
def block_next_lines(duration):
t0 = time.time()
while time.time() - t0 <= duration:
time.sleep(0.05) # to reduce resource consumption
def main():
# your codes
...
#worker(keyboard_interrupt=True)
def process_monitor_loop():
while True:
print("hii", datetime.now().isoformat())
block_next_lines(3)
return process_monitor_loop()
if __name__ == '__main__':
main_worker = main()
main_worker.wait()
here your process_monitor_loop will be able to stop even if it's not exactly 20 sec of interval
You can try registering a signal handler for SIGINT, that way whenever the user presses Ctrl+C you can have a custom handler to clean all of your dependencies, like the interval, and exit gracefully.
See this for a simple implementation.
This is the solution for the second part of the problem, which checks if the launched process exists. If it doesn't exist, it stops the script.
This solution comes on top of the solution, for the first part of the problem, provided above by #danangjoyoo, which deals with stopping the script when CTRL + C is used.
Thank you very much once again, #danangjoyoo! :)
This is the code for the second part of the problem:
import time, psutil, sys, os
from datetime import datetime
from worker import worker, enableKeyboardInterrupt, abort_all_thread, ThreadWorkerManager
from threading import Timer
# make sure to execute this before running the worker to enable keyboard interrupt
enableKeyboardInterrupt()
# block lines with periodic check
def block_next_lines(duration):
t0 = time.time()
while time.time() - t0 <= duration:
time.sleep(0.05) # to reduce resource consumption
def main():
# launches mspaint, gets its PID and checks if it was indeed launched
path = f"C:\Windows\System32\mspaint.exe"
p = psutil.Popen(path)
PID = p.pid
isProcess: bool = True
while isProcess:
for proc in psutil.process_iter():
if(proc.pid == PID):
isProcess = False
interval = 5
global counter
counter = 0
#allows for sub_process to run only once
global run_sub_process_once
run_sub_process_once = 1
#worker(keyboard_interrupt=True)
def process_monitor_loop():
while True:
print("hii", datetime.now().isoformat())
def sub_proccess():
'''
Checks every second if the launched process still exists.
If the process doesn't exist anymore, the script will be stopped.
'''
print("Process online:", psutil.pid_exists(PID))
t = Timer(1, sub_proccess)
t.start()
global counter
counter += 1
print(counter)
# Checks if the worker thread is alive.
# If it is not alive, it will kill the thread spawned by sub_process
# hence, stopping the script.
for _, key in enumerate(ThreadWorkerManager.allWorkers):
w = ThreadWorkerManager.allWorkers[key]
if not w.is_alive:
t.cancel()
if not psutil.pid_exists(PID):
abort_all_thread()
t.cancel()
global run_sub_process_once
if run_sub_process_once:
run_sub_process_once = 0
sub_proccess()
block_next_lines(interval)
return process_monitor_loop()
if __name__ == '__main__':
main_worker = main()
main_worker.wait()
Also, I have to note that #danangjoyoo's solution comes as an alternative to signal.pause() for Windows. This only deals with CTRL + C problem part. signal.pause() works only for Unix systems. This is how it was supposed for its usage, for my case, in case it were a Unix system:
import signal, sys
from threading import Timer
def main():
def signal_handler(sig, frame):
print('\nYou pressed Ctrl+C!')
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
print('Press Ctrl+C')
def process_monitor_loop():
try:
print("hi")
except KeyboardInterrupt:
signal.pause()
Timer(10, process_monitor_loop).start()
process_monitor_loop()
if __name__ == '__main__':
main()
The code above is based on this.
I've been trying to find a good limited-input-time code for Python scripts and I finally got a code to work:
from threading import Timer
timeout = 5
t = Timer(timeout, print, ["Time's up!"])
t.start()
entry = input('> ')
t.cancel()
but, I need to be able to run a function when the timer ends.
Also - I want the function called inside of the timer code - otherwise if you type your entry before the timer runs out, the function will still be called no matter what.
Could anyone kindly edit this code I have to be able to run a function when the timer ends?
If it is fine that you block the main thread when the user has not provided any answer, the above code that you have shared might work.
Otherwise you could use msvcrt in the following sense:
import msvcrt
import time
class TimeoutExpired(Exception):
pass
def input_with_timeout(prompt, timeout, timer=time.monotonic):
sys.stdout.write(prompt)
sys.stdout.flush()
endtime = timer() + timeout
result = []
while timer() < endtime:
if msvcrt.kbhit():
result.append(msvcrt.getwche()) #XXX can it block on multibyte characters?
if result[-1] == '\n': #XXX check what Windows returns here
return ''.join(result[:-1])
time.sleep(0.04) # just to yield to other processes/threads
raise TimeoutExpired
The above code is compliant with Python3 and you will need to test it.
Reading from the Python Documentation https://docs.python.org/3/library/threading.html#timer-objects
I have come up with the following snippet which might work(Try running in your command line prompt)
from threading import Timer
def input_with_timeout(x):
def time_up():
answer= None
print('time up...')
t = Timer(x,time_up) # x is amount of time in seconds
t.start()
try:
answer = input("enter answer : ")
except Exception:
print('pass\n')
answer = None
if answer != True: # it means if variable has something
t.cancel() # time_up will not execute(so, no skip)
input_with_timeout(5) # try this for five seconds
The problem resolves around my multiprocessing segment not working when I have an input question.
I have tried many workaround to the problem but cannot find a solution, except eliminating the input, however I need it to allow others to interact with my tool.
import time
from multiprocessing import Pool
import collections
choice = input("Do you wish to start program? \n")
print("hello")
start_time = time.time()
value = collections.namedtuple('value',['vectx','vecty'])
Values = (value(vectx=0,vecty=5),value(vectx=5,vecty=10),value(vectx=10,vecty=15),value(vectx=15,vecty=20))#,value(vectx=200,vecty=300),value(vectx=300,vecty=400),value(vectx=400,vecty=500),value(vectx=500,vecty=600),value(vectx=600,vecty=700),value(vectx=700,vecty=800),value(vectx=800,vecty=900),value(vectx=900,vecty=1000),value(vectx=1000,vecty=1100),value(vectx=1100,vecty=1200))
print("Start")
def Alter(x):
vectx=x.vectx
vecty=x.vecty
Z=(vectx+vecty)
return(Z)
if choice == "Yes":
print(1)
if __name__ == '__main__':
with Pool(10) as p:
result=p.map(Alter, Values)
new = []
print("end")
print("result Done")
for i in result:
new.append(i)
print( "My program took " +str(time.time() - start_time)+ " to run")
Expected result is program completes.
Your problem is that windows doesn't have fork like Unix based machines. So each process of the Pool running on Windows imports the main file on creation.
So what happens in your program is that each new process asks for input and your programs tangles up with itself. It was a little unclear to me the location of the if __name__ == '__main__':, but the point here is that you need to keep everything that needs to run once there. Put outside of it only important stuff shared between all processes. For example, a working code on windows could be:
import time
from multiprocessing import Pool
import collections
def Alter(x):
vectx = x.vectx
vecty = x.vecty
Z = (vectx + vecty)
return Z
value = collections.namedtuple('value', ['vectx', 'vecty'])
if __name__ == '__main__':
choice = input("Do you wish to start program? \n")
Values = (value(vectx=0,vecty=5),value(vectx=5,vecty=10),value(vectx=10,vecty=15))
if choice == "Yes":
print("Start")
start_time = time.time()
with Pool(10) as p:
result = p.map(Alter, Values)
print("My program took " + str(time.time() - start_time) + " to run")
Gives:
Do you wish to start program?
Yes
Start
My program took 1.9328622817993164 to run
From the docs, under programming guidelines, section Safe importing of main module:
Make sure that the main module can be safely imported by a new Python
interpreter without causing unintended side effects (such a starting a
new process).
I'm trying to write code that create sub-process using another module(demo_2.py),
and exit program if i get wanted value on sub-processes.
But result looks like this.
It seems that demo_1 makes two sub-process that run demo_1 and load demo_2.
I want to make sub-process only runs demo_2.
What did i missed?
demo_1.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from multiprocessing import Process,Queue
import sys
import demo_2 as A
def multi_process():
print ("Function multi_process called!")
process_status_A = Queue()
process_status_B = Queue()
A_Process = Process(target = A.process_A, args = (process_status_A,))
B_Process = Process(target = A.process_A, args = (process_status_B,))
A_Process.start()
B_Process.start()
while True:
process_status_output_A = process_status_A.get()
process_status_output_B = process_status_B.get()
if process_status_output_A == 'exit' and process_status_output_B == 'exit':
print ("Success!")
break
process_status_A.close()
process_status_B.close()
A_Process.join()
B_Process.join()
sys.exit()
print ("demo_1 started")
if __name__ == "__main__":
multi_process()
demo_2.py
class process_A(object):
def __init__(self, process_status):
print ("demo_2 called!")
process_status.put('exit')
def call_exit(self):
pass
if process_status_A == 'exit' and process_status_B == 'exit':
should be
if process_status_A_output == 'exit' and process_status_B_output == 'exit':
Conclusion: The naming of variables is important.
Avoid long variable names which are almost the same (such as process_status_A and process_status_A_output).
Placing the distinguishing part of the variable name first helps clarify the meaning of the variable.
So instead of
process_status_A_output
process_status_B_output
perhaps use
output_A
output_B
Because Windows lacks os.fork,
on Windows every time a new subprocess is spawned, a new Python interpreter is started and the calling module is imported.
Therefore, code that you do not wish to be run in the spawned subprocess must be "protected" inside the if-statement (see in particular the section entitled "Safe importing of main module"):
Thus use
if __name__ == "__main__":
print ("demo_1 started")
multi_process()
to avoid printing the extra "demo_1 started" messages.
I'm struggling with a issue for some time now.
I'm building a little script which uses a main loop. This is a process that needs some attention from the users. The user responds on the steps and than some magic happens with use of some functions
Beside this I want to spawn another process which monitors the computer system for some specific events like pressing specif keys. If these events occur then it will launch the same functions as when the user gives in the right values.
So I need to make two processes:
-The main loop (which allows user interaction)
-The background "event scanner", which searches for specific events and then reacts on it.
I try this by launching a main loop and a daemon multiprocessing process. The problem is that when I launch the background process it starts, but after that I does not launch the main loop.
I simplified everything a little to make it more clear:
import multiprocessing, sys, time
def main_loop():
while 1:
input = input('What kind of food do you like?')
print(input)
def test():
while 1:
time.sleep(1)
print('this should run in the background')
if __name__ == '__main__':
try:
print('hello!')
mProcess = multiprocessing.Process(target=test())
mProcess.daemon = True
mProcess.start()
#after starting main loop does not start while it prints out the test loop fine.
main_loop()
except:
sys.exit(0)
You should do
mProcess = multiprocessing.Process(target=test)
instead of
mProcess = multiprocessing.Process(target=test())
Your code actually calls test in the parent process, and that call never returns.
You can use the locking synchronization to have a better control over your program's flow. Curiously, the input function raise an EOF error, but I'm sure you can find a workaround.
import multiprocessing, sys, time
def main_loop(l):
time.sleep(4)
l.acquire()
# raise an EOFError, I don't know why .
#_input = input('What kind of food do you like?')
print(" raw input at 4 sec ")
l.release()
return
def test(l):
i=0
while i<8:
time.sleep(1)
l.acquire()
print('this should run in the background : ', i+1, 'sec')
l.release()
i+=1
return
if __name__ == '__main__':
lock = multiprocessing.Lock()
#try:
print('hello!')
mProcess = multiprocessing.Process(target=test, args = (lock, ) ).start()
inputProcess = multiprocessing.Process(target=main_loop, args = (lock,)).start()
#except:
#sys.exit(0)