Develop Reference

How do I add limited input time to my code?

I've been trying to find a good limited-input-time code for Python scripts and I finally got a code to work:
from threading import Timer
timeout = 5
t = Timer(timeout, print, ["Time's up!"])
t.start()
entry = input('> ')
t.cancel()
but, I need to be able to run a function when the timer ends.
Also - I want the function called inside of the timer code - otherwise if you type your entry before the timer runs out, the function will still be called no matter what.
Could anyone kindly edit this code I have to be able to run a function when the timer ends?

If it is fine that you block the main thread when the user has not provided any answer, the above code that you have shared might work.
Otherwise you could use msvcrt in the following sense:
import msvcrt
import time
class TimeoutExpired(Exception):
pass
def input_with_timeout(prompt, timeout, timer=time.monotonic):
sys.stdout.write(prompt)
sys.stdout.flush()
endtime = timer() + timeout
result = []
while timer() < endtime:
if msvcrt.kbhit():
result.append(msvcrt.getwche()) #XXX can it block on multibyte characters?
if result[-1] == '\n': #XXX check what Windows returns here
return ''.join(result[:-1])
time.sleep(0.04) # just to yield to other processes/threads
raise TimeoutExpired
The above code is compliant with Python3 and you will need to test it.
Reading from the Python Documentation https://docs.python.org/3/library/threading.html#timer-objects
I have come up with the following snippet which might work(Try running in your command line prompt)
from threading import Timer
def input_with_timeout(x):
def time_up():
answer= None
print('time up...')
t = Timer(x,time_up) # x is amount of time in seconds
t.start()
try:
answer = input("enter answer : ")
except Exception:
print('pass\n')
answer = None
if answer != True: # it means if variable has something
t.cancel() # time_up will not execute(so, no skip)
input_with_timeout(5) # try this for five seconds

More pythonic way to exit a python thread - daemon vs stop_event

The following code:
import threading
import time
from functools import partial
from itertools import count
def daemon_loop(sleep_interval, stop_event):
for j in count():
print(j)
if stop_event.is_set():
break
time.sleep(sleep_interval)
print('Slept %s' % sleep_interval)
print('Prod terminating')
if __name__ == '__main__':
stop_event = threading.Event() #https://stackoverflow.com/a/41139707/281545
target = partial(daemon_loop, sleep_interval=2, stop_event=stop_event)
prod_thread = threading.Thread(target=target,
# daemon=True
)
try:
prod_thread.start()
while True:
time.sleep(10)
except KeyboardInterrupt:
print('Terminating...')
stop_event.set()
prints on a keyboard interrupt:
C:\Users\MrD\.PyCharm2018.2\config\scratches>c:\_\Python363-64\python.exe thread_daemon.py
0
Slept 2
1
Terminating...
Slept 2
2
Prod terminating
Uncommenting the # daemon=True line results in the prod_thread being ended immediately:
C:\Users\MrD\.PyCharm2018.2\config\scratches>c:\_\Python363-64\python.exe thread_daemon.py
0
Slept 2
1
Terminating...
My question is what is the preferred/more pythonic way to deal with thread termination - should I drop the Event machinery and just mark the thread as daemon or is there some edge case I miss?
See:
Daemon Threads Explanation
How to stop daemon thread?

I haven't done enough Python to give you a "Pythonic" answer, but I can answer in more general programming terms.
Firstly, I'm not a fan of terminating threads. There are cases where it is safe and OK, such as your example here - but terminating in the middle of print writing its output would feel a little dirty.
Secondly, if you want to continue using sleep (which I'm also not a fan of) you could repeat your if stop_event.is_set(): and break after the sleep. (Don't move the code, copy it.) The main problem with sleep in this case is that it will wait the full sleep_interval even if the event is set during that time.
Thirdly - and my preference - instead of using sleep, do a wait on the event with a timeout. If the event is not set during the wait, wait returns false after waiting the timeout period. If the event is set before or during the wait, wait returns true immediately (that is, it aborts the timeout, giving you fast, clean shutdown of the thread.)
So your code would look something like this:
def daemon_loop(sleep_interval, stop_event):
for j in count():
print(j)
if stop_event.wait(sleep_interval):
break
print('Slept %s' % sleep_interval)
print('Prod terminating')

How to detect exceptions in concurrent.futures in Python3?

I have just moved on to python3 as a result of its concurrent futures module. I was wondering if I could get it to detect errors. I want to use concurrent futures to parallel program, if there are more efficient modules please let me know.
I do not like multiprocessing as it is too complicated and not much documentation is out. It would be great however if someone could write a Hello World without classes only functions using multiprocessing to parallel compute so that it is easy to understand.
Here is a simple script:
from concurrent.futures import ThreadPoolExecutor
def pri():
print("Hello World!!!")
def start():
try:
while True:
pri()
except KeyBoardInterrupt:
print("YOU PRESSED CTRL+C")
with ThreadPoolExecutor(max_workers=3) as exe:
exe.submit(start)
The above code was just a demo, of how CTRL+C will not work to print the statement.
What I want is to be able to call a function is an error is present. This error detection must be from the function itself.
Another example
import socket
from concurrent.futures import ThreadPoolExecutor
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
def con():
try:
s.connect((x,y))
main()
except: socket.gaierror
err()
def err():
time.sleep(1)
con()
def main():
s.send("[+] Hello")
with ThreadPoolExecutor as exe:
exe.submit(con)

Way too late to the party, but maybe it'll help someone else...
I'm pretty sure the original question was not really answered. Folks got hung up on the fact that user5327424 was using a keyboard interrupt to raise an exception when the point was that the exception (however it was caused) was not raised. For example:
import concurrent.futures
def main():
numbers = range(10)
with concurrent.futures.ThreadPoolExecutor() as executor:
results = {executor.submit(raise_my_exception, number): number for number in numbers}
def raise_my_exception(number):
print('Proof that this function is getting called. %s' % number)
raise Exception('This never sees the light of day...')
main()
When the example code above is executed, you will see the text inside the print statement displayed on the screen, but you will never see the exception. This is because the results of each thread are held in the results object. You need to iterate that object to get to your exceptions. The following example shows how to access the results.
import concurrent.futures
def main():
numbers = range(10)
with concurrent.futures.ThreadPoolExecutor() as executor:
results = {executor.submit(raise_my_exception, number): number for number in numbers}
for result in results:
# This will cause the exception to be raised (but only the first one)
print(result.result())
def raise_my_exception(number):
print('Proof that this function is getting called. %s' % number)
raise Exception('This will be raised once the results are iterated.')
main()
I'm not sure I like this behavior or not, but it does allow the threads to fully execute, regardless of the exceptions encountered inside the individual threads.

Here's a solution. I'm not sure you like it, but I can't think of any other. I've modified your code to make it work.
from concurrent.futures import ThreadPoolExecutor
import time
quit = False
def pri():
print("Hello World!!!")
def start():
while quit is not True:
time.sleep(1)
pri()
try:
pool = ThreadPoolExecutor(max_workers=3)
pool.submit(start)
while quit is not True:
print("hei")
time.sleep(1)
except KeyboardInterrupt:
quit = True
Here are the points:
When you use with ThreadPoolExecutor(max_workers=3) as exe, it waits until all tasks have been done. Have a look at Doc
If wait is True then this method will not return until all the pending futures are done executing and the resources associated with the executor have been freed. If wait is False then this method will return immediately and the resources associated with the executor will be freed when all pending futures are done executing. Regardless of the value of wait, the entire Python program will not exit until all pending futures are done executing.
You can avoid having to call this method explicitly if you use the with statement, which will shutdown the Executor (waiting as if Executor.shutdown() were called with wait set to True)
It's like calling join() on a thread.
That's why I replaced it with:
pool = ThreadPoolExecutor(max_workers=3)
pool.submit(start)
Main thread must be doing "work" to be able to catch a Ctrl+C. So you can't just leave main thread there and exit, the simplest way is to run an infinite loop
Now that you have a loop running in main thread, when you hit CTRL+C, program will enter the except KeyboardInterrupt block and set quit=True. Then your worker thread can exit.
Strictly speaking, this is only a workaround. It seems to me it's impossible to have another way for this.
Edit
I'm not sure what's bothering you, but you can catch exception in another thread without problem:
import socket
import time
from concurrent.futures import ThreadPoolExecutor
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
def con():
try:
raise socket.gaierror
main()
except socket.gaierror:
print("gaierror occurred")
err()
def err():
print("err invoked")
time.sleep(1)
con()
def main():
s.send("[+] Hello")
with ThreadPoolExecutor(3) as exe:
exe.submit(con)
Output
gaierror occurred
err invoked
gaierror occurred
err invoked
gaierror occurred
err invoked
gaierror occurred
...

Break loop when child process finishes

I am trying to make a simple program that will start a child process which writes a string to a pipe while the parent process counts until it gets the string from the pipe. My problem however is that when the program runs it'll either not count or will not stop counting. I want to know how I can check if the child process is still running and depending on that break out of the counting loop.
import os, time
pipein, pipeout = os.pipe()
def child(input, pipeout):
time.sleep(2)
msg = ('child got this %s' % input).encode()
os.write(pipeout, msg)
input = input()
pid = os.fork()
if pid:
i = 0
while True:
print(i)
time.sleep(1)
i += 1
try:
os.kill(pid, 0)
except OSError:
break
line = os.read(pipein, 32)
print(line)
else:
child(input, pipeout)

You should use the subprocess module, and then you can call poll()
use popen.poll()
Explained here
if Popen.poll() is not None:
//child process has terminated
[edit]:
"The only way to control the input and output streams and also retrieve the return codes is to use the subprocess module; these are only available on Unix."
Source

Tools for implementing a watchdog timer in python

I'm writing some code for testing multithreaded programs (student homework--likely buggy), and want to be able to detect when they deadlock. When running properly, the programs regularly produce output to stdout, so that makes it fairly straightforward: if no output for X seconds, kill it and report deadlock. Here's the function prototype:
def run_with_watchdog(command, timeout):
"""Run shell command, watching for output. If the program doesn't
produce any output for <timeout> seconds, kill it and return 1.
If the program ends successfully, return 0."""
I can write it myself, but it's a bit tricky to get right, so I would prefer to use existing code if possible. Anyone written something similar?
Ok, see solution below. The subprocess module might also be relevant if you're doing something similar.

You can use expect (tcl) or pexpect (python) to do this.
import pexpect
c=pexpect.spawn('your_command')
c.expect("expected_output_regular_expression", timeout=10)

Here's a very slightly tested, but seemingly working, solution:
import sys
import time
import pexpect
# From http://pypi.python.org/pypi/pexpect/
DEADLOCK = 1
def run_with_watchdog(shell_command, timeout):
"""Run <shell_command>, watching for output, and echoing it to stdout.
If the program doesn't produce any output for <timeout> seconds,
kill it and return 1. If the program ends successfully, return 0.
Note: Assumes timeout is >> 1 second. """
child = pexpect.spawn('/bin/bash', ["-c", shell_command])
child.logfile_read = sys.stdout
while True:
try:
child.read_nonblocking(1000, timeout)
except pexpect.TIMEOUT:
# Child seems deadlocked. Kill it, return 1.
child.close(True)
return DEADLOCK
except pexpect.EOF:
# Reached EOF, means child finished properly.
return 0
# Don't spin continuously.
time.sleep(1)
if __name__ == "__main__":
print "Running with timer..."
ret = run_with_watchdog("./test-program < trace3.txt", 10)
if ret == DEADLOCK:
print "DEADLOCK!"
else:
print "Finished normally"

Another solution:
class Watchdog:
def __init__(self, timeout, userHandler=None): # timeout in seconds
self.timeout = timeout
if userHandler != None:
self.timer = Timer(self.timeout, userHandler)
else:
self.timer = Timer(self.timeout, self.handler)
def reset(self):
self.timer.cancel()
self.timer = Timer(self.timeout, self.handler)
def stop(self):
self.timer.cancel()
def handler(self):
raise self;
Usage if you want to make sure function finishes in less than x seconds:
watchdog = Watchdog(x)
try
... do something that might hang ...
except Watchdog:
... handle watchdog error ...
watchdog.stop()
Usage if you regularly execute something and want to make sure it is executed at least every y seconds:
def myHandler():
print "Watchdog expired"
watchdog = Watchdog(y, myHandler)
def doSomethingRegularly():
...
watchdog.reset()