I want to deal with keyboard event in console with python. The running script has some persistent output stream, when admin trigger a keypress event, the script will change its output content.
I have done it with code as follows(press 'q' will trigger the output-change), but there are two issues
there is an increased space in my output. After debug, i find the code "tty.setraw(fd)" lead to that, But i don't know how to solve it
ctrl+c couldn't work anymore (if # "tty.setraw(fd)", ctrl+c will work)
If it is too complex, any else module could do what I want ? I tried curse module, it seems that will freeze the window-output and couldn't coordinate in mutlithread
#!/usr/bin/python
import sys
import select
import tty, termios
import threading
import time
def loop():
while loop_bool:
if switch:
output = 'aaaa'
else:
output = 'bbbb'
print output
time.sleep(0.2)
def change():
global switch
global loop_bool
fd = sys.stdin.fileno()
old_settings = termios.tcgetattr(fd)
try:
while loop_bool:
tty.setraw(fd)
i,o,e = select.select([sys.stdin],[],[],1)
if len(i)!=0:
if i[0] == sys.stdin:
input = sys.stdin.read(1)
if input =='q':
if switch:
switch = False
else:
switch = True
termios.tcsetattr(fd, termios.TCSADRAIN, old_settings)
except KeyboardInterrupt:
termios.tcsetattr(fd, termios.TCSADRAIN, old_settings)
loop_bool = False
try:
switch = True
loop_bool = True
t1=threading.Thread(target=loop)
t2=threading.Thread(target=change)
t1.start()
t2.start()
t1.join(1)
t2.join(1)
except KeyboardInterrupt:
loop_bool = False
This probably depends on what platform you're on, and maybe even what terminal emulator you're using, and I'm not sure whether it will solve your problem or not, but…
You should be able to get character-by-character input without calling tty.setraw, just by setting "canonical mode" off, which you do by masking out the ICANON bit in the lflag attribute of tcgetattr(). You may also need to set VMIN or VTIME attributes, but the defaults should already be correct.
For details, see the section "Canonical and noncanonical mode" in the linux man page, "Noncanonical Mode Input Processing" in the OS X man page, or the equivalent if you're on a different platform.
It's probably cleaner to write this as a context manager than to do explicit cleanup. Especially since your existing code does setraw each time through the loop, and only restores at the end; they should ideally be in matched pairs, and using a with statement guarantees that. (Also, this way you don't need to repeat yourself in the except clause and the normal flow.) So:
#contextlib.contextmanager
def decanonize(fd):
old_settings = termios.tcgetattr(fd)
new_settings = old_settings[:]
new_settings[3] &= ~termios.ICANON
termios.tcsetattr(fd, termios.TCSAFLUSH, new_settings)
yield
termios.tcsetattr(fd, termios.TCSAFLUSH, old_settings)
Now:
def change():
global switch
global loop_bool
with decanonize(sys.stdin.fileno()):
try:
while loop_bool:
i,o,e = select.select([sys.stdin],[],[],1)
if i and i[0] == sys.stdin:
input = sys.stdin.read(1)
if input =='q':
switch = not switch
except KeyboardInterrupt:
loop_bool = False
Or maybe you want the with block at a lower level (inside the while, or at least the try).
(PS, I transformed a few lines of your code into equivalent but simpler forms to remove a few levels of nesting.)
YMMV, but here's a test on my Mac:
Retina:test abarnert$ python termtest.py
aaaa
aaaa
aaaa
qbbbb
bbbb
bbbb
qaaaa
aaaa
aaaa
^CRetina:test abarnert$
This makes me think you might want to turn off input echo (which you do by new_settings[3] &= ~termios.ECHO), which implies that you probably want to replace the decanonize function with something more general, for temporarily setting or clearing arbitrary termios flags. (Also, it would be nice if tcgetattr returned a namedtuple instead of a list, so you could do new_settings.lflag instead of new_settings[3], or at least provided symbolic constants for the attribute indices.)
Meanwhile, from your comments, it sounds like ^C only works during the first second or two, and it has something to do with the timeout in the joins. This makes sense—the main thread just kicks off the two threads, does two join(1) calls, and then finishes. So, 2.something seconds after startup, it's finished all of its work—and left the try: block—so there's no longer any way for a KeyboardInterrupt to trigger the loop_bool = False and signal the worker threads to quit.
I'm not sure why you have timeouts on the joins in the first place, and what's supposed to happen when they time out, but there are three possibilities:
You don't want to quit until ^C, and the timeouts aren't there for any good reason. So take them out. Then the main thread will wait forever for the other two threads to finish, and it's still inside the try block, so ^C should be able to set loop_bool = False.
The app is supposed to exit normally after 2 seconds. (I'm guessing you would have preferred a single join-any or join-all on the pair of threads, with a 2-second timeout, but because Python doesn't have any easy way to do that, you joined the threads sequentially.) In this case, you want to set loop_bool = False as soon as the timeouts end. So just change the except to a finally.
The timeouts are always supposed to be generous enough (presumably this is just a simplified version of your real app), and if you get past the timeout, that's an exceptional condition. The previous option may or may not still work. If it doesn't, set daemon = True on the two threads, and they'll be killed (not nicely asked to shut down) when the main thread finishes. (Note that the way this works is a bit different on Windows vs. Unix—although you presumably don't care much about Windows for this app. More importantly, all the docs say is "The entire Python program exits when no alive non-daemon threads are left," so you shouldn't count on any daemon threads being able to do any cleanup, but also shouldn't count on them not doing any cleanup. Don't do anything in a daemon thread that could leave temporary files around, crucial log messages unwritten, etc.)
Related
I have the following Python program running in a Docker container.
Basically, if the Python process exits gracefully (ex. when I manually stop the container) or if the Python process crashes (while inside some_other_module.do_work()) then I need to do some cleanup and ping my DB telling it that process has exited.
What's the best way to accomplish this? I saw one answer where they did a try catch on main(), but that seems a bit odd.
My code:
def main():
some_other_module.do_work()
if __name__ == '__main__':
main()
I assume that the additional cleanup will be done by a different process, since the main process has likely crashed in a not recoverable way (I understood the question in this way).
The simplest way would be that the main process sets a flag somewhere (maybe creates a file in a specified location, or a column value in a database table; could also include the PID of the main process that sets the flag) when it starts and removes (or un-sets) that same flag if it finishes gracefully.
The cleanup process just needs to check the flag:
if the flag is set but the main process has ended already (the flag could contain the PID of the main process, so the cleanup process uses that to find if the main process is still running or not), then a cleanup is in order.
if the flag is set and the main process is running, then nothing is to be done.
if the flag is not set, then nothing is to be done.
Try-catch on main seems simplest, but doesn't/may not work for most things (please see comments below). You can always except specific exceptions:
def main():
some_other_module.do_work()
if __name__ == '__main__':
try:
main()
except Exception as e:
if e == "<INSERT GRACEFUL INTERRUPT HERE>":
# finished gracefully
else:
print(e)
# crash
Use a try/except
def thing_that_crashes():
exit()
try:
thing_that_crashes()
except:
print('oh and by the way, that thing tried to kill me')
I think it is impossible to catch a process with advanced suicidal behaviour (I don't know sending a SYGKILL to itself or something) so if you need your main process to live whatever happens, maybe run the other one in a subprocess.
You could wrap your script with another subprocess script and check the returncode. Inspired by this Relevant question.
from subprocess import Popen
script = Popen("python abspath/to/your/script.py")
script.communicate()
if script.returncode <> 0:
# something went wrong
# do something about it
I have a thread that monitors user input which looks like this:
def keyboard_monitor(temp): #temp is sys.stdin
global flag_exit
while True:
keyin = temp.readline().rstrip().lower()
if keyin == "exit":
flag_exit = True
print("Stopping...")
if flag_exit == True:
break
If I type exit the flag is properly set and all the other threads terminate. If another one of my threads sets the flag, this thread refuses to finish because it's hanging on the user input. I have to input something to get it to finish. How do I change this code so that the program finishes when the flag is set externally?
Its hard to tell exactly what is going wrong without more of your code, but as an easy solution you could rather exit() which is a python built in. This should reliably terminate the application, also sys.exit()
From wim's comment:
You can use the atexit module to register clean up handlers
import atexit
def cleanup():
pass
# TODO cleanup
atexit.register(cleanup)
I understand that there are similar topics on this (such as here) but my intended design is a little more complex.
I'm designing a CLI script that will be run in an SSH window. The script will be hosted and executed on an Ubuntu 14.10 server. It's intended to actively monitor, in the foreground, the current status of ports and clients on a host switch. Every 30 seconds or as defined by the user it will fetch data via SNMP and then refresh information and display it to the screen. When it's waiting for the next refresh there is a timer indicating when it will query the device again for information.
I want to allow the user to press specific keys to change the output view or edit key variables at any time. (The functionality is similar to the Unix top.) For example, pressing t would request them to enter a number of seconds desired between loops. h, m, or i would toggle showing/hiding certain columns. These would not pause the timer nor exit the loop since changes would be applied at the next refresh. r would force an immediate refresh and apply changes. q or Ctrl+C would exit the script.
The primary activity would look like this:
Query loop <-----------------------------
| |
| Process Data
| ^
| |
v Query Data #SNMPBULKWALK
Timer <------------- ^
| | | |
| | Check time remaining |
| | ^ |
| |_______________| |
|___________________________________|
With key-press interrupts it would act like this:
Query loop <----------------------
| | ???<---Change variables
| (Continue) ^
V | |
Timer <--------- !!INTERRUPT!!---------> Identify key
| | | ^
| | Check time remaining |
| | ^ |
| |___________| |
|_____________________________|
I'm kind of stumped here. I'm led to believe that I'll probably need to implement threading - which I do not have experience with - as a while loop by itself doesn't seem to satisfy what we need. I'm also unsure of how to inject the changes to the object that contains the variables (e.g. timer, flags for display formatting) since it will be constantly used by our loop.
It's nothing complicated, and nothing requiring any packages.
Its only problem is that it requires restoring terminal back to normal on program exit.
I.e. If program crashes the terminal will not be restored and user won't see what he is typing.
But if user knows what he is doing, he can force restart the shell and everything will be back to normal.
Of corse, you can use this code in easier manner and use raw_input() to do the stuff.
from thread import start_new_thread as thread
from time import sleep
# Get only one character from stdin without echoing it to stdout
import termios
import fcntl
import sys
import os
fd = sys.stdin.fileno()
oldterm, oldflags = None, None
def prepareterm ():
"""Turn off echoing"""
global oldterm, oldflags
if oldterm!=None and oldflags!=None: return
oldterm = termios.tcgetattr(fd)
newattr = oldterm[:] # Copy of attributes to change
newattr[3] = newattr[3] & ~termios.ICANON & ~termios.ECHO
termios.tcsetattr(fd, termios.TCSANOW, newattr)
oldflags = fcntl.fcntl(fd, fcntl.F_GETFL)
fcntl.fcntl(fd, fcntl.F_SETFL, oldflags | os.O_NONBLOCK)
def restoreterm ():
"""Restore terminal to its previous state"""
global oldterm, oldflags
if oldterm==None and oldflags==None: return
termios.tcsetattr(fd, termios.TCSAFLUSH, oldterm)
fcntl.fcntl(fd, fcntl.F_SETFL, oldflags)
oldterm, oldflags = None, None
def getchar():
"""Get character from stdin"""
prepareterm()
while 1:
try:
c = sys.stdin.read(1)
break
except IOError:
try: sleep(0.001)
except: restoreterm(); raise KeyboardInterrupt
restoreterm()
return c
def control ():
"""Waits for keypress"""
global running, done
while running:
c = getchar().lower()
print "Keypress:", c
if c=="q": print "Quitting!"; running = 0; break
done += 1
def work ():
"""Does the server-client part"""
global done
while running:
# Do your stuff here!!!
# Just to illustrate:
print "I am protending to work!\nPress Q to kill me!"
sleep(1)
print "I am done!\nExiting . . ."
done += 1
# Just to feel better
import atexit
atexit.register(restoreterm)
# Start the program
running = 1
done = 0
thread(work, ())
thread(control, ())
# Block the program not to close when threads detach from the main one:
while running:
try: sleep(0.2)
except: running = 0
# Wait for both threads to finish:
while done!=2:
try: sleep(0.001)
except: pass # Ignore KeyboardInterrupt
restoreterm() # Just in case!
In practice program should never be able to exit without restoring the terminal to normal, but shit happens.
Now, you can simplify things by using only one thread and your work loop put in the main thread, and use raw_input to acquire commands from user.
Or maybe even better, put your server-client code in the background and wait for input in main thread.
It will also probably be safer to use threading module instead of raw threads.
If you use asyncore module, you will get each client running for its own, and your main thread will be occupied with the asyncore.loop().
You can override it, I.e. rewrite it to check for input and other things you wish to do, while keeping asyncore checks synchronized.
Also, heavy loads require to override some nasty functions inside it, because its buffer is fixed to 512 bytes, if I am not mistaken. Otherwise, it may be a nice solution for your problem.
And, lastly, just to be clear, code for no echoing user input is taken and adapted from getpass module.
Just a little bit there is mine.
Ctrl+C is easy: it will throw an exception that you can catch (because the process is sent a signal when this happens).
For providing interactivity while waiting, you should look writing asynchronous code. Twisted is time-tested and capable, but has a slight learning curve (IMHO). There is also asyncore that might be easier to get started with, but more limited and I am not sure it handles your use case. There is also asyncio, but it only exists in Python 3.4.
So i'm on my Raspberry pi, and i'm wanting to check to see if a sensor has been activated. I'm potentially running off two different things though, one from a shell that i remote into, and the other off a LCD screen with a couple buttons directly connected to the RPi. I think the best way to do this is to run a loop to see if the user press a key (like the enter key or something) OR if the LCD interface has selected to go on. i'd like to run a loop that check if the user has pressed a key or if a variable has changed somewhere denoting the LCD interface has been changed and then move on with my code, but i don't know the best way to do this. Currently, this is what i have:
import thread
import time
import globals #this is where i keep my project wide global variables. it this this is a good way to do it....
try:
from msvcrt import getch
except ImportError:
def getch():
import sys, tty, termios
fd = sys.stdin.fileno()
old_settings = termios.tcgetattr(fd)
try:
tty.setraw(sys.stdin.fileno())
ch = sys.stdin.read(1)
finally:
termios.tcsetattr(fd, termios.TCSADRAIN, old_settings)
return ch
char = None
def keypress():
global char
char = getch()
thread.start_new_thread(keypress, ())
globals.LCDVar = 0
while True:
if char is not None:
print "Key pressed is " + char
break
if globals.LCDVar == 1
break
time.sleep(.1)
print "Program is running\n"
<Run whatever code next>
This works, but i'm not sure what happens to the thread that's created. Does the thread stay alive after i press a key? I if change the variable instead of pressing a key, won't the thread still be there? I would feel comfortable with this if the event would only happen once or twice, but this keypress/varble check might happen 100s or 1000s of times and i don't want to keep starting new threads over and over again.
Is there a better way to do this?
Thanks for any advice!!
The thread you created exits when the function called in start_new_thread returns, so you don't need to worry about it running forever (that's from the official documentation on thread.start_new_thread).
As far as the "best" way to do this, I think that reversing what is run in a separate thread and what is run in the main line of execution would be helpful. That is, do work in a separate thread and wait for a key press in the main thread. That's traditionally how functionality like this is implemented and it reduces some of the complexity of your loop.
I know how to drop into an interpreter with pdb and IPython, but this requires me knowing beforehand exactly where I want to stop. However, I often run number crunching scripts that take minutes to hours, and I would like to know exactly what it's progress is. One solution is to simply put lots of logging statements everywhere, but then I either inundate myself with too much information or fail to log exactly what I want to know.
Is there a way to initialize a listener loop that under some key combination will drop me into the code wherever it currently is? Think CTRL+Z but leaving me in Python rather than Bash.
You can use the signal module to setup a handler that will launch the debugger when you hit control-C or control-Z or whatever.. SIGINTR, SIGSUSP.
For example, define a module instant_debug.py that overrides SIGQUIT,
import signal
import pdb
def handler(signum, frame):
pdb.set_trace()
signal.signal(signal.SIGQUIT, handler)
Then make a script
import instant_debug
import time
for i in xrange(1000000):
print i
time.sleep(0.1)
At any point during execution, you can jump into the code by typing CTRL+\, examine the stack with u and d as in normal pdb, then continue with c as if nothing ever happened. Note that you will only jump in at the end of the next "atomic" operation -- that means no stopping in the middle of a giant C module.
You could do this
def main():
i = 1000
while True:
print "Count Down %s" % i
time.sleep(1)
i -= 1
try:
main()
except KeyboardInterrupt:
pass # Swallow ctrl-c
finally:
code.interact("Dropped into interpreter", local=globals())