In my case, I am handling databases throughout the entire runtime of my program, and so I need to keep my 'cursor' open for the entire program. Is there anyway I can implement a termination protocol, so that when I terminate its execution or an error arises, I am able to run this quick piece of code that simply closes the cursor (I am using python sockets btw).
I would suspect I could do something like this:
if __name__ == "__main__":
Menu()
cursor.close()
However, the only reason that this doesn't work in my case is that Menu is simply starting up threads, and so its execution continues on, returning me back to cursor.close() whilst my program continues to run.
I'm not sure if there is a way to get around this problem.
Yes, you could use the signal library in python to achieve some of this functionality, in particular, capturing program termination as well interrupts to the program like ctrl + c. Example:
# a function to register the signal handlers
# once the program terminates or is halted by an interrupt like ctrl + c it executes the quit_gracefully function
def register_signal_handler():
signal.signal(signal.SIGINT, quit_gracefully)
signal.signal(signal.SIGTERM, quit_gracefully)
return
def quit_gracefully():
# close connections etc.
in case of a different error you could use a try-except block which handles the error and runs the quit_gracefully function in the except.
try:
# some code
except:
quit_gracefully()
EDIT:
this is a good post on signal. How do I capture SIGINT in Python?
You can also use the atexit module: https://docs.python.org/3/library/atexit.html.
Something like this:
import atexit
#atexit.register
def close_cursor():
print("Closing cursor before exiting.")
cursor.close()
Related
I have the following Python program running in a Docker container.
Basically, if the Python process exits gracefully (ex. when I manually stop the container) or if the Python process crashes (while inside some_other_module.do_work()) then I need to do some cleanup and ping my DB telling it that process has exited.
What's the best way to accomplish this? I saw one answer where they did a try catch on main(), but that seems a bit odd.
My code:
def main():
some_other_module.do_work()
if __name__ == '__main__':
main()
I assume that the additional cleanup will be done by a different process, since the main process has likely crashed in a not recoverable way (I understood the question in this way).
The simplest way would be that the main process sets a flag somewhere (maybe creates a file in a specified location, or a column value in a database table; could also include the PID of the main process that sets the flag) when it starts and removes (or un-sets) that same flag if it finishes gracefully.
The cleanup process just needs to check the flag:
if the flag is set but the main process has ended already (the flag could contain the PID of the main process, so the cleanup process uses that to find if the main process is still running or not), then a cleanup is in order.
if the flag is set and the main process is running, then nothing is to be done.
if the flag is not set, then nothing is to be done.
Try-catch on main seems simplest, but doesn't/may not work for most things (please see comments below). You can always except specific exceptions:
def main():
some_other_module.do_work()
if __name__ == '__main__':
try:
main()
except Exception as e:
if e == "<INSERT GRACEFUL INTERRUPT HERE>":
# finished gracefully
else:
print(e)
# crash
Use a try/except
def thing_that_crashes():
exit()
try:
thing_that_crashes()
except:
print('oh and by the way, that thing tried to kill me')
I think it is impossible to catch a process with advanced suicidal behaviour (I don't know sending a SYGKILL to itself or something) so if you need your main process to live whatever happens, maybe run the other one in a subprocess.
You could wrap your script with another subprocess script and check the returncode. Inspired by this Relevant question.
from subprocess import Popen
script = Popen("python abspath/to/your/script.py")
script.communicate()
if script.returncode <> 0:
# something went wrong
# do something about it
I currently have code that basically runs an infinite while loop to collect data from users. Constantly updating dictionaries/lists based on the contents of a text file. For reference:
while (True):
IDs2=UpdatePoints(value,IDs2)
time.sleep(10)
Basically, my problem is that I do not know when I want this to end, but after this while loop runs I want to use the information collected, not lose it by crashing my program. Is there a simple, elegant way to simply exit out of the while loop whenever I want? Something like pressing a certain key on my keyboard would be awesome.
You can try wrapping that code in a try/except block, because keyboard interrupts are just exceptions:
try:
while True:
IDs2=UpdatePoints(value,IDs2)
time.sleep(10)
except KeyboardInterrupt:
print('interrupted!')
Then you can exit the loop with CTRL-C.
You could use exceptions. But you only should use exceptions for stuff that isn't supposed to happen. So not for this.
That is why I recommand signals:
import sys, signal
def signal_handler(signal, frame):
print("\nprogram exiting gracefully")
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
you should put this on the beginning of your program and when you press ctrl+c wherever in your program it will shut down gracefully
Code explanation:
You import sys and signals.
Then you make a function that executes on exit. sys.exit(0) stops the programming with exit code 0 (the code that says, everything went good).
When the program get the SIGINT either by ctrl-c or by a kill command in the terminal you program will shutdown gracefully.
I think the easiest solution would be to catch the KeyboardInterrupt when the interrupt key is pressed, and use that to determine when to stop the loop.
except KeyboardInterrupt:
break
The disadvantage of looking for this exception is that it may prevent the user from terminating the program while the loop is still running.
I use python to track stock prices and submit automated buy/sell commands on my portfolio. Long story short, I wanted my tracking program to ping the data server for info, and place trades off of the information gathered, but I also wanted to save the stock data for future reference, on top of being able to start/stop the program whenever I wanted.
What ended up working for me was the following:
trigger = True
while trigger == True:
try:
(tracking program and purchasing program conditions here)
except:
trigger = False
print('shutdown initialized')
df = pd.DataFrame...
save all the datas
print('shutdown complete')
etc.
From here, while the program is in the forever loop spamming away requests for data from my broker's API, using the CTRL-C keyboard interrupt function toggles the exception to the try loop, which nullifies the while loop, allowing the script to finalize the data saving protocol without bringing the entire script to an abrupt halt.
Hope this helps!
Resultant
I would suggest using the try, except syntax within a loop if you are running on an IPYNB file in Google Colab or Jupyter, like:
while True:
try:
IDs2=UpdatePoints(value,IDs2)
time.sleep(10)
except KeyboardInterrupt:
break
except:
continue
the last except is for any other error if occurs the loop will resume
You can catch the KeyboardInterrupt error in Python:
try:
while 1>0:
IDs2=UpdatePoints(value,IDs2)
time.sleep(10)
except KeyboardInterrupt:
print('While loop ended!')
Also, instead of saying:
while True:
It looks more professional to use:
while 1>0:
To read more about Python Error handling (try, except, etc.):
https://www.w3schools.com/python/python_try_except.asp
or:
https://www.w3schools.com/python/gloss_python_try_finally.asp
I have a problem with my script. i want do do an event listender for the whole script from beginning to the end.
if someone presses ctrl-c or something it should ignore, or end the scipt with an print.
I can show you my code if needed.
I tried signal and sys but I didnt know how to use it right
A simple approach for ending the script with a print could be to wrap the whole script in a try/except block:
import sys
try:
while 1:
print 'To infinity and beyond!'
# etc etc...
except KeyboardInterrupt:
print 'Handling the keyboard interrupt...'
The interrupt still kills the script, but I would think this is desired behaviour for any user; personally I wouldn't recommend trying to override it.
Following this example in the ØMQ docs, I'm trying to create a simple receiver. The example uses infinite loop. Everything works just fine. However, on MS Windows, when I hit CTRL+C to raise KeyboardInterrupt, the loop does not break. It seems that recv() method somehow ignores the exception. However, I'd love to exit the process by hiting CTRL+C instead of killing it. Is that possible?
In response to the #Cyclone's request, I suggest the following as a possible solution:
import signal
signal.signal(signal.SIGINT, signal.SIG_DFL);
# any pyzmq-related code, such as `reply = socket.recv()`
A zmq.Poller object seems to help:
def poll_socket(socket, timetick = 100):
poller = zmq.Poller()
poller.register(socket, zmq.POLLIN)
# wait up to 100msec
try:
while True:
obj = dict(poller.poll(timetick))
if socket in obj and obj[socket] == zmq.POLLIN:
yield socket.recv()
except KeyboardInterrupt:
pass
# Escape while loop if there's a keyboard interrupt.
Then you can do things like:
for message in poll_socket(socket):
handle_message(message)
and the for-loop will automatically terminate on Ctrl-C. It looks like the translation from Ctrl-C to a Python KeyboardInterrupt only happens when the interpreter is active and Python has not yielded control to low-level C code; the pyzmq recv() call apparently blocks while in low-level C code, so Python never gets a chance to issue the KeyboardInterrupt. But if you use zmq.Poller then it will stop at a timeout and give the interpreter a chance to issue the KeyboardInterrupt after the timeout is complete.
Don't know if this going to work in Windows, but in Linux I did something like this:
if signal.signal(signal.SIGINT, signal.SIG_DFL):
sys.exit()
Try ctrl+break (as in the key above Page Up, I had to look it up, I don't think I've ever touched that key before)
suggested near the bottom of this thread. I haven't done anything too fancy, but this seems to work well enough in the cases I've tried.
I know how to drop into an interpreter with pdb and IPython, but this requires me knowing beforehand exactly where I want to stop. However, I often run number crunching scripts that take minutes to hours, and I would like to know exactly what it's progress is. One solution is to simply put lots of logging statements everywhere, but then I either inundate myself with too much information or fail to log exactly what I want to know.
Is there a way to initialize a listener loop that under some key combination will drop me into the code wherever it currently is? Think CTRL+Z but leaving me in Python rather than Bash.
You can use the signal module to setup a handler that will launch the debugger when you hit control-C or control-Z or whatever.. SIGINTR, SIGSUSP.
For example, define a module instant_debug.py that overrides SIGQUIT,
import signal
import pdb
def handler(signum, frame):
pdb.set_trace()
signal.signal(signal.SIGQUIT, handler)
Then make a script
import instant_debug
import time
for i in xrange(1000000):
print i
time.sleep(0.1)
At any point during execution, you can jump into the code by typing CTRL+\, examine the stack with u and d as in normal pdb, then continue with c as if nothing ever happened. Note that you will only jump in at the end of the next "atomic" operation -- that means no stopping in the middle of a giant C module.
You could do this
def main():
i = 1000
while True:
print "Count Down %s" % i
time.sleep(1)
i -= 1
try:
main()
except KeyboardInterrupt:
pass # Swallow ctrl-c
finally:
code.interact("Dropped into interpreter", local=globals())