I'm writing a python script, that should behave like a typical shell and providing some self written functions.
It is working quite well already, but it always exits after a successful command, so that it has to be started again to perform a second task.
How can I make it, so it doesn't finish with exit code 0 but returns to shell awaiting new input? How would I have to implement exit methods then?
Following example always exits after typing print-a or print-b:
import click
import click_repl
from prompt_toolkit.history import FileHistory
import os
#click.group(invoke_without_command=True)
#click.pass_context
def cli(ctx):
if ctx.invoked_subcommand is None:
ctx.invoke(repl)
#cli.command()
def print_a():
print("a")
#cli.command()
def print_b():
print("b")
#cli.command()
def repl():
prompt_kwargs = {
'history': FileHistory(os.path.expanduser('~/.repl_history'))
}
click_repl.repl(click.get_current_context(), prompt_kwargs)
def main():
while True:
cli(obj={})
if __name__ == "__main__":
main()
(And a bonus question: In the cmd package it is possible to customize the > prompt tag, is this possible with click to? So that it's something like App> instead?)
Use the standalone_mode argument, try this:
rv = cli(obj={}, standalone_mode=False)
When parsing failed, the code above will throw a UsageError. When --help was passed, rv will be the integer 0. In most other cases the return value of the function that handles the command is returned, although there are a bunch of exceptions and the behavior in general is quite complex, more explanations here:
https://click.palletsprojects.com/en/master/commands/#command-return-values
The advantage of this approach is that you can use return values from command handlers. The disadvantage is that you lose the pretty printed help message when parsing failed (maybe there is way to restore it?).
Another option is to not use standalone_mode and instead wrap your call to cli in a try/except block where you catch a SystemExit:
try:
cli(obj={})
except SystemExit as e:
if e.code != 0:
raise
By catching SystemExit you can stop the program exit process initiated by click. If the command parsed succesfully then SystemExit(0) is caught. Note again that parsing --help also counts as a 'successfull' parse, and therefore also returns SystemExit(0).
The disadvantage of this approach is that you cannot use the return value of a command handler, which makes it more difficult to know when --help was passed. The upside is that all help messages to the console are restored.
I should also note that SystemExit inherits from BaseException but not from Exception. So to actually catch SystemExit you can either catch it directly or catch BaseException.
You could check out click-shell, which is a wrapper for click and the python cmd module. It supports auto completion and help from docstrings out of the box.
Alongside the click_shell there is yet another option which is click_repl.
Related
I need to say first and foremost, I am just learning Python.
I am making a simple python program that has a menu option for exiting the program by using a function I called exit. I have tried making the exit function just call break, but I am getting an error when the exit function is called.
Any help would be greatly appreciated.
Sorry for not posting code earlier....
def exit():
break
evade = evade_fw()
# Main program running dialogue
def main(): # menu goes here
opt_list = [xsl_file,
basic_loud_scan,
fw_main,
exit
]
Just forget about your own exit() function. You can simply do:
from sys import exit
And the exit() function from sys module will do the job.
It's also worth to know what happens under the hood. Function sys.exit() actually throws a special exception. You can do it as well explicitly and without importing anything:
raise SystemExit()
break is for breaking out of for or while loops, but it must be called from within the loop. I'm guessing that you expect the break to break out of your program's main event loop from an event handler, and that is not going to work because, as aforementioned, the break must be within the loop itself.
Instead your exit function can clean up any resources, e.g. open files, database connections, etc. then call [sys.exit()][1] which will cause the Python interpreter to terminate. You can optionally pass a status code to sys.exit() which will be the system exit status available to shell scripts and batch files.
import sys
def exit():
# clean up resources
sys.exit() # defaults to status 0 == success
I use a list of different python scripts for various functions. To help facilitate this, I've organized all of my reusable functions into custom libraries. However, I found that many of these functions will error out for strange reasons, some known and some unknown. I designed the below function to at least let me see the error message before throwing a giant traceback at me. I have the below command in one library:
FooBar = trace_check(lambda: Foo(bar))
This is the error catching function in a separate library:
def trace_check(func):
try:
return func()
except:
TracebackString = traceback.format_exc() ###This gets the traceback as a string.
type, message, tracebacklocation = sys.exc_info() ###This gets the components, particularly the message.
print "An error has occurred with the following error message:"
print type ###Example is IOError
print message ###The message associated with the error
TracebackPrompt = ask("Do you want to see the entire traceback?") #Returns True/False value
if TracebackPrompt:
print TracebackString
print 'Exiting Python Script' ###This shows me it gets to this point.
sys.exit(0) ###This is the problem
print "Did it work?" ###This statement does not print, so exit worked...
When trace_check runs and I get an error, the sys.exit only quits the function back out to main() instead of ending main. if I use os._exit() instead, the main() function ends correctly, but the program running the script also dies. One command is not strong enough and the other is overkill... what could I do instead to ensure the main() function ends?
Note: I tried putting the meat of my trace_check function into the first library, but the same thing happens with the library call ending but not the main().
tl;dr - Python: main() calls function in library that calls a second function in separate library. Second function has a sys.exit() command that only exits to main() instead of ending main(). os._exit() kills shell and is overkill (requires restarting shell TT^TT). Is there another way to end the main() from a function library?
To directly answer your question, if you want to handle sys.exit() calls from main, then you should catch the SystemExit exception that is raised by sys.exit(). The sample code below illustrates how to do that.
import sys
def func():
sys.exit(1)
def main():
try:
func()
except SystemExit:
print 'Someone sys.exit()d'
return 0
if __name__ == '__main__':
sys.exit(main())
However! You should probably redesign your library. Instead of calling sys.exit() when something unexpected happens, you should raise an Exception. Having a library abruptly exit the interpreter is bad design.
You could try setting this off by throwing an exception:
class ExitFromMain(Exception):
pass
def trace_check(func):
try:
# try stuff
except:
# traceback stuff you had
raise ExitFromMain()
def main():
try:
# Stuff
trace_check()
# More stuff that will not run if the exception is thrown
except ExitFromMain:
print "I hit my excception to flag a quit from this function"
sys.exit(0)
I'm using custom exceptions to differ my exceptions from Python's default exceptions.
Is there a way to define a custom exit code when I raise the exception?
class MyException(Exception):
pass
def do_something_bad():
raise MyException('This is a custom exception')
if __name__ == '__main__':
try:
do_something_bad()
except:
print('Oops') # Do some exception handling
raise
In this code, the main function runs a few functions in a try code.
After I catch an exception I want to re-raise it to preserve the traceback stack.
The problem is that 'raise' always exits 1.
I want to exit the script with a custom exit code (for my custom exception), and exit 1 in any other case.
I've looked at this solution but it's not what I'm looking for:
Setting exit code in Python when an exception is raised
This solution forces me to check in every script I use whether the exception is a default or a custom one.
I want my custom exception to be able to tell the raise function what exit code to use.
You can override sys.excepthook to do what you want yourself:
import sys
class ExitCodeException(Exception):
"base class for all exceptions which shall set the exit code"
def getExitCode(self):
"meant to be overridden in subclass"
return 3
def handleUncaughtException(exctype, value, trace):
oldHook(exctype, value, trace)
if isinstance(value, ExitCodeException):
sys.exit(value.getExitCode())
sys.excepthook, oldHook = handleUncaughtException, sys.excepthook
This way you can put this code in a special module which all your code just needs to import.
Some programmers use sys.exit, others use SystemExit.
What is the difference?
When do I need to use SystemExit or sys.exit inside a function?
Example:
ref = osgeo.ogr.Open(reference)
if ref is None:
raise SystemExit('Unable to open %s' % reference)
or:
ref = osgeo.ogr.Open(reference)
if ref is None:
print('Unable to open %s' % reference)
sys.exit(-1)
No practical difference, but there's another difference in your example code - print goes to standard out, but the exception text goes to standard error (which is probably what you want).
sys.exit(s) is just shorthand for raise SystemExit(s), as described in the former's docstring; try help(sys.exit). So, instead of either one of your example programs, you can do
sys.exit('Unable to open %s' % reference)
There are 3 exit functions, in addition to raising SystemExit.
The underlying one is os._exit, which requires 1 int argument, and exits immediately with no cleanup. It's unlikely you'll ever want to touch this one, but it is there.
sys.exit is defined in sysmodule.c and just runs PyErr_SetObject(PyExc_SystemExit, exit_code);, which is effectively the same as directly raising SystemExit. In fine detail, raising SystemExit is probably faster, since sys.exit requires an LOAD_ATTR and CALL_FUNCTION vs RAISE_VARARGS opcalls. Also, raise SystemExit produces slightly smaller bytecode (4bytes less), (1 byte extra if you use from sys import exit since sys.exit is expected to return None, so includes an extra POP_TOP).
The last exit function is defined in site.py, and aliased to exit or quit in the REPL. It's actually an instance of the Quitter class (so it can have a custom __repr__, so is probably the slowest running. Also, it closes sys.stdin prior to raising SystemExit, so it's recommended for use only in the REPL.
As for how SystemExit is handled, it eventually causes the VM to call os._exit, but before that, it does some cleanup. It also runs atexit._run_exitfuncs() which runs any callbacks registered via the atexit module. Calling os._exit directly bypasses the atexit step.
My personal preference is that at the very least SystemExit is raised (or even better - a more meaningful and well documented custom exception) and then caught as close to the "main" function as possible, which can then have a last chance to deem it a valid exit or not. Libraries/deeply embedded functions that have sys.exit is just plain nasty from a design point of view. (Generally, exiting should be "as high up" as possible)
According to documentation sys.exit(s) effectively does raise SystemExit(s), so it's pretty much the same thing.
While the difference has been answered by many answers, Cameron Simpson makes an interesting point in https://mail.python.org/pipermail/python-list/2016-April/857869.html:
TL;DR: It's better to just raise a "normal" exception, and use SystemExit or sys.exit only at the top levels of a script.
I m on python 2.7 and Linux , I have a simple code need suggestion if I
I could replace sys.exit(1) with raise SystemExit .
==Actual code==
def main():
try:
create_logdir()
create_dataset()
unittest.main()
except Exception as e:
logging.exception(e)
sys.exit(EXIT_STATUS_ERROR)
if __name__ == '__main__': main()
==Changed Code==
def main():
try:
create_logdir()
create_dataset()
unittest.main()
except Exception as e:
logging.exception(e)
raise SystemExit
if __name__ == '__main__':
main()
I am against both of these personally. My preferred pattern is like
this:
def main(argv):
try:
...
except Exception as e:
logging.exception(e)
return 1
if __name__ == '__main__':
sys.exit(main(sys.argv))
Notice that main() is back to being a normal function with normal
returns.
Also, most of us would avoid the "except Exception" and just let a top
level except bubble out: that way you get a stack backtrace for
debugging. I agree it prevents logging the exception and makes for
uglier console output, but I think it is a win. And if you do want
to log the exception there is always this:
try:
... except Exception as e:
logging.exception(e)
raise
to recite the exception into the log and still let it bubble out
normally.
The problem with the "except Exception" pattern is that it catches and
hides
every exception, not merely the narrow set of specific exceptions that you understand.
Finally, it is frowned upon to raise a bare Exception class. In
python 3 I believe it is actually forbidden, so it is nonportable
anyway. But even In Python to it is best to supply an Exception
instance, not the class:
raise SystemExit(1)
All the functions in try block have exception bubbled out using raise
Example for create_logdir() here is the function definition
def create_logdir():
try:
os.makedirs(LOG_DIR)
except OSError as e:
sys.stderr.write("Failed to create log directory...Exiting !!!")
raise
print "log file: " + corrupt_log
return True
def main():
try:
create_logdir()
except Exception as e:
logging.exception(e)
raise SystemExit
(a) In case if create_logdir() fails we will get the below error ,is
this fine or do I need to improve this code.
Failed to create log directory...Exiting !!!ERROR:root:[Errno 17] File
exists: '/var/log/dummy'
Traceback (most recent call last):
File "corrupt_test.py", line 245, in main
create_logdir()
File "corrupt_test.py", line 53, in create_logdir
os.makedirs(LOG_DIR)
File "/usr/local/lib/python2.7/os.py", line 157, in makedirs
OSError: [Errno 17] File exists: '/var/log/dummy'
I prefer the bubble out approach, perhap with a log or warning
messages as you have done, eg:
logging.exception("create_logdir failed: makedirs(%r): %s" %
(LOG_DIR, e)) raise
(Also not that that log message records more context: context is very
useful when debugging problems.)
For very small scripts sys.stderr.write is ok, but in general any of
your functions that turned out to be generally useful might migrate
into a library in order to be reused; consider that stderr is not
always the place for messages; instead reading for the logging module
with error() or wanr() or exception() as appropriate. There is more
scope for configuring where the output goes that way without wiring
it into your inner functions.
Can I have just raise , instead of SystemExit or sys.exit(1) . This
looks wrong to me
def main():
try:
create_logdir()
except Exception as e
logging.exception(e)
raise
This is what I would do, myself.
Think: has the exception been "handled", meaning has the situation
been dealt with because it was expected? If not, let the exception
bubble out so that the user knows that something not understood by
the program has occurred.
Finally, it is generally bad to SystemExit or sys.exit() from inside
anything other than the outermost main() function. And I resist it
even there; the main function, if written well, may often be called
from somewhere else usefully, and that makes it effectively a library
function (it has been reused). Such a function should not
unilaterally abort the program. How rude! Instead, let the exception
bubble out: perhaps the caller of main() expects it and can handle
it. By aborting and not "raise"ing, you have deprived the caller of
the chance to do something appropriate, even though you yourself
(i.e. "main") do not know enough context to handle the exception.
So I am for "raise" myself. And then only because you want to log the
error. If you didn't want to log the exception you could avoid the
try/except entirely and have simpler code: let the caller worry
about unhandled exceptions!
SystemExit is an exception, which basically means that your progam had a behavior such that you want to stop it and raise an error. sys.exit is the function that you can call to exit from your program, possibily giving a return code to the system.
EDIT: they are indeed the same thing, so the only difference is in the logic behind in your program. An exception is some kind of "unwanted" behaviour, whether a call to a function is, from a programmer point of view, more of a "standard" action.
Is it bad form to exit() from within function?
def respond_OK():
sys.stdout.write('action=OK\n\n')
sys.stdout.flush() # redundant when followed by exit()
sys.exit(0)
Rather than setting an exit code and exit()ing from the __main__ name space?
def respond_OK():
global exit_status
sys.stdout.write('action=OK\n\n')
sys.stdout.flush()
exit_status = 0
sys.exit(exit_status)
The difference is negligible from a function perspective, just wondered what the consensus is on form. If you found the prior in someone else's code, would you look at it twice?
I would prefer to see an exception raised and handled from a main entry point, the type of which is translated into the exit code. Subclassing exceptions is so simple in python it's almost fun.
As posted in this answer's comments: Using sys.exit also means that the point of termination needs to know the actual status code, as opposed to the kind of error it encountered. Though that could be solved by an set of constants, of course. Using exceptions has other advantages, though: if one method fails, you could try another without re-entry, or print some post-mortem debugging info.
It makes no difference in terms of functionality, but it will likely make your code harder to follow, unless you take appropriate steps, e.g. commenting each of the calls from the main namespace which could lead to an exit.
Update: Note #mgilson's answer re the effect of catching an exception [It is possible to catch the exception that system.exit raises, and thus prevent exit]. You could make your code even more confusing that way.
Update 2: Note #sapht's suggestion to use an exception to orchestrate an exit. This is good advice, if you really want to do a non-local exit. Much better than setting a global.
There are a few cases where it's reasonably idiomatic.
If the user gives you bad command-line arguments, instead of this:
def usage(arg0):
print ... % (arg0,)
return 2
if __name__ == '__main__':
if ...:
sys.exit(usage(sys.argv[0]))
You often see this:
def usage():
print ... % (sys.argv[0],)
sys.exit(2)
if __name__ == '__main__':
if ...:
usage()
The only other common case I can think of is where initializing some library (via ctypes or a low-level C extension module) fails unexpectedly and leaves you in a state you can't reason about, so you just want to get out as soon as possible (e.g., to reduce the chance of segfaulting or printing garbage) For example:
if libfoo.initialize() != 0:
sys.exit(1)
Some might object to that because sys.exit doesn't actually bail out of the interpreter as soon as possible (it throws and catches an exception), so it's a false sense of safety. But you still see it reasonably often.