I'm using click in Python to create a CLI for my program. I want to execute a function (below represented through the placeholder do_something()) if an exception occurs, but only if a certain argument is passed through the CLI (here: --verbose or -v).
I know that I can access command line input via sys.argv. This is my conceptual minimal working example using this approach:
#filename: my_app.py
import click
#click.group()
#click.option('--verbose', '-v', is_flag=True)
def cli():
pass
#click.command()
def greeting():
print('hello world')
raise Exception
#cli.command()
def spam():
print('SPAM')
raise Exception
def do_something():
print('Doing something...')
if __name__ == '__main__':
try:
main()
except e:
if '--verbose' in sys.argv or '-v' in sys.argv:
do_something()
raise e
To be as exact as I can be: In case of an exception in any of the subcommands, I want the program to do_something() when the --verbose flag is on, but not if it's off. E.g. for the greeting subcommand the CLI should behave like this:
~$ python my_app.py --verbose greeting
hello world
Doing something...
Exception
~$ python my_app.py greeting
hello world
Exception
While the sys.argv solution shown above works, it would be nice if I could refer to the command line option with click's internal name (which is 'verbose' in this case). I.e., I would like to do something like this here:
...
if __name__ == '__main__':
try:
main()
except e:
if 'verbose' in click.argv:
do_something()
raise e
Is this a feature that click provides?
Correct me if I'm wrong, but it seems to me like you'd like to be able to output more information/clean up something in the case of an exception.
There are a few options you could try:
Move the try/except logic inside the greeting method that leverages the verbose flag, and call do_something there
Consider looking at callbacks and see if you can leverage these to do what you want. Without more context into do_something, I can't tell for sure whether it would be a good use case for callbacks.
The closest thing to your question is to have a global scope variable _verbose like in the example below:
import click
global _verbose
_verbose = False
def do_something():
print(f'Doing something')
#click.group()
def cli():
pass
#cli.command()
#click.option('--verbose', '-v', is_flag=True)
def greeting():
global _verbose
_verbose = verbose
if verbose:
print('hello world')
raise Exception("Test")
else:
print('hello')
#cli.command()
#click.option('--count', '-c')
def spam(count):
print(count * 'SPAM')
if __name__ == '__main__':
try:
cli()
except Exception as e:
if _verbose:
do_something()
raise e
I've updated my answer with a working snippet that doesn't rely on globals to do this, but instead leverages Python's pass-by-reference and click's advanced context examples
import click
def do_something():
print(f'Doing something')
#click.group()
#click.option('--verbose', '-v', is_flag=True)
#click.pass_context
def cli(ctx, verbose):
ctx.obj['verbose'] = verbose
pass
#cli.command()
#click.pass_context
def greeting(ctx):
if ctx.obj['verbose']:
print('hello world')
raise Exception("Test")
else:
print('hello')
#cli.command()
#click.option('--count', '-c')
def spam(count):
print(count * 'SPAM')
if __name__ == '__main__':
flags = {}
try:
cli(obj=flags)
except Exception as e:
if flags['verbose']:
do_something()
raise e
Related
Is there a way to automatically start the debugger at the point at which a unittest fails?
Right now I am just using pdb.set_trace() manually, but this is very tedious as I need to add it each time and take it out at the end.
For Example:
import unittest
class tests(unittest.TestCase):
def setUp(self):
pass
def test_trigger_pdb(self):
#this is the way I do it now
try:
assert 1==0
except AssertionError:
import pdb
pdb.set_trace()
def test_no_trigger(self):
#this is the way I would like to do it:
a=1
b=2
assert a==b
#magically, pdb would start here
#so that I could inspect the values of a and b
if __name__=='__main__':
#In the documentation the unittest.TestCase has a debug() method
#but I don't understand how to use it
#A=tests()
#A.debug(A)
unittest.main()
I think what you are looking for is nose. It works like a test runner for unittest.
You can drop into the debugger on errors, with the following command:
nosetests --pdb
import unittest
import sys
import pdb
import functools
import traceback
def debug_on(*exceptions):
if not exceptions:
exceptions = (AssertionError, )
def decorator(f):
#functools.wraps(f)
def wrapper(*args, **kwargs):
try:
return f(*args, **kwargs)
except exceptions:
info = sys.exc_info()
traceback.print_exception(*info)
pdb.post_mortem(info[2])
return wrapper
return decorator
class tests(unittest.TestCase):
#debug_on()
def test_trigger_pdb(self):
assert 1 == 0
I corrected the code to call post_mortem on the exception instead of set_trace.
Third party test framework enhancements generally seem to include the feature (nose and nose2 were already mentioned in other answers). Some more:
pytest supports it.
pytest --pdb
Or if you use absl-py's absltest instead of unittest module:
name_of_test.py --pdb_post_mortem
A simple option is to just run the tests without result collection and letting the first exception crash down the stack (for arbitrary post mortem handling) by e.g.
try: unittest.findTestCases(__main__).debug()
except:
pdb.post_mortem(sys.exc_info()[2])
Another option: Override unittest.TextTestResult's addError and addFailure in a debug test runner for immediate post_mortem debugging (before tearDown()) - or for collecting and handling errors & tracebacks in an advanced way.
(Doesn't require extra frameworks or an extra decorator for test methods)
Basic example:
import unittest, pdb
class TC(unittest.TestCase):
def testZeroDiv(self):
1 / 0
def debugTestRunner(post_mortem=None):
"""unittest runner doing post mortem debugging on failing tests"""
if post_mortem is None:
post_mortem = pdb.post_mortem
class DebugTestResult(unittest.TextTestResult):
def addError(self, test, err):
# called before tearDown()
traceback.print_exception(*err)
post_mortem(err[2])
super(DebugTestResult, self).addError(test, err)
def addFailure(self, test, err):
traceback.print_exception(*err)
post_mortem(err[2])
super(DebugTestResult, self).addFailure(test, err)
return unittest.TextTestRunner(resultclass=DebugTestResult)
if __name__ == '__main__':
##unittest.main()
unittest.main(testRunner=debugTestRunner())
##unittest.main(testRunner=debugTestRunner(pywin.debugger.post_mortem))
##unittest.findTestCases(__main__).debug()
To apply #cmcginty's answer to the successor nose 2 (recommended by nose available on Debian-based systems via apt-get install nose2), you can drop into the debugger on failures and errors by calling
nose2
in your test directory.
For this, you need to have a suitable .unittest.cfg in your home directory or unittest.cfg in the project directory; it needs to contain the lines
[debugger]
always-on = True
errors-only = False
To address the comment in your code "In the documentation the unittest.TestCase has a debug() method but I don't understand how to use it", you can do something like this:
suite = unittest.defaultTestLoader.loadTestsFromModule(sys.modules[__name__])
suite.debug()
Individual test cases are created like:
testCase = tests('test_trigger_pdb') (where tests is a sub-class of TestCase as per your example). And then you can do testCase.debug() to debug one case.
Here's a built-in, no extra modules, solution:
import unittest
import sys
import pdb
####################################
def ppdb(e=None):
"""conditional debugging
use with: `if ppdb(): pdb.set_trace()`
"""
return ppdb.enabled
ppdb.enabled = False
###################################
class SomeTest(unittest.TestCase):
def test_success(self):
try:
pass
except Exception, e:
if ppdb(): pdb.set_trace()
raise
def test_fail(self):
try:
res = 1/0
#note: a `nosetests --pdb` run will stop after any exception
#even one without try/except and ppdb() does not not modify that.
except Exception, e:
if ppdb(): pdb.set_trace()
raise
if __name__ == '__main__':
#conditional debugging, but not in nosetests
if "--pdb" in sys.argv:
print "pdb requested"
ppdb.enabled = not sys.argv[0].endswith("nosetests")
sys.argv.remove("--pdb")
unittest.main()
call it with python myunittest.py --pdb and it will halt. Otherwise it won't.
Some solution above modifies business logic:
try: # <-- new code
original_code() # <-- changed (indented)
except Exception as e: # <-- new code
pdb.post_mortem(...) # <-- new code
To minimize changes to the original code, we can define a function decorator, and simply decorate the function that's throwing:
def pm(func):
import functools, pdb
#functools.wraps(func)
def func2(*args, **kwargs):
try:
return func(*args, **kwargs)
except Exception as e:
pdb.post_mortem(e.__traceback__)
raise
return func2
Use:
#pm
def test_xxx(...):
...
Buildt a module with a decorator which post mortems into every type of error except AssertionError. The decorator can be triggered by the logging root level
#!/usr/bin/env python3
'''
Decorator for getting post mortem on errors of a unittest TestCase
'''
import sys
import pdb
import functools
import traceback
import logging
import unittest
logging.basicConfig(format='%(asctime)s %(message)s', level=logging.DEBUG)
def debug_on(log_level):
'''
Function decorator for post mortem debugging unittest functions.
Args:
log_level (int): logging levels coesponding to logging stl module
Usecase:
class tests(unittest.TestCase):
#debug_on(logging.root.level)
def test_trigger_pdb(self):
assert 1 == 0
'''
def decorator(f):
#functools.wraps(f)
def wrapper(*args, **kwargs):
try:
return f(*args, **kwargs)
except BaseException as err:
info = sys.exc_info()
traceback.print_exception(*info)
if log_level < logging.INFO and type(err) != AssertionError:
pdb.post_mortem(info[2])
return wrapper
return decorator
class Debug_onTester(unittest.TestCase):
#debug_on(logging.root.level)
def test_trigger_pdb(self):
assert 1 == 0
if __name__ == '__main__':
unittest.main()
I would like to use sys.exit() within a function my_func(), how do I use it?
def my_func():
try:
some_method()
except:
print('Error')
sys.exit()
if __name__ == "__main__":
my_func()
You need to import sys before you can use it. Just add it to the top of your script:
import sys
def my_func():
try:
some_method()
except:
print('Error')
sys.exit()
if __name__ == "__main__":
my_func()
I see three programming issues. Besides the missing import sys, you flag an error but simply call sys.exit() which exits with a zero indicating success to the invoker -- it needs to exit non-zero to indicate error. Finally you print the error message to sys.stdout when it really should go to sys.stderr. A rework of your code:
import sys
def my_func():
try:
some_method()
except:
print('Error', file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
my_func()
Fortunately, Python allows us to combine these last two issues by instead doing:
import sys
def my_func():
try:
some_method()
except:
sys.exit('Error')
if __name__ == "__main__":
my_func()
This will exit non-zero and print the message to sys.stderr.
#!/usr/bin/env python
class Functions() :
def A(self):
print "hey"
self.B()
return 1
def B(self):
print "hello"
exit(0)
func_obj = Functions()
def main() :
A = func_obj.A()
print A
print "awesome"
if __name__ == '__main__' :
main()
Above is my code. What I'm trying to do is that I want to call functionA from the main() function and when functionA executes functionB, I want functionB to raise error and exit back to the main() function without going back to functionA. How can I achieve this? Basically I want the main function to print "awesome" after functionB exits. I'm not sure what is the correct keyword to look it up.
What you're looking for are exceptions - they are actually designed to do just this: break the normal flow of execution and propagate up the call stack until someone take care of them (as a last resort, the runtime will catch them, display the error message and a full traceback, and exit).
There are two parts to the process: first raising the exception, then catching it at the right place. In your example it might look like:
# declare our own exception type so we can catch specifically this one
class MyOwnException(Exception):
pass
def a():
print("in a - before b")
b()
print("in a - after b (shouldn't see this)")
def b():
print("in b, before raise")
raise MyOwnException("hello houston ?")
print("in b, after raise (shouldn't see this)")
if __name__ == "__main__":
print("calling a")
try:
a()
print("after a (shouldn't see this)")
except MyOwnException as e:
print("caugth error {}".format(e))
FWIW, your example using exit() was really close since exit() actually works by raising a SysExit exception. The first and main use case for exception is of course error handling, but it's actually really a way to control the execution flow of your program (as an example the StopIteration exception is used to signal an exhausted iterator).
Hey so after someone pointed out my original answer didnt work i went searching! you can create custom exception classes to achieve what you're looking for!
class HaltException(Exception):
pass
class Functions():
def a(self):
print("hey")
self.b()
return "1"
def b(self):
print("hello")
raise HaltException("This is an exception error.")
def main():
func_obj = Functions()
try:
func_obj.a()
except HaltException as error:
print(error)
print("Awesome")
if __name__ == "__main__":
main()
This would then return the following when run :
hey
hello
This is an exception error.
Awesome
I have the following code:
import click
#click.group()
def cli():
pass
#click.command()
def initdb():
click.echo('Initialized the database')
#click.command()
def dropdb():
click.echo('Dropped the database')
cli.add_command(initdb)
cli.add_command(dropdb)
At the command line I want to be able to do something like the following:
python clicktest.py cli initdb
and have the following happen echo at the terminal:
Initialized the database
Or to enter into the terminal:
python clicktest.py cli dropdb
and have the following happen on the terminal:
Dropped the database
My problem is currently when I do this at the terminal:
python clicktest.py cli initdb
Nothing happens at the terminal, nothing prints when I think something should, namely the 'Initialized Database' echo. What am i doing wrong??
First, you should use it in command line like:
python clicktest.py initdb
python clicktest.py dropdb
And in your clicktest.py file, place these lines at the bottom of your code:
if __name__ == '__main__':
cli()
Unless, your code won't get work.
EDIT:
If you really want to use it in a way python clicktest.py cli initdb, then you have another choice:
#click.group()
def main():
pass
#click.group()
def cli():
pass
#click.command()
def initdb():
click.echo('Initialized the database')
#click.command()
def dropdb():
click.echo('Dropped the database')
cli.add_command(initdb)
cli.add_command(dropdb)
main.add_command(cli)
if __name__ == '__main__':
main()
Or even better (Using decorators instead):
#click.group()
def main():
pass
#main.group()
def cli():
pass
#cli.command()
def initdb():
click.echo('Initialized the database')
#cli.command()
def dropdb():
click.echo('Dropped the database')
if __name__ == '__main__':
main()
I'm using click (http://click.pocoo.org/3/) to create a command line application, but I don't know how to create a shell for this application.
Suppose I'm writing a program called test and I have commands called subtest1 and subtest2
I was able to make it work from terminal like:
$ test subtest1
$ test subtest2
But what I was thinking about is a shell, so I could do:
$ test
>> subtest1
>> subtest2
Is this possible with click?
This is not impossible with click, but there's no built-in support for that either. The first you would have to do is making your group callback invokable without a subcommand by passing invoke_without_command=True into the group decorator (as described here). Then your group callback would have to implement a REPL. Python has the cmd framework for doing this in the standard library. Making the click subcommands available there involves overriding cmd.Cmd.default, like in the code snippet below. Getting all the details right, like help, should be doable in a few lines.
import click
import cmd
class REPL(cmd.Cmd):
def __init__(self, ctx):
cmd.Cmd.__init__(self)
self.ctx = ctx
def default(self, line):
subcommand = cli.commands.get(line)
if subcommand:
self.ctx.invoke(subcommand)
else:
return cmd.Cmd.default(self, line)
#click.group(invoke_without_command=True)
#click.pass_context
def cli(ctx):
if ctx.invoked_subcommand is None:
repl = REPL(ctx)
repl.cmdloop()
#cli.command()
def a():
"""The `a` command prints an 'a'."""
print "a"
#cli.command()
def b():
"""The `b` command prints a 'b'."""
print "b"
if __name__ == "__main__":
cli()
There is now a library called click_repl that does most of the work for you. Thought I'd share my efforts in getting this to work.
The one difficulty is that you have to make a specific command the repl command, but we can repurpose #fpbhb's approach to allow calling that command by default if another one isn't provided.
This is a fully working example that supports all click options, with command history, as well as being able to call commands directly without entering the REPL:
import click
import click_repl
import os
from prompt_toolkit.history import FileHistory
#click.group(invoke_without_command=True)
#click.pass_context
def cli(ctx):
"""Pleasantries CLI"""
if ctx.invoked_subcommand is None:
ctx.invoke(repl)
#cli.command()
#click.option('--name', default='world')
def hello(name):
"""Say hello"""
click.echo('Hello, {}!'.format(name))
#cli.command()
#click.option('--name', default='moon')
def goodnight(name):
"""Say goodnight"""
click.echo('Goodnight, {}.'.format(name))
#cli.command()
def repl():
"""Start an interactive session"""
prompt_kwargs = {
'history': FileHistory(os.path.expanduser('~/.repl_history'))
}
click_repl.repl(click.get_current_context(), prompt_kwargs=prompt_kwargs)
if __name__ == '__main__':
cli(obj={})
Here's what it looks like to use the REPL:
$ python pleasantries.py
> hello
Hello, world!
> goodnight --name fpbhb
Goodnight, fpbhb.
And to use the command line subcommands directly:
$ python pleasntries.py goodnight
Goodnight, moon.
I know this is super old, but I've been working on fpbhb's solution to support options as well. I'm sure this could use some more work, but here is a basic example of how it could be done:
import click
import cmd
import sys
from click import BaseCommand, UsageError
class REPL(cmd.Cmd):
def __init__(self, ctx):
cmd.Cmd.__init__(self)
self.ctx = ctx
def default(self, line):
subcommand = line.split()[0]
args = line.split()[1:]
subcommand = cli.commands.get(subcommand)
if subcommand:
try:
subcommand.parse_args(self.ctx, args)
self.ctx.forward(subcommand)
except UsageError as e:
print(e.format_message())
else:
return cmd.Cmd.default(self, line)
#click.group(invoke_without_command=True)
#click.pass_context
def cli(ctx):
if ctx.invoked_subcommand is None:
repl = REPL(ctx)
repl.cmdloop()
#cli.command()
#click.option('--foo', required=True)
def a(foo):
print("a")
print(foo)
return 'banana'
#cli.command()
#click.option('--foo', required=True)
def b(foo):
print("b")
print(foo)
if __name__ == "__main__":
cli()
I was trying to do something similar to the OP, but with additional options / nested sub-sub-commands. The first answer using the builtin cmd module did not work in my case; maybe with some more fiddling.. But I did just run across click-shell. Haven't had a chance to test it extensively, but so far, it seems to work exactly as expected.