I've been thinking about switching from nose to behave for testing (mocha/chai etc have spoiled me). So far so good, but I can't seem to figure out any way of testing for exceptions besides:
#then("It throws a KeyError exception")
def step_impl(context):
try:
konfigure.load_env_mapping("baz", context.configs)
except KeyError, e:
assert (e.message == "No baz configuration found")
With nose I can annotate a test with
#raises(KeyError)
I can't find anything like this in behave (not in the source, not in the examples, not here). It sure would be grand to be able to specify exceptions that might be thrown in the scenario outlines.
Anyone been down this path?
I'm pretty new to BDD myself, but generally, the idea would be that the tests document what behaves the client can expect - not the step implementations. So I'd expect the canonical way to test this would be something like:
When I try to load config baz
Then it throws a KeyError with message "No baz configuration found"
With steps defined like:
#when('...')
def step(context):
try:
# do some loading here
context.exc = None
except Exception, e:
context.exc = e
#then('it throws a {type} with message "{msg}"')
def step(context, type, msg):
assert isinstance(context.exc, eval(type)), "Invalid exception - expected " + type
assert context.exc.message == msg, "Invalid message - expected " + msg
If that's a common pattern, you could just write your own decorator:
def catch_all(func):
def wrapper(context, *args, **kwargs):
try:
func(context, *args, **kwargs)
context.exc = None
except Exception, e:
context.exc = e
return wrapper
#when('... ...')
#catch_all
def step(context):
# do some loading here - same as before
This try/catch approach by Barry works, but I see some issues:
Adding a try/except to your steps means that errors will be hidden.
Adding an extra decorator is inelegant. I would like my decorator to be a modified #where
My suggestion is to
have the expect exception before the failing statement
in the try/catch, raise if the error was not expected
in the after_scenario, raise error if expected error not found.
use the modified given/when/then everywhere
Code:
def given(regexp):
return _wrapped_step(behave.given, regexp) #pylint: disable=no-member
def then(regexp):
return _wrapped_step(behave.then, regexp) #pylint: disable=no-member
def when(regexp):
return _wrapped_step(behave.when, regexp) #pylint: disable=no-member
def _wrapped_step(step_function, regexp):
def wrapper(func):
"""
This corresponds to, for step_function=given
#given(regexp)
#accept_expected_exception
def a_given_step_function(context, ...
"""
return step_function(regexp)(_accept_expected_exception(func))
return wrapper
def _accept_expected_exception(func):
"""
If an error is expected, check if it matches the error.
Otherwise raise it again.
"""
def wrapper(context, *args, **kwargs):
try:
func(context, *args, **kwargs)
except Exception, e: #pylint: disable=W0703
expected_fail = context.expected_fail
# Reset expected fail, only try matching once.
context.expected_fail = None
if expected_fail:
expected_fail.assert_exception(e)
else:
raise
return wrapper
class ErrorExpected(object):
def __init__(self, message):
self.message = message
def get_message_from_exception(self, exception):
return str(exception)
def assert_exception(self, exception):
actual_msg = self.get_message_from_exception(exception)
assert self.message == actual_msg, self.failmessage(exception)
def failmessage(self, exception):
msg = "Not getting expected error: {0}\nInstead got{1}"
msg = msg.format(self.message, self.get_message_from_exception(exception))
return msg
#given('the next step shall fail with')
def expect_fail(context):
if context.expected_fail:
msg = 'Already expecting failure:\n {0}'.format(context.expected_fail.message)
context.expected_fail = None
util.show_gherkin_error(msg)
context.expected_fail = ErrorExpected(context.text)
I import my modified given/then/when instead of behave, and add to my environment.py initiating context.expected fail before scenario and checking it after:
def after_scenario(context, scenario):
if context.expected_fail:
msg = "Expected failure not found: %s" % (context.expected_fail.message)
util.show_gherkin_error(msg)
The try / except approach you show is actually completely correct because it shows the way that you would actually use the code in real life. However, there's a reason that you don't completely like it. It leads to ugly problems with things like the following:
Scenario: correct password accepted
Given that I have a correct password
When I attempt to log in
Then I should get a prompt
Scenario: incorrect password rejected
Given that I have an incorrect password
When I attempt to log in
Then I should get an exception
If I write the step definition without try/except then the second scenario will fail. If I write it with try/except then the first scenario risks hiding an exception, especially if the exception happens after the prompt has already been printed.
Instead those scenarios should, IMHO, be written as something like
Scenario: correct password accepted
Given that I have a correct password
When I log in
Then I should get a prompt
Scenario: correct password accepted
Given that I have a correct password
When I try to log in
Then I should get an exception
The "I log in" step should not use try; The "I try to log in" matches neatly to try and gives away the fact that there might not be success.
Then there comes the question about code reuse between the two almost, but not quite identical steps. Probably we don't want to have two functions which both login. Apart from simply having a common other function you call, you could also do something like this near the end of your step file.
#when(u'{who} try to {what}')
def step_impl(context):
try:
context.execute_steps("when" + who + " " + what)
context.exception=None
except Exception as e:
context.exception=e
This will automatically convert all steps containing the word "try to" into steps with the same name but with try to deleted and then protect them with a try/except.
There are some questions about when you actually should deal with exceptions in BDD since they aren't user visible. It's not part of the answer to this question though so I've put them in a separate posting.
Behave is not in the assertion matcher business. Therefore, it does not provide a solution for this. There are already enough Python packages that solve this problem.
SEE ALSO: behave.example: Select an assertion matcher library
Related
I have the strangest problem I have ever met in my life.
I have a part of my code that looks like this:
class AzureDevOpsServiceError(Exception):
pass
skip = ["auto"]
def retrieve_results():
print(variable_not_defined)
... # some useful implementation
if not "results" in skip:
try:
print("before")
retrieve_results()
print("after")
except AzureDevOpsServiceError as e:
print(f"Error raised: {e}")
Obviously, this shall raise an error because variable_not_defined is, well, not defined.
However, for some strange reasons, the code executes correctly and prints
before
after
I have tried to call the function with an argument (retrieve_results(1234)) or adding an argument in the function (def retrieve_results(arg1) and retrieve_results()): both modifications will trigger an exception, so obviously the function is called.
Anyone has got a similar issue and knows what happens?
FYI: this is actually what my implementation looks like:
from azure.devops.exceptions import AzureDevOpsServiceError
import logging
def _retrieve_manual_results(connect: Connectivity, data: DataForPickle) -> None:
"""Retrieve the list of Test Results"""
print("G" + ggggggggggggggggggggggggggggggggggggg)
logger = connect.logger
data.run_in_progress = [165644]
if __name__ == "__main__":
p = ...
connect = ...
data = ...
if not "results" in p.options.skip:
try:
print("........B.........")
_retrieve_manual_results(connect, data)
print("........A.........")
except AzureDevOpsServiceError as e:
logging.error(f"E004: Error while retrieving Test Results: {e}")
logging.debug("More details below...", exc_info=True)
As highlighted by #gmds, it was a problem of cache.
Deleting the .pyc file didn't do much.
However, I have found a solution:
Renaming the function (e.g. adding _)
Running the program
Renaming back (i.e. removing _ in the previous example)
Now, the issue is solved.
If anyone knows what is going behind the scene I am very interested.
I am trying to implement the retry decorator on a serial query. A general idea of my code is shown below. I am struggling to get it to retry when the decorator is one method up in the hierarchy. How can I have the method be retried when it's one method up from the method that throws the exception?
One complication that is frustrating is my increment time per retry depends on the actual command. Some commands require more time than others. That's why I have the extra_time_per_retry passed in, and couldn't implement the retry decorator using the traditional #retry style.
FYI the _serial is created in the class on init via pySerial.
I got it to work with the retry decorator directly above the method that throws the exception. I would like it to be two above to keep my code clean.
I have tried feeding the retry decorator the exact exception type, but couldn't get it to work.
def _query_with_retries(self, cmd, extra_time_per_retry):
_retriable_query = retry(stop_max_attempt_number=5,
wait_incrementing_start=self._serial.timeout + extra_time_per_retry,
wait_incrementing_increment=10)(self._query)
return _retriable_query(cmd)
def _query(self, cmd):
cmd_msg = cmd + '\r'
self._serial.reset_input_buffer()
self._serial.reset_output_buffer()
self._serial.write(cmd_msg)
return self._readlines()
def _readlines(self):
response_str = self._serial.read_until('\r', 256) # Max 256 bytes
# Parse response here, if a bad one set bad_response = true
if bad_response:
raise ResponseError("Response had custom error xyz")
I guess you could pack your call to _readlines() into an exception handling block, reraising the error:
python 3.x
#retry
def _query(self, cmd):
cmd_msg = cmd + '\r'
self._serial.reset_input_buffer()
self._serial.reset_output_buffer()
self._serial.write(cmd_msg)
try:
answer = self._readlines()
except Exception as e:
raise e
return answer
python 2.x
#retry
def _query(self, cmd):
cmd_msg = cmd + '\r'
self._serial.reset_input_buffer()
self._serial.reset_output_buffer()
self._serial.write(cmd_msg)
try:
answer = self._readlines()
except Exception:
t, v, tb = sys.exc_info()
raise t, v, tb
return answer
This way, you catch the exception directly when it occurs, and raise it inside the method which will be retried. I am not sure whether this declutters enough for you, however it should work.
Some might complain about using a blank except Exception, however since I am reraising it always immediately, I do not see any harm.
This demo function I want to test is pretty straight forward.
def is_email_deliverable(email):
try:
return external.verify(email)
except Exception:
logger.error("External failed failed")
return False
This function uses an external service which I want to mock out.
But I can't figure out how to throw an exception from external.verify(email) i.e. how to force the except clause to be executed.
My attempt:
#patch.object(other_module, 'external')
def test_is_email_deliverable(patched_external):
def my_side_effect(email):
raise Exception("Test")
patched_external.verify.side_effects = my_side_effect
# Or,
# patched_external.verify.side_effects = Exception("Test")
# Or,
# patched_external.verify.side_effects = Mock(side_effect=Exception("Test"))
assert is_email_deliverable("some_mail#domain.com") == False
This question claims to have the answer, but didn't work for me.
You have used side_effects instead of side_effect.
Its something like this
#patch.object(Class, "attribute")
def foo(attribute):
attribute.side_effect = Exception()
# Other things can go here
BTW, its not good approach to catch all the Exception and handle according to it.
You can set the side_effect value to None.
Hi im currently doing a program like this.
class MyError(Exception):
def __init__(self, text = "Correct")
self.text = text
def __str__(self):
return (self.kod)
class Atom(self):
.
.
.
try:
function()
else:
raise MyError("Incorrect use of function")
def main():
try:
a = Atom()
except:
# Here i want to print the error that was raised
What I think I understand is that the error is raised in an object created in Atom().
But I want to send it to my main program and do the print of the error MyError there.
Is it possible to do this and how should I write it so that the correct text of exception is printed since i will have several different error messages.
If i come to the except statement I would want to get the message "Incorrect use of function" printed.
It seems that you're pretty close:
class MyError(Exception):
def __init__(self, text = "Correct")
self.text = text
def __str__(self):
return (self.kod)
class Atom(self):
.
.
.
try:
function()
except: # try: ... else: raise ... seems a bit funky to me.
raise MyError("Incorrect use of function")
def main():
try:
a = Atom()
except Exception as err: # Possibly `except MyError as err` to be more specific
print err
The trick is that when you catch the error, you want to bind the exception instance to a name using the as clause. Then you can print it, look at it's attributes, re-raise or pretty much do anything you choose with it.
Please note that this code still isn't "clean". Generally, you want to limit exception handling as much as possible -- only catch exceptions that expect to see and that you know how to handle. Otherwise, you can sometimes mask hard to find bugs in your code. Because of this:
try:
do_something()
except:
...
is discouraged (it catches all sorts of things like KeyboardInterrupt and SystemExit) ... Instead:
try:
do_something()
except ExceptionIKnowHowToHandle:
...
is advised.
Firstly, never do a blank except. That will catch all errors, including things like KeyboardInterrupt - so you won't be able to ctrl-c out of your program. Here you should just catch MyError.
The except clause also allows you to assign the actual exception to a variable, which you can then print or do anything else with. So you can do:
try:
...
except MyError as e:
print e.text
I'm having some problems implementing an exception system in my program.
I found somewhere the following piece of code that I am trying to use for my program:
class InvalidProgramStateException(Exception):
def __init__(self, expr, msg):
self.expr = expr
self.msg = msg
I think msg must be a string message to be shown, but how do I fill the "expr" when I want to raise this exception? Do I have to write it by hand?
raise InvalidProgramStateException(what_here?, "there was an error")
Your custom exceptions don't actually need to take parameters at all. If you haven't got any particular error message or state to encapsulate in the Exception, this will work just fine:
class MyException(Exception):
pass
This would allow your program to catch cases of this exception by type:
try:
raise MyException()
except MyException:
print "Doing something with MyException"
except:
print "Some other error occurred... handling it differently"
If you want the Exception to have some meaningful string representation, or have properties that would provide your application greater details on what went wrong, that's when you pass additional arguments to the constructor. The number, name and type of these arguments is not really pre-defined by Python... they can be anything. Just be sure to provide a custom __str__ or __unicode__ method so you can provide a meaningful text depiction:
class MyException(Exception):
def __init__(self, msg):
self.msg = msg
def __str__(self):
return "MyException with %s" % self.msg
In the case of the example you're quoting, the expr and msg parameters are specific to the fictional case of the example. A contrived scenario for how these might be used is:
def do_something(expr):
if 'foo' in expr:
raise InvalidProgramStateException(expr, "We don't allow foos here")
return 5
user_input = 'foo bar'
try:
do_something(user_input)
except InvalidProgramStateException, e:
print "%s (using expression %s)" % (e.msg, e.expr)
Since it doesn't appear that your application requires it, just drop the parameters you don't require.