I have a function with several points of failure:
def setup_foo(creds):
"""
Creates a foo instance with which we can leverage the Foo virtualization
platform.
:param creds: A dictionary containing the authorization url, username,
password, and version associated with the Foo
cluster.
:type creds: dict
"""
try:
foo = Foo(version=creds['VERSION'],
username=creds['USERNAME'],
password=creds['PASSWORD'],
auth_url=creds['AUTH_URL'])
foo.authenticate()
return foo
except (OSError, NotFound, ClientException) as e:
raise UnreachableEndpoint("Couldn't find auth_url {0}".format(creds['AUTH_URL']))
except Unauthorized as e:
raise UnauthorizedUser("Wrong username or password.")
except UnsupportedVersion as e:
raise Unsupported("We only support Foo API with major version 2")
and I'd like to test that all the relevant exceptions are caught (albeit not handled well currently).
I have an initial test case that passes:
def test_setup_foo_failing_auth_url_endpoint_does_not_exist(self):
dummy_creds = {
'AUTH_URL' : 'http://bogus.example.com/v2.0',
'USERNAME' : '', #intentionally blank.
'PASSWORD' : '', #intentionally blank.
'VERSION' : 2
}
with self.assertRaises(UnreachableEndpoint):
foo = osu.setup_foo(dummy_creds)
but how can I make my test framework believe that the AUTH_URL is actually a valid/reachable URL?
I've created a mock class for Foo:
class MockFoo(Foo):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
and my thought is mock the call to setup_foo and remove the side effect of raising an UnreachableEndpoint exception. I know how to add side-effects to a Mock with unittest.mock, but how can I remove them?
Assuming your exceptions are being raised from foo.authenticate(), what you want to realize here is that it does not necessarily matter whether the data is in fact really valid in your tests. What you are trying to say really is this:
When this external method raises with something, my code should behave accordingly based on that something.
So, with that in mind, what you want to do is have different test methods where you pass what should be valid data, and have your code react accordingly. The data itself does not matter, but it provides a documented way of showing how the code should behave with data that is passed in that way.
Ultimately, you should not care how the nova client handles the data you give it (nova client is tested, and you should not care about it). What you care about is what it spits back at you and how you want to handle it, regardless of what you gave it.
In other words, for the sake of your tests, you can actually pass a dummy url as:
"this_is_a_dummy_url_that_works"
For the sake of your tests, you can let that pass, because in your mock, you will raise accordingly.
For example. What you should be doing here is actually mocking out Client from novaclient. With that mock in hand, you can now manipulate whatever call within novaclient so you can properly test your code.
This actually brings us to the root of your problem. Your first exception is catching the following:
except (OSError, NotFound, ClientException)
The problem here, is that you are now catching ClientException. Almost every exception in novaclient inherits from ClientException, so no matter what you try to test beyond that exception line, you will never reach those exceptions. You have two options here. Catch ClientException, and just raise a custom exception, or, remote ClientException, and be more explicit (like you already are).
So, let us go with removing ClientException and set up our example accordingly.
So, in your real code, you should be now setting your first exception line as:
except (OSError, NotFound) as e:
Furthermore, the next problem you have is that you are not mocking properly. You are supposed to mock with respect to where you are testing. So, if your setup_nova method is in a module called your_nova_module. It is with respect to that, that you are supposed to mock. The example below illustrates all this.
#patch("your_nova_module.Client", return_value=Mock())
def test_setup_nova_failing_unauthorized_user(self, mock_client):
dummy_creds = {
'AUTH_URL': 'this_url_is_valid',
'USERNAME': 'my_bad_user. this should fail',
'PASSWORD': 'bad_pass_but_it_does_not_matter_what_this_is',
'VERSION': '2.1',
'PROJECT_ID': 'does_not_matter'
}
mock_nova_client = mock_client.return_value
mock_nova_client.authenticate.side_effect = Unauthorized(401)
with self.assertRaises(UnauthorizedUser):
setup_nova(dummy_creds)
So, the main idea with the example above, is that it does not matter what data you are passing. What really matters is that you are wanting to know how your code will react when an external method raises.
So, our goal here is to actually raise something that will get your second exception handler to be tested: Unauthorized
This code was tested against the code you posted in your question. The only modifications were made were with module names to reflect my environment.
If you wish to mock out http servers from bogus urls, I suggest you check out HTTPretty. It mocks out urls at a socket level so it can trick most Python HTTP libraries that it's a valid url.
I suggest the following setup for your unittest:
class FooTest(unittest.TestCase):
def setUp(self):
httpretty.register_uri(httpretty.GET, "http://bogus.example.com/v2.0",
body='[{"response": "Valid"}]',
content_type="application/json")
#httpretty.activate
def test_test_case(self):
resp = requests.get("http://bogus.example.com/v2.0")
self.assertEquals(resp.status_code, 200)
Note that the mock will only apply to stacks that are decorated with http.activate decorator, so it won't leak to other places in your code that you don't want to mock. Hope that makes sense.
Related
I am searching for a way to change the printable output of an Exception to a silly message in order to learn more about python internals (and mess with a friend ;), so far without success.
Consider the following code
try:
x # is not defined
except NameError as exc:
print(exc)
The code shall output name 'x' is not defined
I would like the change that output to the name 'x' you suggested is not yet defined, my lord. Improve your coding skills.
So far, I understood that you can't change __builtins__ because they're "baked in" as C code, unless:
You use forbiddenfruit.curse method which adds / changes properties of any object
You manually override the dictionnaries of an object
I've tried both solutions, but without success:
forbiddenfruit solution:
from forbiddenfruit import curse
curse(BaseException, 'repr', lambda self: print("Test message for repr"))
curse(BaseException, 'str', lambda self: print("Test message for str"))
try:
x
except NameError as exc:
print(exc.str()) # Works, shows test message
print(exc.repr()) # Works, shows test message
print(repr(exc)) # Does not work, shows real message
print(str(exc)) # Does not work, shows real message
print(exc) # Does not work, shows real message
Dictionnary overriding solution:
import gc
underlying_dict = gc.get_referents(BaseException.__dict__)[0]
underlying_dict["__repr__"] = lambda self: print("test message for repr")
underlying_dict["__str__"] = lambda self: print("test message for str")
underlying_dict["args"] = 'I am an argument list'
try:
x
except NameError as exc:
print(exc.__str__()) # Works, shows test message
print(exc.__repr__()) # Works, shows test message
print(repr(exc)) # Does not work, shows real message
print(str(exc)) # Does not work, shows real message
print(exc) # Does not work, shows real message
AFAIK, using print(exc) should rely on either __repr__ or __str__, but it seems like the print function uses something else, which I cannot find even when reading all properties of BaseException via print(dir(BaseException)).
Could anyone give me an insight of what print uses in this case please ?
[EDIT]
To add a bit more context:
The problem I'm trying to solve began as a joke to mess with a programmer friend, but now became a challenge for me to understand more of python's internals.
There's no real business problem I'm trying to solve, I just want to get deeper understanding of things in Python.
I'm quite puzzled that print(exc) won't make use of BaseException.__repr__ or __str__ actually.
[/EDIT]
Intro
I'd go with a more critical approach on why you'd even want to do what you want to do.
Python provides you with an ability to handle specific exceptions. That means if you had a business problem, you'd use a particular exception class and provide a custom message for that specific case. Now, remember this paragraph and let's move on, I'll refer to this later.
TL;DR
Now, let's go top-down:
Catching all kinds of errors with except Exception is generally not a good idea if want you catch let's say a variable name error. You'd use except NameError instead. There's really not much you'd add to it that's why it had a default message that perfectly described the issue. So it's assumed you'd use it as it's given. These are called concrete exceptions.
Now, with your specific case notice the alias as exc. By using the alias you can access arguments passed to the exception object, including the default message.
try:
x # is not defined
except NameError as exc:
print(exc.args)
Run that code (I put it in app.py) and you'll see:
$ python app.py
("name 'x' is not defined",)
These args are passed to the exception as a series (list, or in this case immutable list that is a tuple).
This leads to the idea of the possibility of easily passing arguments to exceptions' constructors (__init__). In your case "name 'x' is not defined" was passed as an argument.
You can use this to your advantage to solve your problem without much effort by just providing a custom message, like:
try:
x # is not defined
except NameError as exc:
your_custom_message = "the name 'x' you suggested is not yet defined, my lord. Improve your coding skills"
# Now, you can handle it based on your requirement:
# print(your_custom_message)
# print(NameError(your_custom_message))
# raise NameError(your_custom_message)
# raise NameError(your_custom_message) from exc
The output is now what you wanted to achieve.
$ python app.py
the name 'x' you suggested is not yet defined, my lord. Improve your coding skills
Remember the first paragraph when I said I'd refer to it later? I mentioned providing a custom message for a specific case. If you build your own library when you want to handle name errors to specific variables relevant to your product, you assume your users will use your code that might raise that NameError exception. They will most likely catch it with except Exception as exc or except NameError as exc. And when they do print(exc), they will see your message now.
Summary
I hope that makes sense to you, just provide a custom message and pass it as an argument to NameError or simply just print it. IMO, it's better to learn it right together with why you'd use what you use.
Errors like this are hard-coded into the interpreter (in the case of CPython, anyway, which is most likely what you are using). You will not be able to change the message printed from within Python itself.
The C source code that is executed when the CPython interpreter tries to look up a name can be found here: https://github.com/python/cpython/blob/master/Python/ceval.c#L2602. If you would want to change the error message printed when a name lookup fails, you would need to change this line in the same file:
#define NAME_ERROR_MSG \
"name '%.200s' is not defined"
Compiling the modified source code would yield a Python interpreter that prints your custom error message when encountering a name that is not defined.
I'll just explain the behaviour you described:
exc.__repr__()
This will just call your lambda function and return the expected string. Btw you should return the string, not print it in your lambda functions.
print(repr(exc))
Now, this is going a different route in CPython and you can see this in a GDB session, it's something like this:
Python/bltinmodule.c:builtin_repr will call Objects/object.c:PyObject_Repr - this function gets the PyObject *v as the only parameter that it will use to get and call a function that implements the built-in function repr(), BaseException_repr in this case. This function will format the error message based on a value from args structure field:
(gdb) p ((PyBaseExceptionObject *) self)->args
$188 = ("name 'x' is not defined",)
The args value is set in Python/ceval.c:format_exc_check_arg based on a NAME_ERROR_MSG macro set in the same file.
Update: Sun 8 Nov 20:19:26 UTC 2020
test.py:
import sys
import dis
def main():
try:
x
except NameError as exc:
tb = sys.exc_info()[2]
frame, i = tb.tb_frame, tb.tb_lasti
code = frame.f_code
arg = code.co_code[i + 1]
name = code.co_names[arg]
print(name)
if __name__ == '__main__':
main()
Test:
# python test.py
x
Note:
I would also recommend to watch this video from PyCon 2016.
I'm trying to understand why, how, and if to unit test methods that seem to return nothing. I've read in a couple other threads that:
The point of a unit test is to test something that the function does. If its not returning a value, then what is it actually doing?
unittest for none type in python?
In my example, I am using the XMLSigner and XMLVerifier from the sign_XML library.
def verify_xml(signed_xml: str, cert_file: str) -> None:
with open(cert_file, 'rb') as file:
cert = file.read()
with open(signed_xml, 'rb') as input_file:
input_data = input_file.read()
XMLVerifier().verify(input_data, x509_cert=cert)
I started looking up documentaion I found for SignXML. I read that verify():
class signxml.XMLVerifier Create a new XML Signature Verifier object,
which can be used to hold configuration information and verify
multiple pieces of data. verify(data, require_x509=True,
x509_cert=None, cert_subject_name=None, ca_pem_file=None,
ca_path=None, hmac_key=None, validate_schema=True, parser=None,
uri_resolver=None, id_attribute=None, expect_references=1)
Verify the
XML signature supplied in the data and return the XML node signed by
the signature, or raise an exception if the signature is not valid. By
default, this requires the signature to be generated using a valid
X.509 certificate.
This is my first time working with this and I'm confused even more now. So this apparently does return something.
What I've attempted
For another method which ends up calling verify_xml I've used #patch and just checked that the method I patched was called and with the correct arguments. This also seems like it's not the way to do it, but I didn't know how else to test it.
It feels weird doing something similar with the verify_xml method and just checking that it has been called once.
I've also tried self.assertIsNone... and that passes but that seems weird to me and not like it's a way one does this.
Could someone help me understand why, how, and if to unit test methods that seem to return nothing).
Thanks
to test verify_xml() is to test the Exception triggered by XMLVerifer().verify() if input parameters is not valid
There are a few types of exceptions you can tested.
from signxml import (XMLSigner, XMLVerifier, InvalidInput, InvalidSignature, InvalidCertificate, InvalidDigest)
class TestVerifyXML(unittest.TestCase):
def setUpCls(cls):
cls.signed_xml = from_magic()
cls.cert_file = from_magic2()
cls.ceft_file_bad = from_magic_bad()
def test_verify_xml(self):
# no Exception with correct xml
verify_xml(self.signed_xml, self.cert_file)
with self.assertRaises(InvalidSignature):
verify_xml(self.signed_xml, self.cert_file_bad)
I am relatively new to Python and want to use a assertRaises test to check for a ValidationError, which works ok. However, I have many ValidationErrors and I want to make sure the right one is returned. I figured I could pass something into assertRaises but it doesn't look like I can, so I figured I would just do an assertTrue and check the exception message. However, I don't know how to access it. Is this even a good way to approach this issue? thanks.
class DailyEntriesTests(TestCase):
def test_cant_have_ip_and_user(self):
u = createUser(False)
de = createDailyEntry(u, "1.1.1.1", 1)
with self.assertRaises(ValidationError) as cm:
de.full_clean()
# this line bombs - message doesn't exist. I also tried "error_code" like I saw in the documentation, but that doesn't work
print(cm.exception.message)
self.assertTrue(cm.exception.message.contains("Both"))
You can just use assertRaisesRegexp.
with self.assertRaisesRegexp(ValidationError, "Both"):
de.full_clean()
When you use it as a context manager the 2nd argument is a regular expression to search through the exception's string representation.
Since the question is related to Django, you could also use the assertRaisesMessage context manager when inheriting from django's TestCase.
from django.test import TestCase
class ExceptionTest(TestCase):
def test_call_raises_exception_with_custom_message(self):
with self.assertRaisesMessage(Exception, 'My custom message!'):
call_that_causes_exception()
Note: The assertRaisesMessage manager does an in lookup on the exceptions message: Say your exception raises "My custom message!", asserting for "custom message" passes. Bear this in mind especially if you have multiple (custom) exceptions with similar messages.
(E.g. two different exceptions raising "My custom message! Further details..." and "My custom message! No details." would both pass an assert for "My custom message!").
Nowadays you can use assertRaises as a context manager. This way you can capture the exception and inspect it later.
with self.assertRaises(SomeException) as cm:
do_something()
the_exception = cm.exception
self.assertEqual(the_exception.error_code, 3)
I'd like to have something like a custom error code/message database and use it when raising exceptions (in Python 3.4). So I did the following:
class RecipeError(Exception):
# Custom error codes
ERRBADFLAVORMIX = 1
ERRNOINGREDIENTS = ERRBADFLAVORMIX + 1
# Custom messages
ERRMSG = {ERRBADFLAVORMIX: "Bad flavor mix",
ERRNOINGREDIENTS: "No ingredients to mix"}
raise RecipeError(RecipeError.ERRMSG[RecipeError.ERRBADFLAVORMIX])
This works as expected, but the raise statement is just monstrous. Sure, I could have stored the values in a more compact way, but what I really want to know is: Can I just do something like raise RecipeError(code) and leave the work of getting the message to RecipeError?
Sure. Exception classes are just normal classes, so you can define your own __init__ that calls super appropriately:
class RecipeError(BaseException):
# existing stuff
def __init__(self, code):
super().__init__(self, RecipeError.ERRMSG[code])
You might also want to save the code:
class RecipeError(BaseException):
# existing stuff
def __init__(self, code):
msg = RecipeError.ERRMSG[code]
super().__init__(self, msg)
self.code, self.msg = code, msg
Take a look at the information stored in the standard library's exceptions (which are pretty decent in 3.4, although there are still more changes to come…) to see what kinds of things might be useful to stash.
Some side notes:
First, it may be better to use subclasses instead of error codes. For example, if someone wants to write code that catches an ERRBADFLAVORMIX but not an ERRNOINGREDIENTS, they have to do this:
try:
follow_recipe()
except RecipeError as e:
if e != RecipeError.ERRBADFLAVORMIX:
raise
print('Bad flavor, bad!')
Or, if you'd used subclasses:
try:
follow_recipe():
except BadFlavorRecipeError as e:
print('Bad flavor, bad!')
That's exactly why Python no longer has a monolithic OSError with an errno value that you have to switch on, and instead has separate subclasses like FileNotFoundError.
If you do want to use error codes, you might want to consider using an Enum, or maybe one of the fancier enum types on PyPI that make it easier to attach a custom string to each one.
You almost never want to inherit from BaseException, unless you're specifically trying to make sure your exception doesn't get caught.
This one's a structure design problem, I guess. Back for some advice.
To start: I'm writing a module. Hence the effort of making it as usable to potential developers as possible.
Inside an object (let's call it Swoosh) I have a method which, when called, may result in either success (a new object is returned -- for insight: it's an httplib.HTTPResponse) or failure (surprising, isn't it?).
I'm having trouble deciding how to handle failures. There are two main cases here:
user supplied data that was incorrect
data was okay, but user interaction will be needed () - I need to pass back to the user a string that he or she will need to use in some way.
In (1) I decided to raise ValueError() with an appropriate description.
In (2), as I need to actually pass a str back to the user.. I'm not sure about whether it would be best to just return a string and leave it to the user to check what the function returned (httplib.HTTPResponse or str) or raise a custom exception? Is passing data through raising exceptions a good idea? I don't think I've seen this done anywhere, but on the other hand - I haven't seen much.
What would you, as a developer, expect from an object/function like this?
Or perhaps you find the whole design ridiculous - let me know, I'll happily learn.
As much as I like the approach of handling both cases with specifically-typed exceptions, I'm going to offer a different approach in case it helps: callbacks.
Callbacks tend to work better if you're already using an asynchronous framework like Twisted, but that's not their only place. So you might have a method that takes a function for each outcome, like this:
def do_request(on_success, on_interaction_needed, on_failure):
"""
Submits the swoosh request, and awaits a response.
If no user interaction is needed, calls on_success with a
httplib.HTTPResponse object.
If user interaction is needed, on_interaction_needed is
called with a single string parameter.
If the request failed, a ValueError is passed to on_failure
"""
response = sumbit_request()
if response.is_fine():
on_success(response)
elif response.is_partial()
on_interaction_needed(response.message)
else:
on_failure(ValueError(response.message))
Being Python, there are a million ways to do this. You might not like passing an exception to a function, so you maybe just take a callback for the user input scenario. Also, you might pass the callbacks in to the Swoosh initialiser instead.
But there are drawbacks to this too, such as:
Carelessness may result in spaghetti code
You're allowing your caller to inject logic into your function (eg. exceptions raised in the callback will propagate out of Swoosh)
My example here is simple, your actual function might not be
As usual, careful consideration and good documentation should avoid these problems. In theory.
I think raising an exception may actually be a pretty good idea in this case. Squashing multiple signals into a single return value of a function isn't ideal in Python, due to duck typing. It's not very Pythonic; every time you need to do something like:
result = some_function(...)
if isinstance(result, TypeA):
do_something(result)
elif isinstance(result, TypeB):
do_something_else(result)
you should be thinking about whether it's really the best design (as you're doing).
In this case, if you implement a custom exception, then the code that calls your function can just treat the returned value as a HTTPResponse. Any path where the function is unable to return something its caller can treat that way is handled by throwing an exception.
Likewise, the code that catches the exception and prompts the user with the message doesn't have to worry about the exact type of the thing its getting. It just knows that it's been explicitly instructed (by the exception) to show something to the user.
If the user interaction case means the calling code has to show a prompt, get some input and them pass control back to your function, it might be ugly trying to handle that with an exception. Eg,
try:
Swoosh.method()
except UserInteraction, ex:
# do some user interaction stuff
# pass it back to Swoosh.method()?
# did Swoosh need to save some state from the last call?
except ValueError:
pass # whatever
If this user interaction is a normal part of the control flow, it might be cleaner to pass a user-interaction function into your method in the first place - then it can return a result to the Swoosh code. For example:
# in Swoosh
def method(self, userinteractor):
if more_info_needed:
more_info = userinteractor.prompt("more info")
...
ui = MyUserInteractor(self) # or other state
Swoosh.method(ui)
You can return a tuple of (httplib.HTTPResponse, str) with the str being optionally None.
Definitely raise an exception for 1).
If you don't like returning a tuple, you can also create a "response object" i.e. an instance of a new class ( lets say SomethingResponse ) that encapsulates the HTTPResponse with optional messages to the end-user( in the simplest case, just a str).