I am working on a project where it would be very handy if I could mock out urlopen during testing. Someone pointed out to me that this is possible (and easy) by mocking out an opener and using urllib2.install_opener.
However, I'm concerned because of this note in the documentation:
Install an OpenerDirector instance as the default global opener.
Doesn't this mean that my program could unexpectedly break if other code that I rely on is using urlopen?
The implications are exactly what you'd expect. All subsequent calls to urllib2.urlopen in your program, until your either exit or call install_opener again, will use your opener.
Whether that's "dangerous" depends on your use case. If there are other parts of your code that are using urllib2.open and you don't want them mocked, then yes, this is a bad idea, because they will be mocked.
In that case, you'll have to get the to-be-mocked code to call my_opener.open instead of urllib2.open. If you design your code to be tested, this should be easy. If you need to monkeypatch code after the fact, it's a little trickier, but there are all kinds of possibilities. For example, if you want to mock all calls in a given module, just replace foomodule.urllib2 = my_opener and set my_opener.urlopen = my_opener.open.
Related
i have a function that has the following code in it
ses = boto3.Session()
as_role = aws_assume_role_lib.assume_role(session, "some_role")
i'm trying to unit test it and i need to mock those two calls.
What's the best way to do so?
Without more information, there's no way to answer your question. Any direct answer would require a lot more information. What is it that you want to test? It certainly can't be the code you're showing us because if you mock out both of these calls, there's nothing left to test.
Mocking involves injecting stand-ins for specific test cases such that for those test cases, the mocks return predetermined values without calling the real routines. So what do you want those values to be? The mocks could also have side effects, or change their behavior based on external state, but you want to try to avoid either of those situations. Mocks are usually functional...returning a specific output for each set of specific explicit inputs, and neither modifying or being affected by implicit (external) state.
I assume that ses should be session so that the result of the first call is passed to the second call. In this case, neither of these calls takes varying data, so you can mock the result of the two calls by just assigning a static value to as_role...whatever value your later code wants to see. Since there is only one set of inputs, there should only be one possible output.
It would be more complicated if you need the mocks to change their behavior based on external state. How you might mock in that case is entirely dependent on the external states that are possible, where they come from, and how they affects the value of as_role. Likewise, if you need your mocks to cause a change in external state, like modifying global variables, you'd need to specify that, and that might affect how you'd choose to mock.
In short, you should define a spec describing how you want your mocks to act under a restricted set of particular conditions. Having that spec will let you begin to decide how to implement them.
If you're just asking how to generally build mocks in Python, check out one or more of the existing Python mocking frameworks, like mock, flexmock, mox, Mocker, etc. A quick Google pointed me to this, which might be helpful: https://garybernhardt.github.io/python-mock-comparison/. Asking for library recommendations is against SO policy as it involves mostly personal opinions, so if you're asking for that, you should look elsewhere.
I was wandering in the source code of the fabulous python-chess library when I saw the following code:
def _reset_board(self):
# code...
def reset_board(self):
self._reset_board()
The reset_board() function only does one thing, call its private counterpart. Is there a reason behind this? Wouldn't putting the code directly in the private function be faster due to python not have to resolve the name _reset_board()?
_reset_board exists so it can be called from both reset_board and __init__. __init__ can't call self.reset_board, because that method is overridden in subclasses, and __init__ wants to call the specific _reset_board implementation from its own class. (Subclass reset_board implementations may depend on initialization that hasn't happened yet, among other problems.)
I agree with you, here this _reset_board is not necessary. The author probably did some wrapping/cleaning in the reset_board method before, removed it, and didn't take the time to remove the _reset_board. Or maybe he plans to add some wrapping/cleaning in there in the future.
Some project also may generate Documentation automatically based on the code, and may skip functions/method that start with a _, and he may not want to publish any documentation for this function, but being open source, it's probably not the real reason.
Recently, Ned Batchelder during his talk at PyCon 2016 noted:
If you are using unittest to write your tests, definitely use
addCleanup, it's much better than tearDown.
Up until now, I've never used addCleanup() and got used to setUp()/tearDown() pair of methods for test "set up" and "tear down" phases.
Why should I switch to addCleanup() instead of tearDown()?
It was also recently discussed in the Python unittest with Robert Collins podcast.
Per the addCleanup doc string:
Cleanup items are called even if setUp fails (unlike tearDown)
addCleanup can be used to register multiple functions, so you could use
separate functions for each resource you wish to clean up. That would allow your
code to be a bit more reusable/modular.
addCleanup() methods will run even if one of them fails, and will run even if setUp() fails. You should also consider using pytest.
Another good thing about addCleanup is that it just works as you'd expect.
For example, if you call it in a setUp function, then all test methods will call the cleanup function in the end.
If you call it in a test method, only this method calls the cleanup function.
I'm writing a Python wrapper for an authenticated RESTful API. I'm writing up my test suite right now (Also first time test-writer here) but have a few questions:
1.a) How can I make a call, but not have to hardcode credentials into the tests since I'll be throwing it on Github?
1.b) I kind of know about mocking, but have no idea how to go about it. Would this allow me to not have to call the actual service? What would be the best way to go about this?
2) What do I test for - Just ensure that my methods are passing certains items in the dictionary?
3) Any best practices I should be following here?
Hey TJ if you can show me an example of one function that you are writing (code under test, not the test code) then I can give you an example test.
Generally though:
1.a You would mock the call to the external api, you are not trying to test whether their authentication mechanism, or your internet connection is working. You are just trying to test that you are calling their api with the correct signature.
1.b Mocking in Python is relatively straight forward. I generally use the mocking library written by Michael Foord. pip install mock will get you started. Then you can do things like
import unittest
from mock import call, patch
from my_module import wrapper_func
class ExternalApiTest(unittest.TestCase):
#patch('my_module.api_func')
def test_external_api_call(self, mocked_api_func):
response = wrapper_func('user', 'pass')
self.assertTrue(mocked_api_func.called)
self.assertEqual(
mocked_api_func.call_args_list,
[call('user', 'pass')]
)
self.assertEqual(mocked_api_func.return_value, response)
In this example we are replacing the api_func inside my_module with a mock object. The mock object records what has been done to it. It's important to remember where to patch. You don't patch the location you imported the object from. You patch it in the location that you will be using it.
You test that your code is doing the correct thing with a given input. Testing pure functions (pure in the functional programming sense) is pretty simple. You assert that given a input a, this function returns output b. It gets a bit trickier when your functions have lots of side effects.
If you are finding it too hard or complicated to test a certain functiob/method it can mean that it's a badly written piece of code. Try breaking it up into testable chunks and rather than passing objects into functions try to pass primitives where possible.
I'm currently writing a set of unit tests for a Python microblogging library, and following advice received here have begun to use mock objects to return data as if from the service (identi.ca in this case).
However, surely by mocking httplib2 - the module I am using to request data - I am tying the unit tests to a specific implementation of my library, and removing the ability for them to function after refactoring (which is obviously one primary benefit of unit testing in the firt place).
Is there a best of both worlds scenario? The only one I can think of is to set up a microblogging server to use only for testing, but this would clearly be a large amount of work.
You are right that if you refactor your library to use something other than httplib2, then your unit tests will break. That isn't such a horrible dependency, since when that time comes it will be a simple matter to change your tests to mock out the new library.
If you want to avoid that, then write a very minimal wrapper around httplib2, and your tests can mock that. Then if you ever shift away from httplib2, you only have to change your wrapper. But notice the number of lines you have to change is the same either way, all that changes is whether they are in "test code" or "non-test code".
Not sure what your problem is. The mock class is part of the tests, conceptually at least. It is ok for the tests to depend on particular behaviour of the mock objects that they inject into the code being tested. Of course the injection itself should be shared across unit tests, so that it is easy to change the mockup implementation.