How to I can define the "limit context" of my tests?
I say this because of mocks, where my service has many other libs integrated, like: rabbit-mq, redis, etc, and instances from some one classes. Then, the greather part of time I'm writing test code, creating "complex" mocks to the service be possible to test.
Is possible defines this limit? Should be a possible way "no test" theses "expensive" method, and just test the "simple related method", that are simple to input the paramters, like a sum(a, b).
Is obvious that the more test coverage the better, but is many, many time writing some tests with a questionable useful result.
When is using TDD to well defined methods, like the sum(a,b), it is very useful, but when this method will receive instances or use instance from another integrated services, the mocks receive the greather part of attemption, not the "method objective".
Imagine this service/method to test:
class ClientService:
def ok_here_is_the_discount(self, some_args):
# Ok, receive the discount to thi signature, now is possible
# calcula the it, and response to the broker
self.calculate_user_discount_based_on_signature_type(some_args.discount_value)
def calculate_user_discount_based_on_signature_type(self, some_args):
# here will send the "result" to the broker
some_message_borker_publisher(
some_args.value - some_args.discount_signature
)
def request_the_signature_type_discount_in_another_service(self, some_args):
# Ok, I receive the client that has the signature type,
# now is need request what is the value to this signature
# in signature service
some_message_borker_publisher(
queue='siganture.service.what_discount_for_this_signature',
signature_type=some_args.client.singature_type
)
#Ok, this message goes to the broker, and the signature.service receive it
def method_that_receive_messages(self, some_args):
# Here we receives that want to calculate discount
# but just receive the client(dict/class/instance) with the signature type,
# we still dont know the value of theses discount, because when is responsible
# to manage the siganture, is the signature server
if some_args.message_type == 'please_calculate_discount':
self.request_the_signature_type_discount_in_another_service(some_args.client)
if some_args.message_type == 'message_from_signature_discount':
self.ok_here_is_the_discount(some_args.value)
1 - it receive a message 'please_calculate_discount' and call the method
self.request_the_signature_type_discount_in_another_service(some_args.client)
But it still not have the discount value, because this is signature service. (message to the broker)
2 - The supose that the signature server response to 'message_from_signature_discount', then it call the method:
self.ok_here_is_the_discount(some_args.value)
3 - ok, the method ok_here_is_the_discount receive discount and call method
calculate_user_discount_based_on_signature_type()
That now has the values to calculate to the discount and send these value to the broker.
Understand the complexity of these tests (TDD)? I need tests the "method_that_receive_messages" mocking all related nexted actions, or just test the related method, like "calculate_user_discount_based_on_signature_type"?
In this case is better uses a really broker to be possible test it?
Well it is easy to mock things in Python but it still comes with some overhead of course. What I have tended towards over many years now is to set up integration tests with mocks that tests the happy path through the system and then design the system with side effect free functions as much as possible and throw unit tests at them. Then you can start your TDDing with setting up the overall test for the happy path and then unit test particulars in easy to test functions.
It is still useless when the thing behind the mock changes though but it gives a great sense of security when refactoring.
The mock libraries in Python are fine but sometimes it is easier to just write your own and replace the real thing with patch. Pretty sweet actually.
Related
i have wrote an api wrapper which has around 70 endpoints. To test them all
https://paste.ubuntu.com/p/V6mkK4dSdh/
i have wrote this script without actually usign the unit testing module. Is it a good practice and what are the downsides when compared with unit testing? Really could use some comments.
This is not "testing". This is just checking that "they work".
However, in this context, "work" just means that they don't raise exceptions. That's a requirement, but not even close to "testing". The fact that you endpoint return some result does not mean that the result is correct.
Really testing would mean: When I call f(1, 2) I expect to get 5 as a response. So, you would need multiple tests, manually written, for each endpoint. Of course, that takes time...
So I'm trying to decide the way to plan and organize a testing suite for my python project but I have a doubt of when a unit test is no longer a unit test and I would love to have some feedback from the comunity.
If I understand correctly:
A Unit test test a minimal part of your code, be if a function/method that does one and only one simple thing, even if it has several use cases.
An Integration test tests that two or more units of you code that are executed under the same context, environment, etc (but trying to keep it to a minimmum of units per integration test) work well together and not only by themselves.
My doubt is: say I have a simple function that performs a HTTP request and returns the content of such request, be it HTML, JSON, etc, it doesn't matter, the fact is that the function is very, very simple but requests information from an external source, like:
import requests
def my_function(arg):
# do something very simple with `arg`, like removing spaces or the simplest thing you can imagine
return requests.get('http://www.google.com/' + arg).content
Now this is a very stupid example, but my doubt is:
Given that this function is requesting information from an external source, when you write a test for it, can you still consider such test a Unit test?
UPDATE: The test for my_function() would stub out calls to the external source so that it doesn't depend on network/db/filesystem/etc so that it's isolated. But the fact that the function that is being tested depends on external sources when running, for example, in production.
Thanks in advance!! :)
P.S.: Of course Maybe I'm not understading 100% de purposes of Unit and Integration testing, so, if I'm mistaken, please point me out where, I'll appreciate it.
Based on your update:
The test for my_function() would stub out calls to the external source
so that it doesn't depend on network/db/filesystem/etc so that it's
isolated. But the fact that the function that is being tested depends
on external sources when running, for example, in production.
As long as the external dependencies are stubbed out during your test then yes you can call it a Unit Test. It's a Unit Test based on how the unit under test behaves in your test suite rather than how the unit behaves in production.
Based on your original question:
Given that this function is requesting information from an external
source, when you write a test for it, can you still consider such test
a Unit test?
No, any test for code that touches or depends on things external to the unit under test is an integration test. This includes the existence of any web, file system, and database requests.
If your code is not 100% isolated from it's dependencies and not 100% reproducible without other components then it is an integration test.
For your example code to be properly unit tested you would need to mock out the call to google.com.
With the code calling google.com, your test would fail if google went down or you lost connection to the internet (ie the test is not 100% isolated). Your test would also fail if the behavior of google changed (ie the test is not 100% reproducable).
I don't think this is an integration test since it doesn't make use of different parts of your application. The function does really one thing and tests for it can be called unit.
On the other side, this particular function has an external dependency and you don't want to be dependent on the network in your tests. This is where mocking would really help a lot.
In other words, isolate the function and make unit tests for it.
Also, make integration tests that would use a more high-level approach and would test parts of your application which call my_function().
Whether or not your code is tested in isolation is not a distinguishing criterion whether your test is a unit test or not. For example, you would (except in very rare cases) not stub standard library functions like sin(x). But, if you don't stub sin(x) that does not make your test an integration test.
When is a test an integration test? A test is an integration test if your goal with the test is to find bugs on integration level. That means, with an integration test you want to find out whether the interaction between two (or more) components is based on the same assumptions on both (all) sides.
Mocking, however, is an orthogonal technique that can be combined with almost all kinds of tests. (In an integration test, however, you can not mock the partners of the interaction of which you want to test - but you can mock additional components).
Since mocking typically causes some effort, it must bring a benefit, like:
significantly speeding up your tests
testing although some software part is not ready yet or buggy
testing exceptional cases that are hard to set up in an integrated software
getting rid of nondeterministic behaviour like timing or randomness
...
If, however, mocking does not solve a real problem, you might be better off using the depended-on component directly.
I'm writing a Python wrapper for an authenticated RESTful API. I'm writing up my test suite right now (Also first time test-writer here) but have a few questions:
1.a) How can I make a call, but not have to hardcode credentials into the tests since I'll be throwing it on Github?
1.b) I kind of know about mocking, but have no idea how to go about it. Would this allow me to not have to call the actual service? What would be the best way to go about this?
2) What do I test for - Just ensure that my methods are passing certains items in the dictionary?
3) Any best practices I should be following here?
Hey TJ if you can show me an example of one function that you are writing (code under test, not the test code) then I can give you an example test.
Generally though:
1.a You would mock the call to the external api, you are not trying to test whether their authentication mechanism, or your internet connection is working. You are just trying to test that you are calling their api with the correct signature.
1.b Mocking in Python is relatively straight forward. I generally use the mocking library written by Michael Foord. pip install mock will get you started. Then you can do things like
import unittest
from mock import call, patch
from my_module import wrapper_func
class ExternalApiTest(unittest.TestCase):
#patch('my_module.api_func')
def test_external_api_call(self, mocked_api_func):
response = wrapper_func('user', 'pass')
self.assertTrue(mocked_api_func.called)
self.assertEqual(
mocked_api_func.call_args_list,
[call('user', 'pass')]
)
self.assertEqual(mocked_api_func.return_value, response)
In this example we are replacing the api_func inside my_module with a mock object. The mock object records what has been done to it. It's important to remember where to patch. You don't patch the location you imported the object from. You patch it in the location that you will be using it.
You test that your code is doing the correct thing with a given input. Testing pure functions (pure in the functional programming sense) is pretty simple. You assert that given a input a, this function returns output b. It gets a bit trickier when your functions have lots of side effects.
If you are finding it too hard or complicated to test a certain functiob/method it can mean that it's a badly written piece of code. Try breaking it up into testable chunks and rather than passing objects into functions try to pass primitives where possible.
Ok I am programing a way to interface with Grooveshark (http://grooveshark.com). Right now I have a class Grooveshark and several methods, one gets a session with the server, another gets a token that is based on the session and another is used to construct api calls to the server (and other methods use that). Right now I use it like so.... Note uses twisted and t.i.defer in twisted
g = Grooveshark()
d = g.get_session()
d.addCallback(lambda x: g.get_token())
## and then something like.... ##
g.search("Song")
I find this unpythonic and ugly sense even after initializing the class you have to call two methods first or else the other methods won't work. To solve this I am trying to get it so that the method that creates api calls takes care of the session and token. Currently those two methods (the session and token methods) set class variables and don't return anything (well None). So my question is, is there a common design used when interfacing with sites that require tokens and sessions? Also the token and session are retrieved from a server so I can't have them run in the init method (as it would either block or may not be done before a api call is made)
I find this unpythonic and ugly sense
even after initializing the class you
have to call two methods first or else
the other methods won't work.
If so, then why not put the get_session part in your class's __init__? If it always must be performed before anything else, that would seem to make sense. Of course, this means that calling the class will still return a yet-unusable instance -- that's kind of inevitable with asynchronous, event-drive programming... you don't "block until the instance is ready for use".
One possibility would be to pass the callback to perform as an argument to the class when you call it; a more Twisted-normal one would be to have Grooveshark be a function which returns a deferred (you'll add to the deferred the callback to perform, and call it with the instance as the argument when that instance is finally ready to be used).
I would highly recommend looking at the Facebook graph API. Just because you need sessions and some authentication doesn't mean you can build a clean REST API. Facebook uses OAuth to handle the authentication but there are other possibilities.
I'd like to know what would you propose as the best way to unit test aspect-oriented application features (well, perhaps that's not the best name, but it's the best I was able to come up with :-) ) such as logging or security?
These things are sort of omni-present in the application, so how to test them properly?
E.g. say that I'm writing a Cherrypy web server in Python. I can use a decorator to check whether the logged-in user has the permission to access a given page. But then I'd need to write a test for every page to see whether it works oK (or more like to see that I had not forgotten to check security perms for that page).
This could maybe (emphasis on maybe) be bearable if logging and/or security were implemented during the web server "normal business implementation". However, security and logging usually tend to be added to the app as an afterthough (or maybe that's just my experience, I'm usually given a server and then asked to implement security model :-) ).
Any thoughts on this are very welcome. I have currently 'solved' this issue by, well - not testing this at all.
Thanks.
IMHO, the way of testing users permissions to the pages depends on the design of your app and design of the framework you're using.
Generally, it's probably enough to cover your permission checker decorator with unit tests to make sure it always works as expected and then write a test that cycles through your 'views' (or whatever term cherrypy uses, haven't used it for a very long time) and just check if these functions are decorated with appropriate decorator.
As for logging it's not quite clear what you want test specifically. Anyway, why isn't it possible to stub the logging functionality and check what's going on there?
Well... let's see. In my opinion you are testing three different things here (sorry for the "Java AOP jargon"):
the features implemented by the interceptors (i.e. the methods that implement the functions activated at the cutpoints)
the coverage of the filters (i.e. whether the intended cutpoints are activated correctly or not)
the interaction between the cutpoints and the interceptors (with the side effects)
You are unit testing (strictly speaking) if you can handle these three layers separatedly. You can actually unit test the first; you can use a coverage tool and some skeleton crew application with mock objects to test the second; but the third is not exactly unit testing, so you may have to setup a test environment, design an end-to-end test and write some scripts to input data in your application and gather the results (if it was a web app you could use Selenium, for example).
My answer is for the specific example you give, not for the possible problems with bolted-on security. Decorators are just regular functions, and you can test them as such. For instance:
# ... inside included module ...
def require_admin(function):
if (admin):
return function
else:
return None
#require_admin
def func1(arg1, arg2):
pass
# ... inside unit test ...
def test_require_admin(self):
admin = False
f = lambda x: x
g = require_admin(f)
assert_equal(g, None)
admin = True
g = require_admin(f)
assert_equal(g, f)
Note: this is a really terrible way to do a security check, but it gets the point across about testing decorators. That's one of the nice things about Python: it's REALLY consistent. The following expressions are equivalent:
#g
def f(x):
return x
and
def f(x):
return x
f = g(f)