I have some celery task. I want to test it via unittest.
I'm doing something very similar to:
class TestMe(unittest.TestCase):
def test_celery_task(self):
self.assertRaises(ValueError, celery_task.apply, args)
what is strange for me:
this assert fails, because ValueError not raised, but during executing process I can see ValueError as a result of this celery task.
I'm not sure, but it looks like assert is checking faster than ValueError is rising.
Is it possible to check the result of executed celery task?
or how it may be tested?
That can't possibly work. When you enqueue a Celery task, all that happens is that you put a message into the queue for a separate process to pick up; it is that process that runs the task and, potentially, raises the exception.
If you want to check that the task itself raises ValueError, then you should call the task, not the delay function:
self.assertRaises(ValueError, celery_task, args)
I see 3 options here.
1) Try to call get() on apply(). Here is what you will get:
class TestMe(unittest.TestCase):
def test_celery_task(self):
self.assertRaises(ValueError, celery_task.apply().get(), args)
2) You can either enable eager mode by setting 'task_always_eager' to True, however it does not guarantee that your code will be able to catch up.
3) A better option would be to mock the celery tasks. From the point of unit testing it is not actually correct to test a unit of code with actual 'alive' part of systems like celery.
Here is a sample of code taken from celery testing documentation.
from pytest import raises
from celery.exceptions import Retry
# for python 2: use mock.patch from `pip install mock`.
from unittest.mock import patch
from proj.models import Product
from proj.tasks import send_order
class test_send_order:
#patch('proj.tasks.Product.order') # < patching Product in module above
def test_success(self, product_order):
product = Product.objects.create(
name='Foo',
)
send_order(product.pk, 3, Decimal(30.3))
product_order.assert_called_with(3, Decimal(30.3))
#patch('proj.tasks.Product.order')
#patch('proj.tasks.send_order.retry')
def test_failure(send_order_retry, product_order):
product = Product.objects.create(
name='Foo',
)
# set a side effect on the patched method
# so that it raises the error we want.
product_order.side_effect = OperationalError()
with raises(Retry):
send_order(product.pk, 3, Decimal(30.6))
Related
How would I unit test the following?
def sigterm_handler(signum, frame):
pid = os.getpid() # type: int
sys.exit(0)
signal.signal(signal.SIGTERM, sigterm_handler)
Should I mock and ensure mock is called?
I would write a test that runs your code in a subprocess which can check if you terminated successfully.
For example, let's say your question code lives in a module called signals.py. You can write a test wrapper module that looks like this:
test_signals_wrapper.py
from time import sleep
from sys import exit
# import this last to ensure it overrides any prior settings
import signals
while True:
sleep(1)
exit(1) # just in case the loop ends for other reasons
Now you can write a unit test that looks like this:
test_signals.py
from subprocess import run, TimeoutExpired
from sys import executable
def test_sigterm_handler():
try:
status = run([executable, '-m', 'test_signals_wrapper'], timeout=30)
except TimeoutExpired:
assert False, 'Did not trigger assertion in 30 seconds'
assert status.returncode == 0, f'Wrong return code: {status.returncode}'
This requires a bit of extra infrastructure for your test, but it solves all the problems with testing this code. By running in a subprocess, you can freely execute sys.exit and get the return value. By having a wrapper script, you can control how the code is loaded and run. You don't need to mock anything, just make sure that your packages are set up correctly, and that your test runner doesn't attempt to pick up the wrapper script as a test.
The code lines you have shown are not suited to be unit-tested, but should rather be integration tested. The reason is, that your code lines consist only of interactions with other components (in this case the signal, sys and os modules).
Therefore, the bugs you can expect to encounter lie in the interactions with these other components: Are you calling the right functions in the right components with the right values for the arguments in the right order and are the results/reactions as you expect them to be?
All these questions can not be answered in a unit-test, where bugs shall be found that can be found in the isolated units: If you mock the signal, sys and/or the os dependencies, then you will write your mocks such that they reflect your (potentially wrong) understanding of these components. The unit-tests will therefore succeed, although the code in the integrated system may fail. If your intent is that the code works on different systems, you might even encounter the situation that the code works in one integration (maybe Linux) but fails in another (maybe Windows).
Therefore, for code like yours, unit-testing and thus mocking for unit-testing does not have much value.
Monkey patch the handler and send the signal when testing?
import os
import signal
import sys
import time
# your handler
def sigterm_handler(signum, frame):
print("Handled")
pid = os.getpid() # type: int FIXME: what's this for?
sys.exit(0)
signal.signal(signal.SIGTERM, sigterm_handler)
# Mock out the existing sigterm_handler
_handled = False
def mocked_sigterm_handler(signum, frame):
print("Mocked")
_handled = True
# register the handler
signal.signal(signal.SIGTERM, mocked_sigterm_handler)
# test sending the signal
os.kill(os.getpid(), signal.SIGTERM)
print(f"done ({_handled})")
# reset your handler?
signal.signal(signal.SIGTERM, sigterm_handler)
If you want to test you handler itself you'll probably have to put some kind of code like this.. in the handler which is not beautiful.
if _unittesting_sigterm_handler:
_handled = True
else:
sys.exit(0)
and then you can just call the handler directly (or pass the test flag in the call).
_unittesting_sigterm_handler = True
sigterm_handler(0, None)
Let's say I have a test as shown below:
import pytest
import copy
#pytest.fixture(scope='session')
def session_tool(request):
tool = request.config.tool
# Build is the critical part and may fail, raising an exception
tool.build()
return tool
#pytest.fixture
def tool(session_tool):
return copy.deepcopy(session_tool)
def test_tool(tool, args):
assert tool.run(args) == 0
It builds a session-scoped tool and then creates a copy of it for each testcase. But when the build fails, session_tool fixture is executed again for the next testcase, which fails again... until it fails for all testcases. As there are a lot of testcases, it takes some time until the process is finished.
Is there any way to tell pytest to skip all tests which use session_fixture after the first attempt to build fails?
I can think of two approaches:
1) calling pytest.skip() will cause the test to be skipped. This works if it's called from within a fixture as well. In your case, it will cause all the remaining tests to be skipped.
2) calling pytest.exit() will cause your test suite to stop running, as if KeyboardInterrupt was triggered.
I have a post_save hook that triggers a task to run in celery. The task also updates the model, which causes the post_save hook to run. The catch is I do not want to .delay() the call in this instance, I just want to run it synchronously because it's already being run in a worker.
Is there an environmental variable or something else I can use to detect when the code is being run in celery?
To clarify: I'm aware that Celery tasks can still be called as normal functions, that's exactly what I'm trying to take advantage of. I want to do something like this:
if os.environ['is_celery']:
my_task(1, 2, 3)
else:
my_task.delay(1, 2, 3)
Usually you'd have common.py, production.py, test.py and local.py/dev.py. You could just add a celery_settings.py with the following content:
from production import *
IS_CELERY = True
Then in your celery.py (I'm assuming you have one) you'll do
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.celery_settings")
Then in your script you can now do:
if getattr(settings, 'IS_CELERY', None):
my_task(1, 2, 3)
else:
my_task.delay(1, 2, 3)
I've been writing a few Twisted servers and have created a WatchDog timer that runs periodically. It's default behavior is to check if it was called within some delta of time from it's schedule, which helps to report if the program is being blocked unduly. It also provides a way for a user defined callback function to the WatchDog that could be used to check the health of other parts of the system. The WatchDog timer is implemented using the twisted.internet.task.LoopingCall. I'm concerned if the user defined function creates an exception the WatchDog timer will stop being called. I have Exception handling in the code, but I'd like to have a way to restart the WatchDog timer if it should still manage to crash. However, I don't understand how to use the deferred returned by the LoopingCall().start() method. Here's some sample code to show what I mean:
import sys
from twisted.internet import reactor, defer, task
from twisted.python import log
def periodic_task():
log.msg("periodic task running")
x = 10 / 0
def periodic_task_crashed():
log.msg("periodic_task broken")
log.startLogging(sys.stdout)
my_task = task.LoopingCall(periodic_task)
d = my_task.start(1)
d.addErrback(periodic_task_crashed)
reactor.run()
When I run this code I get one "periodic task running" message from the periodic_task() function and that's it. The deferred returned by my_task.start(1) never has it's errback called, which by my reading of the documentation is what's supposed to happen.
Can someone help me out and point me to what I'm doing wrong?
Thanks in advance!
Doug
The signature of periodic_task_crashed is wrong. It is an error callback on a Deferred, so it will be called with an argument, the Failure representing the error result the Deferred got. Since it is defined to take no arguments, calling it produces a TypeError which becomes the new error result of the Deferred.
Redefine it like this:
def periodic_task_crashed(reason):
log.err(reason, "periodic_task broken")
I'm building an application using gevent. My app is getting rather big now as there are a lot of jobs being spawned and destroyed. Now I've noticed that when one of these jobs crashes my entire application just keeps running (if the exception came from a non main greenlet) which is fine. But the problem is that I have to look at my console to see the error. So some part of my application can "die" and I'm not instantly aware of that and the app keeps running.
Jittering my app with try catch stuff does not seem to be a clean solution.
Maybe a custom spawn function which does some error reporting?
What is the proper way to monitor gevent jobs/greenlets? catch exceptions?
In my case I listen for events of a few different sources and I should deal with each different.
There are like 5 jobs extremely important. The webserver greenlet, websocket greenlet,
database greenlet, alarms greenlet, and zmq greenlet. If any of those 'dies' my application should completely die. Other jobs which die are not that important. For example, It is possible that websocket greenlet dies due to some exception raised and the rest of the applications keeps running fine like nothing happened. It is completely useless and dangerous now and should just crash hard.
I think the cleanest way would be to catch the exception you consider fatal and do sys.exit() (you'll need gevent 1.0 since before that SystemExit did not exit the process).
Another way is to use link_exception, which would be called if the greenlet died with an exception.
spawn(important_greenlet).link_exception(lambda *args: sys.exit("important_greenlet died"))
Note, that you also need gevent 1.0 for this to work.
If on 0.13.6, do something like this to kill the process:
gevent.get_hub().parent.throw(SystemExit())
You want to greenlet.link_exception() all of your greenlets to a to janitor function.
The janitor function will be passed any greenlet that dies, from which it can inspect its greenlet.exception to see what happened, and if necessary do something about it.
As #Denis and #lvo said, link_exception is OK, but I think there would be a better way for that, without change your current code to spawn greenlet.
Generally, whenever an exception is thrown in a greenlet, _report_error method (in gevent.greenlet.Greenlet) will be called for that greenlet. It will do some stuff like call all the link functions and finally, call self.parent.handle_error with exc_info from current stack. The self.parent here is the global Hub object, this means, all the exceptions happened in each greenlet will always be centralize to one method for handling. By default Hub.handle_error distinguish the exception type, ignore some type and print the others (which is what we always saw in the console).
By patching Hub.handle_error method, we can easily register our own error handlers and never lose an error anymore. I wrote a helper function to make it happen:
from gevent.hub import Hub
IGNORE_ERROR = Hub.SYSTEM_ERROR + Hub.NOT_ERROR
def register_error_handler(error_handler):
Hub._origin_handle_error = Hub.handle_error
def custom_handle_error(self, context, type, value, tb):
if not issubclass(type, IGNORE_ERROR):
# print 'Got error from greenlet:', context, type, value, tb
error_handler(context, (type, value, tb))
self._origin_handle_error(context, type, value, tb)
Hub.handle_error = custom_handle_error
To use it, just call it before the event loop is initialized:
def gevent_error_handler(context, exc_info):
"""Here goes your custom error handling logics"""
e = exc_info[1]
if isinstance(e, SomeError):
# do some notify things
pass
sentry_client.captureException(exc_info=exc_info)
register_error_handler(gevent_error_handler)
This solution has been tested under gevent 1.0.2 and 1.1b3, we use it to send greenlet error information to sentry (a exception tracking system), it works pretty well so far.
The main issue with greenlet.link_exception() is that it does not give any information on traceback which can be really important to log.
For logging with traceback, I use a decorator to spwan jobs which indirect job call into a simple logging function:
from functools import wraps
import gevent
def async(wrapped):
def log_exc(func):
#wraps(wrapped)
def wrapper(*args, **kwargs):
try:
func(*args, **kwargs)
except Exception:
log.exception('%s', func)
return wrapper
#wraps(wrapped)
def wrapper(*args, **kwargs):
greenlet = gevent.spawn(log_exc(wrapped), *args, **kwargs)
return wrapper
Of course, you can add the link_exception call to manage jobs (which I did not need)