I have some Twisted code which creates multiple chains of Deferreds. Some of these may fail without having an errback which puts them back on the callback chain. I haven't been able to write a unit test for this code - the failing Deferred causes the test to fail after the test code has completed. How can I write a passing unit test for this code? Is it expected that every Deferred which could fail in normal operation should have an errback at the end of the chain which puts it back on the callback chain?
The same thing happens when there's a failed Deferred in a DeferredList, unless I create the DeferredList with consumeErrors. This is the case even when the DeferredList is created with fireOnOneErrback and is given an errback which puts it back on the callback chain. Are there any implications for consumeErrors besides suppressing test failures and error logging? Should every Deferred which may fail without an errback be put a DeferredList?
Example tests of example code:
from twisted.trial import unittest
from twisted.internet import defer
def get_dl(**kwargs):
"Return a DeferredList with a failure and any kwargs given."
return defer.DeferredList(
[defer.succeed(True), defer.fail(ValueError()), defer.succeed(True)],
**kwargs)
def two_deferreds():
"Create a failing Deferred, and create and return a succeeding Deferred."
d = defer.fail(ValueError())
return defer.succeed(True)
class DeferredChainTest(unittest.TestCase):
def check_success(self, result):
"If we're called, we're on the callback chain."
self.fail()
def check_error(self, failure):
"""
If we're called, we're on the errback chain.
Return to put us back on the callback chain.
"""
return True
def check_error_fail(self, failure):
"""
If we're called, we're on the errback chain.
"""
self.fail()
# This fails after all callbacks and errbacks have been run, with the
# ValueError from the failed defer, even though we're
# not on the errback chain.
def test_plain(self):
"""
Test that a DeferredList without arguments is on the callback chain.
"""
# check_error_fail asserts that we are on the callback chain.
return get_dl().addErrback(self.check_error_fail)
# This fails after all callbacks and errbacks have been run, with the
# ValueError from the failed defer, even though we're
# not on the errback chain.
def test_fire(self):
"""
Test that a DeferredList with fireOnOneErrback errbacks on failure,
and that an errback puts it back on the callback chain.
"""
# check_success asserts that we don't callback.
# check_error_fail asserts that we are on the callback chain.
return get_dl(fireOnOneErrback=True).addCallbacks(
self.check_success, self.check_error).addErrback(
self.check_error_fail)
# This succeeds.
def test_consume(self):
"""
Test that a DeferredList with consumeErrors errbacks on failure,
and that an errback puts it back on the callback chain.
"""
# check_error_fail asserts that we are on the callback chain.
return get_dl(consumeErrors=True).addErrback(self.check_error_fail)
# This succeeds.
def test_fire_consume(self):
"""
Test that a DeferredList with fireOnOneCallback and consumeErrors
errbacks on failure, and that an errback puts it back on the
callback chain.
"""
# check_success asserts that we don't callback.
# check_error_fail asserts that we are on the callback chain.
return get_dl(fireOnOneErrback=True, consumeErrors=True).addCallbacks(
self.check_success, self.check_error).addErrback(
self.check_error_fail)
# This fails after all callbacks and errbacks have been run, with the
# ValueError from the failed defer, even though we're
# not on the errback chain.
def test_two_deferreds(self):
# check_error_fail asserts that we are on the callback chain.
return two_deferreds().addErrback(self.check_error_fail)
There are two important things about trial related to this question.
First, a test method will not pass if a Failure is logged while it is running. Deferreds which are garbage collected with a Failure result cause the Failure to be logged.
Second, a test method which returns a Deferred will not pass if the Deferred fires with a Failure.
This means that neither of these tests can pass:
def test_logit(self):
defer.fail(Exception("oh no"))
def test_returnit(self):
return defer.fail(Exception("oh no"))
This is important because the first case, the case of a Deferred being garbage collected with a Failure result, means that an error happened that no one handled. It's sort of similar to the way Python will report a stack trace if an exception reaches the top level of your program.
Likewise, the second case is a safety net provided by trial. If a synchronous test method raises an exception, the test doesn't pass. So if a trial test method returns a Deferred, the Deferred must have a success result for the test to pass.
There are tools for dealing with each of these cases though. After all, if you couldn't have a passing test for an API that returned a Deferred that fired with a Failure sometimes, then you could never test your error code. That would be a pretty sad situation. :)
So, the more useful of the two tools for dealing with this is TestCase.assertFailure. This is a helper for tests that want to return a Deferred that's going to fire with a Failure:
def test_returnit(self):
d = defer.fail(ValueError("6 is a bad value"))
return self.assertFailure(d, ValueError)
This test will pass because d does fire with a Failure wrapping a ValueError. If d had fired with a success result or with a Failure wrapping some other exception type, then the test would still fail.
Next, there's TestCase.flushLoggedErrors. This is for when you're testing an API that's supposed to log an error. After all, sometimes you do want to inform an administrator that there's a problem.
def test_logit(self):
defer.fail(ValueError("6 is a bad value"))
gc.collect()
self.assertEquals(self.flushLoggedErrors(ValueError), 1)
This lets you inspect the failures which got logged to make sure your logging code is working properly. It also tells trial not to worry about the things you flushed, so they'll no longer cause the test to fail. (The gc.collect() call is there because the error isn't logged until the Deferred is garbage collected. On CPython, it'll be garbage collected right away due to the reference counting GC behavior. However, on Jython or PyPy or any other Python runtime without reference counting, you can't rely on that.)
Also, since garbage collection can happen at pretty much any time, you might sometimes find that one of your tests fails because an error is logged by a Deferred created by an earlier test being garbage collected during the execution of the later test. This pretty much always means your error handling code is incomplete in some way - you're missing an errback, or you failed to chain two Deferreds together somewhere, or you're letting your test method finish before the task it started actually finishes - but the way the error is reported sometimes makes it hard to track down the offending code. Trial's --force-gc option can help with this. It causes trial to invoke the garbage collector between each test method. This will slow down your tests significantly, but it should cause the error to be logged against the test which is actually triggering it, not an arbitrary later test.
Related
For any possible try-finally block in Python, is it guaranteed that the finally block will always be executed?
For example, let’s say I return while in an except block:
try:
1/0
except ZeroDivisionError:
return
finally:
print("Does this code run?")
Or maybe I re-raise an Exception:
try:
1/0
except ZeroDivisionError:
raise
finally:
print("What about this code?")
Testing shows that finally does get executed for the above examples, but I imagine there are other scenarios I haven't thought of.
Are there any scenarios in which a finally block can fail to execute in Python?
"Guaranteed" is a much stronger word than any implementation of finally deserves. What is guaranteed is that if execution flows out of the whole try-finally construct, it will pass through the finally to do so. What is not guaranteed is that execution will flow out of the try-finally.
A finally in a generator or async coroutine might never run, if the object never executes to conclusion. There are a lot of ways that could happen; here's one:
def gen(text):
try:
for line in text:
try:
yield int(line)
except:
# Ignore blank lines - but catch too much!
pass
finally:
print('Doing important cleanup')
text = ['1', '', '2', '', '3']
if any(n > 1 for n in gen(text)):
print('Found a number')
print('Oops, no cleanup.')
Note that this example is a bit tricky: when the generator is garbage collected, Python attempts to run the finally block by throwing in a GeneratorExit exception, but here we catch that exception and then yield again, at which point Python prints a warning ("generator ignored GeneratorExit") and gives up. See PEP 342 (Coroutines via Enhanced Generators) for details.
Other ways a generator or coroutine might not execute to conclusion include if the object is just never GC'ed (yes, that's possible, even in CPython), or if an async with awaits in __aexit__, or if the object awaits or yields in a finally block. This list is not intended to be exhaustive.
A finally in a daemon thread might never execute if all non-daemon threads exit first.
os._exit will halt the process immediately without executing finally blocks.
os.fork may cause finally blocks to execute twice. As well as just the normal problems you'd expect from things happening twice, this could cause concurrent access conflicts (crashes, stalls, ...) if access to shared resources is not correctly synchronized.
Since multiprocessing uses fork-without-exec to create worker processes when using the fork start method (the default on Unix), and then calls os._exit in the worker once the worker's job is done, finally and multiprocessing interaction can be problematic (example).
A C-level segmentation fault will prevent finally blocks from running.
kill -SIGKILL will prevent finally blocks from running. SIGTERM and SIGHUP will also prevent finally blocks from running unless you install a handler to control the shutdown yourself; by default, Python does not handle SIGTERM or SIGHUP.
An exception in finally can prevent cleanup from completing. One particularly noteworthy case is if the user hits control-C just as we're starting to execute the finally block. Python will raise a KeyboardInterrupt and skip every line of the finally block's contents. (KeyboardInterrupt-safe code is very hard to write).
If the computer loses power, or if it hibernates and doesn't wake up, finally blocks won't run.
The finally block is not a transaction system; it doesn't provide atomicity guarantees or anything of the sort. Some of these examples might seem obvious, but it's easy to forget such things can happen and rely on finally for too much.
Yes. Finally always wins.
The only way to defeat it is to halt execution before finally: gets a chance to execute (e.g. crash the interpreter, turn off your computer, suspend a generator forever).
I imagine there are other scenarios I haven't thought of.
Here are a couple more you may not have thought about:
def foo():
# finally always wins
try:
return 1
finally:
return 2
def bar():
# even if he has to eat an unhandled exception, finally wins
try:
raise Exception('boom')
finally:
return 'no boom'
Depending on how you quit the interpreter, sometimes you can "cancel" finally, but not like this:
>>> import sys
>>> try:
... sys.exit()
... finally:
... print('finally wins!')
...
finally wins!
$
Using the precarious os._exit (this falls under "crash the interpreter" in my opinion):
>>> import os
>>> try:
... os._exit(1)
... finally:
... print('finally!')
...
$
I'm currently running this code, to test if finally will still execute after the heat death of the universe:
try:
while True:
sleep(1)
finally:
print('done')
However, I'm still waiting on the result, so check back here later.
According to the Python documentation:
No matter what happened previously, the final-block is executed once the code block is complete and any raised exceptions handled. Even if there's an error in an exception handler or the else-block and a new exception is raised, the code in the final-block is still run.
It should also be noted that if there are multiple return statements, including one in the finally block, then the finally block return is the only one that will execute.
Well, yes and no.
What is guaranteed is that Python will always try to execute the finally block. In the case where you return from the block or raise an uncaught exception, the finally block is executed just before actually returning or raising the exception.
(what you could have controlled yourself by simply running the code in your question)
The only case I can imagine where the finally block will not be executed is when the Python interpretor itself crashes for example inside C code or because of power outage.
I found this one without using a generator function:
import multiprocessing
import time
def fun(arg):
try:
print("tried " + str(arg))
time.sleep(arg)
finally:
print("finally cleaned up " + str(arg))
return foo
list = [1, 2, 3]
multiprocessing.Pool().map(fun, list)
The sleep can be any code that might run for inconsistent amounts of time.
What appears to be happening here is that the first parallel process to finish leaves the try block successfully, but then attempts to return from the function a value (foo) that hasn't been defined anywhere, which causes an exception. That exception kills the map without allowing the other processes to reach their finally blocks.
Also, if you add the line bar = bazz just after the sleep() call in the try block. Then the first process to reach that line throws an exception (because bazz isn't defined), which causes its own finally block to be run, but then kills the map, causing the other try blocks to disappear without reaching their finally blocks, and the first process not to reach its return statement, either.
What this means for Python multiprocessing is that you can't trust the exception-handling mechanism to clean up resources in all processes if even one of the processes can have an exception. Additional signal handling or managing the resources outside the multiprocessing map call would be necessary.
You can use a finally with an if statement, below example is checking for network connection and if its connected it will run the finally block
try:
reader1, writer1 = loop.run_until_complete(self.init_socket(loop))
x = 'connected'
except:
print("cant connect server transfer") #open popup
x = 'failed'
finally :
if x == 'connected':
with open('text_file1.txt', "r") as f:
file_lines = eval(str(f.read()))
else:
print("not connected")
I have the following function,
def load():
with open(PATH_CONFIG, 'r') as file:
return json.loads(file.read())
Will there be a file.close() called? I know that the with keyword normally calls the close() method for the file at the end of the indented block, but at the same time the return keyword means that the rest of the function does not run.
Just like try/finally, anything that exits the with block (return, break/continue that affects a loop surrounding it, exception thrown, sys.exit called, etc.) will perform appropriate cleanup as execution bubbles out of the with block.
The only exceptions are:
When there are actual bugs (in the interpreter, or in misuse of intrinsically dangerous tools like ctypes) where the interpreter itself crashes or otherwise exits "forcefully" (e.g. due to a segfault)
Calling os._exit bypasses all cleanup procedures (that's why it should never be used in anything but forked worker processes)
Return exits the with block like a normal dedent
Yes.
If an exception is raised, a context manager has the option of changing its behavior, but there's no difference between a return and falling off the end of the statement body, and with few exceptions most context managers will perform their cleanup and allow the exception to propagate.
The idea is that it's comparable to a finally statement, and will be executed no matter how the block is exited. The contextmanager class from the standard library makes this analogy concrete.
from contextlib import contextmanager
#contextmanager
def example():
print('enter')
try:
yield
print('unexceptional return')
finally:
print('exit')
We can use with example(): in various ways to see how the with statement performs in a more visible example than closing a file.
In this pseudo code, the destructor of o is never called.
class MyObj():
def __del__():
print "Destroyed!"
do_upon_death()
def parse():
o = MyObj()
print "Ok so far..."
raise Exception
run_in_QThread(parse)
do_other_things()
I see the exception printed on the terminal, but o seems not being garbage-collected. The interpreter does not exit upon the exception (the main thread keeps running).
The whole purpose behind this is to be able to do certain cleanup in the destructor when exceptions occur.
Another way to explain this is with a different pattern:
try:
run_in_thread(parse)
except:
print "Will I ever get printed?"
Exceptions happening in parse will not get caught.
Even further:
with MyContextManager() as manager:
run_in_thread(lambda: do_something(manager))
Here manager.__exit__() will get called with all None arguments as if the code completed cleanly, even if there is an exception in do_something.
The purpose behind this is because I'm running certain tasks which can fail for too many different reasons, and these tasks can be done partially in the main thread, and partially in a worker thread. It can fail in either.
So my initial idea was to keep references to an object while the task was running, so if the task completed by exiting the context or by an exception, references would be lost and eventually the destructor would be run.
I'm trying to keep track of these tasks and keep a visual indicator on a GUI of which tasks are still doing their job.
You can catch Exception after it was raised and do what you want to do
try:
o = MyObj()
print "Ok so far..."
raise Exception
except Exception:
del o
or use such form if is no difference was exception raised or not:
finaly:
del o
If you need just keep Thread alive, may be, you should use: Thread.sleep()
As i understood in Thread is only parse(), so you need to create signal by emit() about deleting and send it in main thread
I believe that caughting destructor is not the best decision, it will be better to catch some signal. Please add more info about why you do this all.
i have use funcargs in my tests:
def test_name(fooarg1, fooarg2):
all of them have pytest_funcarg__ factories, which returns request.cached_setup, so all of them have setup/teardown sections.
sometimes i have a problem with fooarg2 teardown, so i raise exception in here. in this case ignore all the others teardowns(fooarg1.teardown, teardown_module, etc) and just goes to pytest_sessionfinished section.
is there any option in pytest not to collect exceptions and execute all remaining teardowns functions?
Are you using pytest-2.5.1? pytest-2.5 and in particular issue287 is supposed to have brought support for running all finalizers and re-raising the first failed exception if any.
In your teardown function you can catch the error and print it.
def teardown(): # this is the teardown function with the error
try:
# the original code with error
except:
import traceback
traceback.print_exc() # no error should pass silently
Am on a project using txrdq, am testing (using trial) for a case where a queued job may fail, trial marks the testcase as failed whenever it hits a failure in a errback ..
The errback is a normal behaviour, since a queued job may fail to launch, how to test this case using trial without failing the test ?
here's an example of the test case:
from twisted.trial.unittest import TestCase
from txrdq.rdq import ResizableDispatchQueue
from twisted.python.failure import Failure
class myTestCase(TestCase):
def aFailingJob(self, a):
return Failure("This is a failure")
def setUp(self):
self.queue = ResizableDispatchQueue(self.aFailingJob, 1)
def tearDown(self):
pass
def test_txrdq(self):
self.queue.put("Some argument", 1)
It seems likely that the exception is being logged, since the error handler just raises it. I'm not exactly sure what the error handling code in txrdq looks like, so this is just a guess, but I think it's a pretty good one based on your observations.
Trial fails any unit test that logs an exception, unless the test cleans that exception up after it's logged. Use TestCase.flushLoggedErrors(exceptionType) to deal with this:
def test_txrdq(self):
self.queue.put("Some argument", 1)
self.assertEqual(1, len(self.flushLoggedErrors(SomeException)))
Also notice that you should never do Failure("string"). This is analogous to raise "string". String exceptions are deprecated in Python since a looooong time ago. Always construct a Failure with an exception instance:
class JobError(Exception):
pass
def aFailingJob(self, a):
return Failure(JobError("This is a failure"))
This makes JobError the exception type you'd pass to flushLoggedErrors.
Make sure that you understand whether queue processing is synchronous or asynchronous. If it is synchronous, your test (with the flushLoggedErrors call added) is fine. If it is asynchronous, your error handler may not have run by the time your test method returns. In that case, you're not going to be testing anything useful, and the errors might be logged after the call to flush them (making the flush useless).
Finally, if you're not writing unit tests '''for''' txrdq, then you might not want to write tests like this. You can probably unit test txrdq-using code without using an actual txrdq. A normal Queue object (or perhaps another more specialized test double) will let you more precisely target the units in your application, making your tests faster, more reliable, and easier to debug.
This issue has now (finally!) been solved, by L. Daniel Burr. There's a new version (0.2.14) of txRDQ on PyPI.
By the way, in your test you should add from txrdq.job import Job, and then do something like this:
d = self.queue.put("Some argument", 1)
return self.assertFailure(d, Job)
Trial will make sure that d fails with a Job instance. There are a couple of new tests at the bottom of txrdq/test/test_rdq.py that illustrate this kind of assertion.
I'm sorry this problem caused so much head scratching for you - it was entirely my fault.
Sorry to see you're still having a problem. I don't know what's going on here, but I have been playing with it for over an hour trying to...
The queue.put method returns a Deferred. You can attach an errback to it to do the flush as #exarkun describes, and then return the Deferred from the test. I expected that to fix things (having read #exarkun's reply and gotten a comment from #idnar in #twisted). But it doesn't help.
Here's a bit of the recent IRC conversation, mentioning what I think could be happening: https://gist.github.com/2177560
As far as I can see, txRDQ is doing the right thing. The job fails and the deferred that is returned by queue.put is errbacked.
If you look in _trial_temp/test.log after you run the test, what do you see? I see an error that says Unhandled error in Deferred and the error is a Failure with a Job in it. So it seems likely to me that the error is somewhere in txRDQ. That there is a deferred that's failing and it's passing on the failure just fine to whoever needs it, but also returning the failure - causing trial to complain. But I don't know where that is. I put a print into the init of the Deferred class just out of curiousity to see how many deferreds were made during the running of the test. The answer: 12!
Sorry not to have better news. If you want to press on, go look at every deferred made by the txRDQ code. Is one of them failing with an errback that returns the failure? I don't see it, and I've put print statements in all over the place to check that things are right. I guess I must be missing something.
Thanks, and thanks too #exarkun.