I have a recursive function that I'm looking to test, however I'm having difficulty limiting the recursive call during testing. For example, below is a simple example of a recursive function that calls a bool_function(n) to check if it should break the recursive loop.
def factorial(n):
if bool_function(n):
return 1
else:
return n * factorial(n-1)
What would be the best way to test or mock bool_function(n) so that it is true for the first iteration and false for any call after?
You could always implement a class to encapsulate the state and give you more flexibility, here's a sketch:
>>> class MockBoolCheck:
... def __init__(self, fail_after=0):
... self.count = 0
... self.fail_after = fail_after
... def __call__(self, n):
... called = self.count
... self.count += 1
... return called <= self.fail_after
...
>>> bool_function = MockBoolCheck()
>>> bool_function(42)
True
>>> bool_function(42)
False
>>> bool_function(42)
False
>>> bool_function(42)
False
>>> bool_function(42)
False
If, beside other suggested solutions, you really want to mock it, and want to do it yourself (without the mocking libraries) by just replacing the mocked function.
# Your code (or module):
def bool_function(n):
print('REAL bool-function {}'.format(n))
return n <= 0
def factorial(n):
print('FACT {}'.format(n))
if bool_function(n):
return 1
else:
return n * factorial(n-1)
# Mocking code (or module):
def mock_function(n):
print('MOCK bool-function {}'.format(n))
global bool_function
bool_function = bool_func_orig # restore on the first use
return False
bool_func_orig = bool_function
bool_function = mock_function # mock it
# Go run it!
factorial(10)
If these are two separate modules, then instead of global bool_function & bool_function=... just use the somemodule.bool_function=....
If you want to use the mocking library, then it depends on which library you use. If that is unittest.mock, then you should play with side_effect=... & wraps=... (see the manual). The same approach: mock it, and un-mock it from inside the side effect on the first use.
I generally try not to leave debug code around unless I expect to use it regularly, but you could just include a default argument for the sake of debugging to force the execution to follow a particular path.
def factorial(n, debug=False):
if bool_function(n) or debug:
return 1
else:
return n * factorial(n-1)
This naturally implies that you're also externally testing bool_function()
Just pass the function as an argument. If function is None you can apply some default behavior if that is desired.
This is a common approach used in queries to iterables (e.g. Django queries or Peewee queries) in most of languages.
A function that returns boolean is usually called a predicate
def factorial(n, predicate=None):
if not predicate:
predicate = lambda x: x > 2
if predicate(n):
return 1
else:
return n * factorial(n-1)
For python > 3.6
import mock
class RecursividadeTest(unittest.TestCase):
def test_recursive(self):
with mock.patch('path.factorial') as mock_fact:
factorial(3)
self.assertTrue(mock_fact.called)
self.assertGreaterEqual(mock_fact.call_count, 2)
def test_recursive_2(self):
with mock.patch('incolumepy.sequences.fibonacci.fibonacci') as mock_fib:
for i in range(1, 5, -1):
expected = i - 1
fibonacci(i)
self.assertTrue(mock_fib.called)
self.assertEqual(mock_fib.call_count, expected)
Related
I would like to write some code to check student submissions to ensure that a given function includes np.random.choice.
For example:
import numpy as np
def checkme(z, x=[1,2,3], y=4):
tmp = np.random.choice(x, size=y)
if z:
print("z")
return(list(tmp))
I have seen that I can use calls like
tmp = inspect.signature(checkme)
for param in tmp.parameters.values():
print(param.name, ",", param.default)
To determine the parameters and values, which is great, but I want to take this one step further and ensure that the body of the function included a specific function or method. So above, I would want to ensure the students' code included np.random.choice.
How can I access the body of the function to "inspect" and determine if this is True or False?
You can temporarily replace the method you want to check with a wrapper that will let you know if it's called or not through a global variable (or something else of your choosing). I think this is the only real solution, as both checking with string matching and checking the disassembeld code like I suggested in my other answer is error prone and will inevitably miss edge cases.
Here's an example:
class Checker:
def check(self, func, funcargs):
real_np_random_choice = np.random.choice
self.called = False
def wrapper(*args, **kwargs):
self.called = True
return real_np_random_choice(*args, **kwargs)
np.random.choice = wrapper
func(*funcargs)
np.random.choice = real_np_random_choice
return self.called
Checker().check(checkme, (3, [1,2,3], 4)) # -> True
I'm using a class here only because I need to carry the result out of wrapper somehow. This could also be done with a global variable of course.
A more general solution to check that a given method of a given module is called would be:
class Checker:
def __init__(self, module, method):
self.module = module
self.method = method
def check(self, func, funcargs):
real_method = getattr(self.module, self.method)
self.called = False
def wrapper(*args, **kwargs):
self.called = True
return real_method(*args, **kwargs)
setattr(self.module, self.method, wrapper)
func(*funcargs)
setattr(self.module, self.method, real_method)
return self.called
c = Checker(np.random, 'choice')
print(c.check(checkme, (3, [1,2,3], 4)))
You could do something like this:
import inspect
f = 'XXX' # Your student's submission is stored in XXX.py
assert('np.random.choice' in inspect.getsource(__import__(f).checkme))
Instead of inspecting the source code, you could check if the function has been called or not. You can apply a decorator to perform this check:
import numpy as np
# Create your decorator
def iscalled_decorator(func):
def wrapper(*args, **kwargs):
global iscalled
iscalled = True
return func(*args, **kwargs)
return wrapper
# Decorate np.random.choice
np.random.choice = iscalled_decorator(np.random.choice)
# Import your student's function
f = 'XXX'
checkme = __import__(f).checkme
# Set a flag iscalled and call the function
iscalled = False
checkme(3, [1,2,3], 4)
# Check if the flag is True
assert(iscalled)
Assuming you want to do this having the function at hand (even in compiled pyc form) and not through string search operations (which I guess you would have already thought about), then you can use the dis module.
A call of the form y = np.random.choice(x) will be compiled into something like this (output of dis.dis()):
8 0 LOAD_GLOBAL 0 (np)
2 LOAD_ATTR 1 (random)
4 LOAD_METHOD 2 (choice)
6 LOAD_FAST 1 (x)
8 CALL_METHOD 1
10 STORE_FAST 2 (y)
The order of these instructions and their arguments should always be the same assuming that your students are using the global import numpy as np. The third LOAD_METHOD could become LOAD_ATTR depending on how the method is being loaded.
The actual call is more difficult to detect, it could become either CALL_METHOD, CALL_FUNCTION_EX, CALL_FUNCTION_KW or CALL_FUNCTION depending on how it's done. It's also not so straightforward to check that the function being called is actually the one you want like in the above case where it's obvious. Checking that the actual call is made is still possible, but requires keeping track of the Python stack and doing real parsing of the instructions and their arguments, you can check the documentation if you wish to dive into that.
I'll limit myself to just checking if np.random.choice is actually loaded in the checked function. You can do so with the following code:
import dis
def zip3(g):
try:
a, b, c = next(g), next(g), next(g)
while 1:
yield a, b, c
a, b, c = b, c, next(g)
except StopIteration:
pass
def check(func):
for a, b, c in zip3(dis.get_instructions(func)):
if a.opname == 'LOAD_GLOBAL' and a.argval == 'np':
if b.opname == 'LOAD_ATTR' and b.argval == 'random':
if c.opname in ('LOAD_ATTR', 'LOAD_METHOD') and c.argval == 'choice':
return True
return False
check(checkme) # -> True
NOTE: opcodes could change depending on Python version, but I am assuming you will run all the tests under the same Python version, so you can adjust the matches to fit your needs (use dis.dis() to check). Of course with this method you will not be able to catch more convoluted stuff like a = np; b = a.random; b.choice(x) or import numpy as whatever, but that's also true for string matching anyway.
You can use another method of inspect, inspect.getsource that will get the source code of your function as string.
import inspect
import numpy as np
def checkme(z, x=[1,2,3], y=4):
tmp = np.random.choice(x, size=y)
if z:
print("z")
return(list(tmp))
code = inspect.getsource(checkme)
lines = code.split("\n")
for line in lines:
print(line, "np.random.choice" in line)
output:
# def checkme(z, x=[1,2,3], y=4): False
# tmp = np.random.choice(x, size=y) True
# if z: False
# print("z") False
# return(list(tmp)) False
# False
I split the code string to check also on which exact line the method is called
The problem with this approach is for the aliases
if your student as imported numpy in other forms, for example
import numpy as mynp
import numpy
import numpy as npy
or of course for commented code:
# np.random.choice
etc
here some details on the class inspect.
I am coming from c
The concept of first class function is interesting and exiting.
However, I am struggling to find a practical use-case to returning a function.
I have seen the examples of building a function that returns print a grating...
Hello = grating('Hello')
Hi = grating('Hi')
But why is this better then just using
grating('Hello') and grating('Hi')?
Consider:
def func_a():
def func_b():
do sonthing
return somthing
return func_b
When this is better then:
def func_b():
do sonthing
return somthing
def func_a():
res = func_b()
return (res)
can someone point me to a real world useful example?
Thanks!
Your examples aren't helpful as written. But there are other cases where it's useful, e.g. decorators (which are functions that are called on functions and return functions, to modify the behavior of the function they're called on for future callers) and closures. The closure case can be a convenience (e.g. some other part of your code doesn't want to have to pass 10 arguments on every call when eight of them are always the same value; this is usually covered by functools.partial, but closures also work), or it can be a caching time saver. For an example of the latter, imagine a stupid function that tests which of a set of numbers less than some bound are prime by computing all primes up to that bound, then filtering the inputs to those in the set of primes.
If you write it as:
def get_primes(*args):
maxtotest = max(args)
primes = sieve_of_eratosthenes(maxtotest) # (expensive) Produce set of primes up to maxtotest
return primes.intersection(args)
then you're redoing the Sieve of Eratosthenes every call, which swamps the cost of a set intersection. If you implement it as a closure, you might do:
def make_get_primes(initialmax=1000):
primes = sieve_of_eratosthenes(initialmax) # (expensive) Produce set of primes up to maxtotest
currentmax = initialmax
def get_primes(*args):
nonlocal currentmax
maxtotest = max(args)
if maxtotest > currentmax:
primes.update(partial_sieve_of_eratosthenes(currentmax, maxtotest)) # (less expensive) Fill in additional primes not sieved
currentmax = maxtotest
return primes.intersection(args)
return get_primes
Now, if you need a tester for a while, you can do:
get_primes = make_get_primes()
and each call to get_primes is cheap (essentially free if the cached primes already cover you, and cheaper if it has to compute more).
Imagine you wanted to pass a function that would choose a function based on parameters:
def compare(a, b):
...
def anticompare(a, b): # Compare but backwards
...
def get_comparator(reverse):
if reverse: return anticompare
else: return compare
def sort(arr, reverse=false):
comparator = get_comparator(reverse)
...
Obviously this is mildly contrived, but it separates the logic of choosing a comparator from the comparator functions themselves.
A perfect example of a function returning a function is a Python decorator:
def foo(func):
def bar():
return func().upper()
return bar
#foo
def hello_world():
return "Hello World!"
print(hello_world())
The decorator is the #foo symbol above the hello_world() function declaration. In essence, we tell the interpreter that whenever it sees the mention of hello_world(), to actually call foo(hello_world()), therefore, the output of the code above is:
HELLO WORLD!
The idea of a function that returns a function in Python is for decorators, which is a design pattern that allows to add new functionality to an existing object.
I recommend you to read about decorators
On example would be creating a timer function. Let's say we want a function that accepts another function, and times how long it takes to complete:
import time
def timeit(func, args):
st = time.time()
func(args)
return str('Took ' + str(time.time() - st) + ' seconds to complete.'
timeit(lambda x: x*10, 10)
# Took 1.9073486328125e-06 seconds to complete
Edit: Using a function that returns a function
import time
def timeit(func):
st = time.time()
func(10)
return str('Took ' + str(time.time() - st) + ' seconds to complete.')
def make_func(x):
return lambda x: x
timeit(make_func(10))
# Took 1.90734863281e-06 seconds to complete.
I wanted to practice recursive and decorators and try to do this simple function but it doesn't work:
def dec(func):
def wrapper(number):
print("Recursive count:")
rec_cou(number)
return wrapper
#dec
def rec_cou(number):
""" Count from 0 to a given number from 50 and up """
if number == 0:
print(number)
return number
num = rec_cou(number - 1)
print(num + 1)
return num + 1
rec_cou(53)
The recursive function alone works well, but when i add the decorator generates error: maximun recursion depth exceeded
There are two problems with your decorator:
You try to call the decorated function, effectively invoking the wrapper function again inside the decorator, thus you have an infinite recursive loop; call the original function func instead.
To the outside, the decorated function should behave just like the original function, particularly it should return its result; otherwise you will get type errors for trying to add numbers and None
Also, currently your decorator is not counting anything... try this:
def dec(func):
func.count = 0 # give each decorated function its own counter
def wrapper(number):
print("Recursive count: %d" % func.count)
func.count += 1 # increase counter
return func(number) # call original function 'func' and return result
return wrapper
Update: From your comments, it seems I misunderstood what your decorator is supposed to do, and you misunderstood how decorators work. The decorator is not called once when you first call the function, but it replaces the function with the one defined within the decorator. In other words,
#dec
def foo(...):
...
is equivalent to
def foo(...):
...
foo = dec(foo)
I.e. the decorator in invoked exactly once when the function is decorated, and the function constructed in the decorator is called each time the original function is called, replacing it. If you want to print only once, either use the decorator from the other answer, or rather use no decorator at all: Just create a wrapper that prints and then calls the function. This is not unusual for providing an 'entry point' to recursive functions.
def print_and_run(number):
print("Recursive count:")
rec_cou(number)
BTW, this is the decorator that I usually use to visualize recursive calls:
def trace(f):
trace.depth = 0
def _f(*args, **kwargs):
print " " * trace.depth, ">", f.__name__, args, kwargs
trace.depth += 1
res = f(*args, **kwargs)
trace.depth -= 1
print " " * trace.depth, "<", res
return res
return _f
To solve the maximum recursion depth problem, call the function passed into the decorator (func) rather than rec_cou and return the value of the function call. That is, on line 5, replace rec_cou(number) with return func(number).
Edit:
def decorate(function):
def wrapper(parameter):
if wrapper.initial:
print("Recursive count:")
wrapper.initial = False
result = function(parameter)
wrapper.initial = True
return result
wrapper.initial = True
return wrapper
#decorate
def count(number):
""" Prints integers on the interval [0, number] """
if number:
count(number - 1)
print(number)
count(53)
Without decorator:
def count(number):
""" Prints integers on the interval [0, number] """
if number:
count(number - 1)
else:
print("Recursive count:")
print(number)
count(53)
If all you want is for the function rec_cou to print something before its recursive descent, just modify that function and don't bother with decorators.
def rec_cou(number, internal_call=False):
""" Count from 0 to a given number from 50 and up """
if not internal_call:
print "Now performing recursive count, starting with %d" % number
if number == 0:
return number
num = rec_cou(number - 1, internal_call=True)
return num + 1
As I mentioned in my comments, all I've done is take the idea behind Joel's answer (which was to add a variable--which I called a "flag"--indicating whether the function is being called externally or as part of the recursion) and moved the flag variable (which I've called internal_call, whereas Joel called it initial) inside the function itself.
Additionally, I'm not sure what all this num business is about. Note that:
For the 0 case, rec_cou returns 0.
For number > 0, num is set to the value returned by rec_cou(number-1), then 1+num is returned.
For example, in the case of rec_cou(1), num is set to rec_cou(0), which is 0, then 0 + 1 is returned, which is 1. Similarly, for rec_cou(2), one more than the value of rec_cou(1) is returned, so 2 is returned.
In short, for every natural number, rec_cou(number) returns the value of the input number. It's not clear what you're trying to achieve, but what you've got is an identity function, which seems unlikely to be what you want.
I have seen questions similar to this but not quite the same. I have an expression
if foo == ....
return -1
elif myFunction(bar) != -1:
return myFunction(bar)
elif ...
I don't want to calculate myFunction(bar) twice. If it were simply an if, I could do
temp = myFunction(bar)
if temp != -1
return temp
However doing this with an elif would result in unnecessary calculations of temp if we were to follow the intitial if.
I can see a solution using an
if ...
else
temp = myFunction(bar)
if temp != -1:
return temp
elif ...
But that now starts to become more ugly. Is there a better way to accomplish this?
If you do this a lot, it might pay to have a memoizing decorator around. The only memoizer that is currently in the standard library (for newer Python 3.x) is lru_cache, which is overkill, but shows the general idea:
>>> def func():
... print("foo")
... return 1
...
>>> if func():
... print(func())
...
foo
foo
1
Now memoize func:
>>> from functools import lru_cache
>>> func = lru_cache(1)(func)
>>> if func():
... print(func())
...
foo
1
If you do not want to call myFunction(bar) twice then there is no other way than using an intermediate variable to store the result in. People here start proposing complex caching solutions for this. This can be pretty convenient in extreme cases, but before doing so, let's get back to the basics a bit. You should make proper use of the fact that you want to return from within your conditional blocks. In these situations you can save many elses. What follows now basically is the code block from your question, but without dots, with proper indentation, and with names according to PEP8:
if foo == "bar"
return -1
elif myfunction(bar) != -1:
return myfunction(bar)
else
return None
It can easily be replaced with:
if foo == "bar"
return -1
t = myfunction(bar)
if t != -1:
return t
return None
As already stated in another answer, you can call your function twice if it does not affect the performance of your code. The result would look as simple as
if foo == "bar"
return -1
if myfunction(bar) != -1:
return myfunction(bar)
return None
If your function call is simple enough, you could go with calling twice. Python doesn't have support for assign and compare feature. So, you have to trade off readability/elegance with performance.
you could also make a function promise like this:
class Promise():
def __init__(self, func):
self.__func = func;
self.__computed = False;
def Get(self):
if not self.__computed:
self.__result = self.__func()
self.__computed = True
return self.__result
def func(text):
print text
return 1
f = Promise(lambda: func("compute"))
>>> f.Get() # first call, compute
compute
1
>>> f.Get() # second call, return result
1
I am attempting to write a program to perform arithmetic mod n, given n. I was wondering if there is any way within Python (preferably 2.7) to dynamically define a function such that its behavior depends on the name used to call it. More concretely, I would like to define a function named "*mod", where * is an integer, that then does arithmetic mod *. Perhaps more clearly, I would like to write one function definition for *mod that defines the functions 2mod, 3mod, 4mod, and so on. Is this possible? I apologize if a similar question has already been asked or if my answer is readily available in documentation; I tried to search for it, but I didn't know exactly how to describe the functionality that I'm looking for, so I may have missed it.
Thanks!
You don't want to do that. Just make a simple function and pass both numbers as arguments:
def mod(x, n):
return x % n
print mod(5, 2)
# 1
Well, if you really, really want to, look at this quick hack. It uses a wrapper class to wrap the module in a class, so you can use __getattr__:
import sys
import functools
def add(a, b):
return a + b
def sub(a, b):
return a - b
class Wrapper(object):
def __init__(self, wrapped):
self.wrapped = wrapped
def __getattr__(self, name):
try:
# quick hack. Don't try this at home :-)
f = ''.join(x for x in name if not x.isdigit())
n = ''.join(x for x in name if x.isdigit())
return functools.partial(getattr(self.wrapped, f), int(n))
except:
return getattr(self.wrapped, name)
sys.modules[__name__] = Wrapper(sys.modules[__name__])
Now, when you call e.g. add10(12) on this module, the result is 22. Note that method names must not start with a number, but you could use names like _add and call the methods like _55add(45) and so on.
But I would follow Haidro advice: You don't want to do that. Just calling the method with two arguments is a lot simpler.
Using globals, lambda:
for i in range(2, 5):
globals()['mod{}'.format(i)] = lambda x, n=i: x % n
assert mod2(4) == 0
assert mod2(3) == 1
assert mod3(2) == 2
assert mod3(1) == 1
assert mod4(1) == 1
assert mod4(2) == 2
assert mod4(3) == 3
assert mod4(9) == 1
You could achieve this by generating the functions as a string, and then exec this string to get the function in the current namespace. Something like:
n = 2
s = 'def mod%i(x):' % n
s += ' return x %% %i' % n
exec s
This would define the function mod2(x)