I would like to know, just for fun, if I can create functions using function class constructor, i.e. without language construct def, just like creating class by instantiating type object. I know, function constructor takes 2 args - code object and globals. But I don't know how I should compile the source properly.
>>> def f():
... pass
>>> Function = type(f)
>>> Function
<class 'function'>
>>> code = compile("x + 10", "<string>", "exec")
>>> f = Function(code, globals())
>>> f()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1, in <module>
NameError: name 'x' is not defined
>>> f(20)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: <module>() takes 0 positional arguments but 1 was given
You need to set many attributes on the code object, such as co_varnames, co_nlocals, etc.
What clearly works is
code = compile("def foo(n):return n+10", "<string>", "exec").co_consts[0]
func = Function(code, globals())
but I guess this would be considered cheating. To really define the code object from scratch, do (for 3.3)
code = types.CodeType(1, 0, 1, 2, 67, b'|\x00\x00d\x01\x00\x17S', (None, 10),
(), ('x',), '<string>', 'f', 1, b'\x00\x01')
func = Function(code, globals())
print(func(10))
This, of course, requires you to do the entire compile() yourself.
Well, this works:
>>> x = 0
>>> def f(): pass
...
>>> func = type(f)
>>> code = compile("global x\nx += 10","<string>","exec")
>>> nf = func(code,globals())
>>> nf()
>>> x
10
Don't know how you'd pass arguments to the function though.
Related
I'm trying to change the print builtin function from python.
The reason I'm trying to achieve this is cause my application has an verbose sys.argv, and I want to use print to console out the message whether the verbose is True or False.
I've tried to use create a new function, but I get a recursion error:
>>> import builtins
>>> def new_print(*args, **kwargs):
... print('print:', *args, **kwargs)
...
>>> old_print = builtins.print
>>> old_print(1)
1
>>> builtins.print = new_print
>>> print(1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in new_print
File "<stdin>", line 2, in new_print
File "<stdin>", line 2, in new_print
[Previous line repeated 996 more times]
RecursionError: maximum recursion depth exceeded
I've tried using sys.stdout():
>>> import builtins
>>> import sys
>>> def new_print(*args, **kwargs):
... sys.stdout(*args, **kwargs)
...
>>> old_print = builtins.print
>>> old_print(1)
1
>>> builtins.print = new_print
>>> print(1
... )
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in new_print
TypeError: '_io.TextIOWrapper' object is not callable
Although using those options, none seemed to work properly.
I need the new print function to be accesible for all my module files, without needing to import it every time. That's why I'm trying to change the builtin function, but I'm not sure that changing this in my init.py file will make a difference for my other files.
Please, if you have any idea on what could help me, please leave it below.
You almost had it. Call old_print in your new function:
def new_print(*args, **kwargs):
old_print('print:', *args, **kwargs)
old_print = print
print = new_print
I'm trying to understand the difference between these two mock constructs and when is it appropriate to use either. I tested it in the interpreter, e.g.:
>>> mm = mock.MagicMock(spec=list)
>>> ca = mock.create_autospec(list)
>>> mm
<MagicMock spec='list' id='140372375801232'>
>>> mm()
<MagicMock name='mock()' id='140372384057808'>
>>> mm.append()
<MagicMock name='mock.append()' id='140372375724720'>
>>> mm().append()
<MagicMock name='mock().append()' id='140372375753104'>
>>> ca
<MagicMock spec='list' id='140372384059248'>
>>> ca()
<NonCallableMagicMock name='mock()' spec='list' id='140372384057040'>
>>> ca.append()
<MagicMock name='mock.append()' id='140372375719744'>
>>> ca().append()
<MagicMock name='mock().append()' id='140372375796848'>
>>>
But I can't understand why "constructing" the mock created using create_autospec gives me a NonCallableMagicMock and the MagicMock gives me more MagicMock. The documentation isn't helping much.
The main difference between using the spec argument and using create_autospec is recursiveness. In the first case, the object itself is specced, while the called object is not:
>>> mm = mock.MagicMock(spec=list)
>>> mm
<MagicMock spec='list' id='2868486557120'>
>>> mm.foo
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\Python\Python38\lib\unittest\mock.py", line 635, in __getattr__
raise AttributeError("Mock object has no attribute %r" % name)
AttributeError: Mock object has no attribute 'foo'
>>> mm.append
<MagicMock name='mock.append' id='2868486430240'>
>>> mm.append.foo
<MagicMock name='mock.append.foo' id='2868486451408'>
In the second case, the called objects are also specced (lazily):
>>> ca = mock.create_autospec(list)
>>> ca
<MagicMock spec='list' id='2868486254848'>
>>> ca.foo
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\Python\Python38\lib\unittest\mock.py", line 635, in __getattr__
raise AttributeError("Mock object has no attribute %r" % name)
AttributeError: Mock object has no attribute 'foo'
>>> ca.append
<MagicMock name='mock.append' spec='method_descriptor' id='2868486256336'>
>>> ca.append.foo
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\Python\Python38\lib\unittest\mock.py", line 635, in __getattr__
raise AttributeError("Mock object has no attribute %r" % name)
AttributeError: Mock object has no attribute 'foo'
There is one caveat, that is shown in your example code. If you use create_autospec as shown here, it behaves as if the object is a class, not an instance, so you are able to call it (creating an instance):
>>> ca = mock.create_autospec(list)
>>> ca()
<NonCallableMagicMock name='mock()' spec='list' id='2868485877280'>
If you want to behave it like an instance, you have to use instance=True:
>>> ca = mock.create_autospec(list, instance=True)
>>> ca
<NonCallableMagicMock spec='list' id='2868485875024'>
>>> ca()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'NonCallableMagicMock' object is not callable
Note that using mock.patch with autospec=True creates a mock that behaves like the one created using mock.create_autospec, as described in the documentation.
Also note that the return value of a call is always a MagicMock, regardless of the return value of the real call. So, even if a function returns None, like list.append, a mock is returned if calling the method from a mock, regardless of the spec.
I am trying to do unit testing for nested functions (function inside a function), I am using code("nested" is function name) from below link which would supply closures and returns a valid function that is callable from tests. It works for simple functions. I am trying to make it work for recursive functions.
As an example:
I am trying to get a valid function for "innerfunction" which has an entry in co_freevars as "innerfunction".
I want to get a function (FunctionType I believe) for "innerfunction" as a callable. To get this I need to call FunctionType with a tuple with a FunctionType for "innerfunction". This becomes a recursive "dependency". How can I resolve this dependency for parameters to be sent for "closures"
Function for
def toplevelfunction():
def innerfunction(a):
print ('val of a is ', a)
if a > 0:
innerfunction(a -1)
innerfunction(10)
Original code that I am using:
def freeVar(val):
def nested():
return val
return nested.__closure__[0]
codeAttribute = '__code__' if sys.version_info[0] == 3 else 'func_code'
def nested(outer, innerName, **freeVars):
if isinstance(outer, (types.FunctionType, types.MethodType)):
outer = outer.__getattribute__(codeAttribute)
for const in outer.co_consts:
if isinstance(const, types.CodeType) and const.co_name == innerName:
return types.FunctionType(const, globals(), None, None, tuple(
freeVar(freeVars[name]) for name in const.co_freevars))
https://code.activestate.com/recipes/580716-unit-testing-nested-functions/
How to add support for closures so below works:
func = nested(toplevelfunction, 'innerfunction')
func(5)
would return error need a closure of length 1.
Adding a closure referring to "const" shows up that it is of CodeType and not FunctionType.
Adding a closure value to refer itself seems tricky after reading through the documentation.
I do find innerfunction as:
{code} <code object innerfunction at 0x104b9bc00, file "/<filedirectory>/handle_results.py", line 44>
co_argcount = {int} 1
co_cellvars = {tuple} <class 'tuple'>: ()
co_code = {bytes} b"t\x00\x00d\x01\x00|\x00\x00\x83\x02\x00\x01|\x00\x00d\x02\x00k\x04\x00r'\x00\x88\x00\x00|\x00\x00d\x03\x00\x18\x83\x01\x00\x01d\x00\x00S"
co_consts = {tuple} <class 'tuple'>: (None, 'val of a is ', 0, 1)
co_filename = {str} '/<fildirectory>/handle_results.py'
co_firstlineno = {int} 44
co_flags = {int} 19
co_freevars = {tuple} <class 'tuple'>: ('innerfunction ',)
co_kwonlyargcount = {int} 0
co_lnotab = {bytes} b'\x00\x01\r\x01\x0c\x01'
co_name = {str} 'innerfunction '
co_names = {tuple} <class 'tuple'>: ('print',)
co_nlocals = {int} 1
co_stacksize = {int} 3
co_varnames = {tuple} <class 'tuple'>: ('a',)
Thank you for hinting me on this limitation on my ActiveState recipe you linked to in your question!
You tried to unit test a local recursive function; my minimal example for this looks like this:
def f(x):
def fac(n):
return fac(n-1) * n if n > 1 else 1
print "Faculty of", x, "is", fac(x)
So in a unit test you want to test the inner function fac(). Simply applying my recipe leads to this error:
nestedFac = nested(f, 'fac')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 7, in nested
File "<stdin>", line 7, in <genexpr>
KeyError: 'fac'
This is because fac is an identifier used within the inner function, and you must specify the value of each identifier used within the inner function. Callable identifiers are no exception:
nestedFac = nested(f, 'fac', fac=None)
This sets fac to the value None and the call to nested does not fail anymore.
Unfortunately, the resulting function will try to call None(...) when you call it:
nestedFac(5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in fac
TypeError: 'NoneType' object is not callable
You would need to pass the function itself as the value for fac to avoid this. Unfortunately, you do not have it yet when you call nested(). A repeated wrapping could be done like this:
nestedFac = nested(f, 'fac', fac=None)
nestedFac = nested(f, 'fac', fac=nestedFac)
nestedFac = nested(f, 'fac', fac=nestedFac)
nestedFac = nested(f, 'fac', fac=nestedFac)
This would limit your recursion depth to the number of wraps you applied:
nestedFac(2)
2
nestedFac(3)
6
nestedFac(4)
24
nestedFac(5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in fac
File "<stdin>", line 3, in fac
File "<stdin>", line 3, in fac
File "<stdin>", line 3, in fac
TypeError: 'NoneType' object is not callable
To avoid this, you can pass a local function as proxy:
def q(*args, **kwargs):
return nestedFac(*args, **kwargs)
nestedFac = nested(f, 'fac', fac=q)
Or as a lambda:
nestedFac = nested(f, 'fac', fac=lambda *args, **kwargs:
nestedFac(*args, **kwargs))
Or for the special case of just one parameter:
nestedFac = nested(f, 'fac', fac=lambda n: nestedFac(n))
An extension to the nested() recipe which does this automatically would be the better approach, though. Feel free to fork the recipe and add this aspect! :-)
One of my classes accumulates values in a list, uses the list as an argument to a method on another object and deletes some of the values in this list. Something like
element = element_source.get()
self.elements.append(element)
element_destination.send(elements)
self.remove_outdated_elements()
But when when i was trying to test this behavior, i've found that mocks don't copy their arguments.
>>> from unittest.mock import Mock
>>> m = Mock()
>>> a = [1]
>>> m(a)
<Mock name='mock()' id='139717658759824'>
>>> m.call_args
call([1])
>>> a.pop()
1
>>> m.assert_called_once_with([1])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.3/unittest/mock.py", line 737, in assert_called_once_with
return self.assert_called_with(*args, **kwargs)
File "/usr/lib/python3.3/unittest/mock.py", line 726, in assert_called_with
raise AssertionError(msg)
AssertionError: Expected call: mock([1])
Actual call: mock([])
Is there a way to make Mock copy it's call arguments? If not, what is the best way to test this kind of behavior?
There is a chapter "Coping with mutable arguments" in the documentation, which suggests several solutions to your problem.
I'd go with this one:
>>> from copy import deepcopy
>>> class CopyingMock(MagicMock):
... def __call__(self, *args, **kwargs):
... args = deepcopy(args)
... kwargs = deepcopy(kwargs)
... return super(CopyingMock, self).__call__(*args, **kwargs)
...
>>> c = CopyingMock(return_value=None)
>>> arg = set()
>>> c(arg)
>>> arg.add(1)
>>> c.assert_called_with(set())
>>> c.assert_called_with(arg)
Traceback (most recent call last):
...
AssertionError: Expected call: mock(set([1]))
Actual call: mock(set([]))
>>> c.foo
<CopyingMock name='mock.foo' id='...'>
I have a class which keeps track of errors encountered during a search operation
class SearchError(object):
def __init__(self,severity=0,message=''):
self.severity = severity
self.message = message
My idea is to make the instance variables indexable.
So if I have
a=SearchError(1,"Fatal Error")
I get
>>> a[0]
1
>>> a[1]
'Fatal Error'
>>> a.severity
1
>>> a.message
'Fatal Error'
To do this I add a __getitem__ method to the class. The class now becomes
class SearchError(object):
def __init__(self,severity=0,message=''):
self.severity = severity
self.message = message
def __getitem__(self,val):
if isinstance(val,slice):
return [self.__getitem__(i) for i in xrange(val.start,val.stop,val.step)]
elif val==0:
return self.severity
elif val==1:
return self.message
else:
raise IndexError
This does what I want but fails in cases such as
>>> a[:2]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 23, in __getitem__
TypeError: an integer is required
Or even
>>> a[-1]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 29, in __getitem__
IndexError
I understand my implementation of __getitem__ is limited. What I need to know is -
Is this the way to make instance variables indexable (Without using a list as variable container)?
How do I make the object behave 'sanely' as far as indexing goes?
This does everything:
>>> from collections import namedtuple
>>> _SearchError = namedtuple("SearchError", "severity message")
>>> def SearchError(severity=0, message=''):
return _SearchError(severity, message)
xrange requires all its arguments to be integers, but slice objects have None for unspecified attributes.
The best way to implement what you're after is to use namedtuple:
from collections import namedtuple
class SearchError(namedtuple('SearchError', 'severity message')):
def __new__(cls, severity=0, message=''):
return super(SearchError, cls).__new__(cls, severity, message)
The problem here is that slice objects default to having None values as attributes. So, a[:2] passes in slice(None,2,None). When you break this apart and try to pass it to xrange, you'll get a TypeError:
>>> xrange(None,2,None)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: an integer is required
Try a[0:2:1] and your code will work. Ultimately, you could do something like:
val.start = 0 if val.start is None else val.start
val.stop = 2 if val.stop is None else val.stop
val.stop = 2-val.stop if val.stop < 0 else val.stop
val.step = 1 if val.step is None else val.step
to unravel your slices into useable indices (In the general case, it'd be better to use len(self) instead of 2, but I don't know if your object has defined __len__.
Or, even better:
start,stop,step = val.indices(len(self))
Similarly, in the case where you do a[-1], you're not passing in a slice, a 0 or a 1, so you hit the else clause where you to raise an IndexError.
I mucked around the code and found the following solution.
It uses lists but to only store the names of the variables - Not the actual values. Additionally it also provides the method add to add a new variable with a given name and value. The new variable will also be indexable. (The add function is not needed by my class, but is nice to have around)
Thanks to #mgilson for nudging me in this direction
class SearchError(object):
def __init__(self,severity=0,message=''):
self.severity = severity
self.message = message
self._length = 2
self._vars_list = ['severity','message',]
def __getitem__(self,val):
if isinstance(val,slice):
steps = val.indices(self._length)
return [self.__getitem__(i) for i in xrange(*steps)]
elif val < 0:
i = self._length + val
if i < 0:
raise IndexError,"Index Out of range for SearchError object"
else:
return self.__getitem__(i)
else:
try:
return getattr(self,self._vars_list[val])
except IndexError:
raise IndexError,"Index Out of range for SearchError object"
def add(self,var_name,value):
self._vars_list.append(var_name)
self._length += 1
setattr(self,var_name,value)
The results
>>> a=SearchError(1,"Fatal Error")
>>> a[-1]
'Fatal Error'
>>> a[-2]
1
>>> a[-3]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 14, in __getitem__
IndexError: Index Out of range for SearchError object
>>> a[2]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 21, in __getitem__
IndexError: Index Out of range for SearchError object
>>> a[1]
'Fatal Error'
>>> a[0]
1
>>> a[:]
[1, 'Fatal Error']
>>> a.add('new_severity',8)
>>> a[:]
[1, 'Fatal Error', 8]
>>> a[3]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 21, in __getitem__
IndexError: Index Out of range for SearchError object
>>> a[2]
8
>>> a.new_severity
8
>>> a[:3]
[1, 'Fatal Error', 8]
>>> a[:4]
[1, 'Fatal Error', 8]
>>> a[:2]
[1, 'Fatal Error']
As far as I can see, you need lists (to either store the actual variables or their names). If someone has a better alternative please do post