Develop Reference

How to get the arguments from a method call on a mocked object?

I have a unit test where the setup mocks a client like so:
def setUp(self):
self.mock_client = mock.patch.object(module_name, 'ClientClassName', autospec=True).start()
Then in my test I have a faked return value:
def myTest(self):
self.mock_client.my_method.return_value = ...
Now I want to get the arguments that my_method was called with, however I've been tearing my hair out trying to access them. It seems that I can't just do:
mock_args, mock_kwargs = self.mock_client.my_method.call_args
This gives me back First off why doesn't this work? I did make a little headway and found that:
self.mock_client.method_calls[0]
will give me back a call object that looks like call().my_method(...the arguments), but I have been trying for hours to get access to the individual arguments and cant seem to do it. Where am I going wrong?

Call args are just accessed with subscription on the mock.call object, i.e. __getitem__.
>>> from unittest.mock import MagicMock
>>> m = MagicMock()
>>> m(123, xyz="hello")
<MagicMock name='mock()' id='140736989479888'>
>>> m("another call")
<MagicMock name='mock()' id='140736989479888'>
>>> m.call_args_list
[call(123, xyz='hello'), call('another call')]
>>> m.call_args_list[0][0]
(123,)
>>> m.call_args_list[0][1]
{'xyz': 'hello'}
Item 0 will be a tuple of args, and item 1 will be a dict of kwargs. Attribute access also works, like a namedtuple (item 0 is attribute "args", and item 1 is attribute "kwargs"). If you only need to access the most recent call, you can use call_args instead of call_args_list.
Note that accessing the call args items directly is usually not required, you can use an assertion against another call instance in the tests:
>>> from unittest.mock import call
>>> m(k=123)
<MagicMock name='mock()' id='140736989479888'>
>>> assert m.call_args == call(k=123) # pass
>>> assert m.call_args == call(k=124) # fail
AssertionError
Or an even higher level, you can use m.assert_has_calls on the mock directly.
When mocking methods, whether the mock calls have self or not can be influenced by autospec:
>>> from unittest import mock
>>> class A(object):
... def f(self, *args, **kwargs):
... pass
...
>>> with mock.patch("__main__.A.f") as m:
... a = A()
... a.f('without autospec', n=1)
...
>>> m.call_args
call('without autospec', n=1)
>>> with mock.patch("__main__.A.f", autospec=True) as m:
... a = A()
... a.f('with autospec', n=2)
...
>>> m.call_args
call(<__main__.A object at 0x7fffe3d4e6a0>, 'with autospec', n=2)
This is discussed in more detail in the docs here.

In addition to wim's answer, you can actually dig down fairly deeply into this arguments, although sometimes you seem to find a string instead of a real object.
The main point to understand is that when you iterate through my_mock.call_args_list you get objects of type unittest.mock._Call. These can indeed be compared to call(...) objects which you have created yourself. But that's not all.
unittest.mock._Call is itself iterable, and consists of 2 elements: one is a tuple, the other is a dict. These are none other than the *args and **kwargs passed to the mock method.
Given that this is, as far as I can make out, completely undocumented in the docs, I suppose it is not beyond the bounds of possibility that this could break one day. It does often prove handy to know though, in my experience.

can I know if a function is called directly or via an assigned variable

Assuming I have a function def f(): print 0 and I want to create another function g, however g's implementation is identical to f, so I just assign f to it:
def f():
print 0
g = f
you might ask, if they are identical, why use 2 functions, and that is because logically, they represent 2 different things
Now, can I know who was called? I want to debug f() only when it is called via g. Is the flow different between this 2 calls?
The dup suggestion has nothing to do with my question. I know the function's name, as only one function exists. I just hope to get one step further and understand if a variable that has the same value as the function name was triggered...

No, you can't, at least not this way, as Python does not track "which reference to the object is looked". I strongly recommend that you make g another function that wraps up f, for a less painful debugging experience.
However, if your code follows some patterns, you may have a chance by looking at inspect.stack():
import inspect
def f():
stack = inspect.stack()
print(stack[1].code_context[0].strip())
g = f
f()
g()
Luckily, you'll get
f()
g()
If you have some regex skills, you can actually extract the function name with one:
import re
func_name_re = re.compile(r"\b(\w+)\s*\()")
line = "f()"
print(func_name_re.search(line).group(1))
However, you'll easily run into trouble identifying the real function if your calling code gets marginally more complex:
some = f(g())
some[f()] = g()
some = f("aaa g() sss") # string content looks like function call
some = re.compile(r"^f(?!$)", flags=g()) # same as above
In all the above examples, you have significant difficulty trying to determine if it's f or g.

This won't work.
input:
def f():
pass
g = f
print(f.__name__)
print(g.__name__)
print(id(f))
print(id(g))
output:
f
f
4350036880
4350036880
Only workaround I can think of would be creating some type of wrapper around the functions that calls eval and handling that way(but most likely doesn't make sense for what you will want to do). Then you could call that wrapper with a func_name argument.
func_name = 'f'
if func_name =='f':
## you know f was called
eval('f()')

There is no way to do that without additional helpers or code analysis.
Even the stack is not aware of the called name:
>>> def a(): traceback.print_stack()
...
>>> b = a
>>> a()
File "<stdin>", line 1, in <module>
File "<stdin>", line 1, in a
>>> b()
File "<stdin>", line 1, in <module>
File "<stdin>", line 1, in a
>>>

May I suggest the simplest solution ?
def f():
print 0
def g():
return f()
Now you know clearly when f was called via g - you just have to inspect the call stack.

How can I get an accurate representation of a function/lambda? [duplicate]

Suppose I have a Python function as defined below:
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
I can get the name of the function using foo.func_name. How can I programmatically get its source code, as I typed above?

If the function is from a source file available on the filesystem, then inspect.getsource(foo) might be of help:
If foo is defined as:
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
Then:
import inspect
lines = inspect.getsource(foo)
print(lines)
Returns:
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
But I believe that if the function is compiled from a string, stream or imported from a compiled file, then you cannot retrieve its source code.

The inspect module has methods for retrieving source code from python objects. Seemingly it only works if the source is located in a file though. If you had that I guess you wouldn't need to get the source from the object.
The following tests inspect.getsource(foo) using Python 3.6:
import inspect
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
source_foo = inspect.getsource(foo) # foo is normal function
print(source_foo)
source_max = inspect.getsource(max) # max is a built-in function
print(source_max)
This first prints:
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
Then fails on inspect.getsource(max) with the following error:
TypeError: <built-in function max> is not a module, class, method, function, traceback, frame, or code object

Just use foo?? or ??foo.
If you are using IPython, then you need to type foo?? or ??foo to see the complete source code. To see only the docstring in the function, use foo? or ?foo. This works in Jupyter notebook as well.
In [19]: foo??
Signature: foo(arg1, arg2)
Source:
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
File: ~/Desktop/<ipython-input-18-3174e3126506>
Type: function

dis is your friend if the source code is not available:
>>> import dis
>>> def foo(arg1,arg2):
... #do something with args
... a = arg1 + arg2
... return a
...
>>> dis.dis(foo)
3 0 LOAD_FAST 0 (arg1)
3 LOAD_FAST 1 (arg2)
6 BINARY_ADD
7 STORE_FAST 2 (a)
4 10 LOAD_FAST 2 (a)
13 RETURN_VALUE

While I'd generally agree that inspect is a good answer, I'd disagree that you can't get the source code of objects defined in the interpreter. If you use dill.source.getsource from dill, you can get the source of functions and lambdas, even if they are defined interactively.
It also can get the code for from bound or unbound class methods and functions defined in curries... however, you might not be able to compile that code without the enclosing object's code.
>>> from dill.source import getsource
>>>
>>> def add(x,y):
... return x+y
...
>>> squared = lambda x:x**2
>>>
>>> print getsource(add)
def add(x,y):
return x+y
>>> print getsource(squared)
squared = lambda x:x**2
>>>
>>> class Foo(object):
... def bar(self, x):
... return x*x+x
...
>>> f = Foo()
>>>
>>> print getsource(f.bar)
def bar(self, x):
return x*x+x
>>>

To expand on runeh's answer:
>>> def foo(a):
... x = 2
... return x + a
>>> import inspect
>>> inspect.getsource(foo)
u'def foo(a):\n x = 2\n return x + a\n'
print inspect.getsource(foo)
def foo(a):
x = 2
return x + a
EDIT: As pointed out by #0sh this example works using ipython but not plain python. It should be fine in both, however, when importing code from source files.

Since this post is marked as the duplicate of this other post, I answer here for the "lambda" case, although the OP is not about lambdas.
So, for lambda functions that are not defined in their own lines: in addition to marko.ristin's answer, you may wish to use mini-lambda or use SymPy as suggested in this answer.
mini-lambda is lighter and supports any kind of operation, but works only for a single variable
SymPy is heavier but much more equipped with mathematical/calculus operations. In particular it can simplify your expressions. It also supports several variables in the same expression.
Here is how you can do it using mini-lambda:
from mini_lambda import x, is_mini_lambda_expr
import inspect
def get_source_code_str(f):
if is_mini_lambda_expr(f):
return f.to_string()
else:
return inspect.getsource(f)
# test it
def foo(arg1, arg2):
# do something with args
a = arg1 + arg2
return a
print(get_source_code_str(foo))
print(get_source_code_str(x ** 2))
It correctly yields
def foo(arg1, arg2):
# do something with args
a = arg1 + arg2
return a
x ** 2
See mini-lambda documentation for details. I'm the author by the way ;)

You can use inspect module to get full source code for that. You have to use getsource() method for that from the inspect module. For example:
import inspect
def get_my_code():
x = "abcd"
return x
print(inspect.getsource(get_my_code))
You can check it out more options on the below link.
retrieve your python code

to summarize :
import inspect
print( "".join(inspect.getsourcelines(foo)[0]))

Please mind that the accepted answers work only if the lambda is given on a separate line. If you pass it in as an argument to a function and would like to retrieve the code of the lambda as object, the problem gets a bit tricky since inspect will give you the whole line.
For example, consider a file test.py:
import inspect
def main():
x, f = 3, lambda a: a + 1
print(inspect.getsource(f))
if __name__ == "__main__":
main()
Executing it gives you (mind the indention!):
x, f = 3, lambda a: a + 1
To retrieve the source code of the lambda, your best bet, in my opinion, is to re-parse the whole source file (by using f.__code__.co_filename) and match the lambda AST node by the line number and its context.
We had to do precisely that in our design-by-contract library icontract since we had to parse the lambda functions we pass in as arguments to decorators. It is too much code to paste here, so have a look at the implementation of this function.

If you're strictly defining the function yourself and it's a relatively short definition, a solution without dependencies would be to define the function in a string and assign the eval() of the expression to your function.
E.g.
funcstring = 'lambda x: x> 5'
func = eval(funcstring)
then optionally to attach the original code to the function:
func.source = funcstring

Rafał Dowgird's answer states:
I believe that if the function is compiled from a string, stream or imported from a compiled file, then you cannot retrieve its source code.
However, it is possible to retrieve the source code of a function compiled from a string, provided that the compiling code also added an entry to the linecache.cache dict:
import linecache
import inspect
script = '''
def add_nums(a, b):
return a + b
'''
bytecode = compile(script, 'unique_filename', 'exec')
tmp = {}
eval(bytecode, {}, tmp)
add_nums = tmp["add_nums"]
linecache.cache['unique_filename'] = (
len(script),
None,
script.splitlines(True),
'unique_filename',
)
print(inspect.getsource(add_nums))
# prints:
# """
# def add_nums(a, b):
# return a + b
# """
This is how the attrs library creates various methods for classes automatically, given a set of attributes that the class expects to be initialized with. See their source code here. As the source explains, this is a feature primarily intended to enable debuggers such as PDB to step through the code.

I believe that variable names aren't stored in pyc/pyd/pyo files, so you can not retrieve the exact code lines if you don't have source files.

how to partially apply arbitrary argument of a function?

I want to use partial from functools to partially apply a function's second argument, I know it is easy to do with lambda rather than partial as follows
>>> def func1(a,b):
... return a/b
...
>>> func2 = lambda x:func1(x,2)
>>> func2(4)
2
but I strictly want to use partial here (for the sake of learning) so i came up with this.
>>> def swap_binary_args(func):
... return lambda x,y: func(y,x)
...
>>> func3 = partial(swap_binary_args(func1),2)
>>> func3(4)
2
Is it possible to extend this strategy to a level where I can partial apply any arguments at any place like in the following pseudocode
>>>def indexed_partial(func, list_of_index, *args):
... ###do_something###
... return partially_applied_function
>>>func5=indexed_partial(func1, [1,4,3,5], 2,4,5,6)
in our case I can use this function as follows
>>>func6=indexed_partial(func1, [1], 2)
Is it possible to have an indexed partial like I want ? is there anything similar to this already which I am not aware of ? and more importantly is the idea of indexed partial generally a good or bad idea why ?
This question has been marked as possible duplicate of Can one partially apply the second argument of a function that takes no keyword arguments?
in that question the OP asked is it possible to partially apply second argument but here i am asking how to cook a function that can partially apply any arbitrary argument

I, too, think what you ask can't be done (easily?) with functools.partial. Probably the best (and most readable) solution is to use partial with keyword-arguments.
However, in case you want to use positional arguments (and hence indexed partial arguments), here is a possible definition of indexed_partial:
def indexed_partial(func, list_of_index, *args):
def partially_applied_function(*fargs, **fkwargs):
nargs = len(args) + len(fargs)
iargs = iter(args)
ifargs = iter(fargs)
posargs = ((ifargs, iargs)[i in list_of_index].next() for i in range(nargs))
return func(*posargs, **fkwargs)
return partially_applied_function

decorating a function and adding functionalities preserving the number of argument

I'd like to decorate a function, using a pattern like this:
def deco(func):
def wrap(*a,**kw):
print "do something"
return func(*a,**kw)
return wrap
The problem is that if the function decorated has a prototype like that:
def function(a,b,c): return
When decorated, the prototype is destroyed by the varargs, for example, calling function(1,2,3,4) wouldn't result in an exception. Is that a way to avoid that?
How can define the wrap function with the same prototype as the decorated (func) one?
There's something conceptually wrong?
EDIT
My perverse idea was to lighten the "calling of the parent method" without modifying the signature. Something like
def __init__(self, something)
super(ClassName, self).__init__(something)
to:
#extended
def __init__(self, something):
...
I was figuring out if this was possible and if this makes sense.
EDIT
As Alex pointed out, the following code doesn't give an exception:
function(1,2,3,4)

You're wrong when you state that "calling function(1,2,3,4) wouldn't result in an exception". Check it out:
>>> def deco(f):
... def w(*a, **k):
... print 'do something'
... return f(*a, **k)
... return w
...
>>> def f(a, b, c): return
...
>>> f(1, 2, 3, 4)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: f() takes exactly 3 arguments (4 given)
>>> decorated = deco(f)
>>> decorated(1, 2, 3, 4)
do something
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 4, in w
TypeError: f() takes exactly 3 arguments (4 given)
As you see, you get exactly the same exception as when similarly calling the undecorated f (albeit after the print you've added).
For preserving the wrapped function's metadata (name, docstring, use functools.wraps. Predicting that the wrapped function will raise when called (to avoid doing other work before calling it) is always hard ("impossible" in general, as it's equivalent to the halting problem; "just hard" in specific, when you only care about raising a type-error for argument name and number mismatches and want to treat that specialized exception case differently from any other exception case -- a peculiar requirement;-0).
If you're adamant that you absolutely need it (and can perhaps explain why?) I'll be glad (well, not glad, but will grit my teeth and do it, after all I've got a long weekend in front of me;-) to take you down that labyrinthine path.

The decorator module helps you create a decorator that preserves the function signature.
As a result, you will get the exception you expect when calling the function, and inspect.getargspec will give you the correct signature.
It works by dynamically building a function definition and using exec. Unfortunately, there isn't an easier built-in way to do this.

Here is a trick that involves fetching the original argument specification from the decorated function, then creating a lambda by evaling a string with the same arguments. The decorator is then wrapped in this lambda thus to the outside world it has the same parameter names and default values:
import inspect, time
import functools
def decorator_wrapper(old_function, new_function):
args, arglist, kw, default = inspect.getargspec(old_function)
args = list(args)
if arglist:
args.append(arglist)
if kw:
args.append(kw)
callstring = inspect.formatargspec(args, arglist, kw, default, formatvalue=lambda value: "")
argstring = inspect.formatargspec(args, arglist, kw, default)[1:-1]
unique_name = "_func" + str(int(time.time()))
codestring = "lambda " + argstring + " : " + unique_name + callstring
decorated_function = eval(codestring, {unique_name: new_function})
return functools.wraps(old_function)(decorated_function)

By calling result=func(*a,**kw) first, you get the TypeError before printing "do something".
def deco(func):
def wrap(*a,**kw):
result=func(*a,**kw)
print "do something"
return result
return wrap
#deco
def function(a,b,c): return
function(1,2,3)
# do something
function(1,2,3,4)
# TypeError: function() takes exactly 3 arguments (4 given)