Develop Reference

Can lambda work with *args as its parameter? [duplicate]

This question already has answers here:
Function chaining in Python
(6 answers)
Closed 6 years ago.
I am calculating a sum using lambda like this:
def my_func(*args):
return reduce((lambda x, y: x + y), args)
my_func(1,2,3,4)
and its output is 10.
But I want a lambda function that takes random arguments and sums all of them. Suppose this is a lambda function:
add = lambda *args://code for adding all of args
someone should be able to call the add function as:
add(5)(10) # it should output 15
add(1)(15)(20)(4) # it should output 40
That is, one should be able to supply arbitrary
number of parenthesis.
Is this possible in Python?

This is not possible with lambda, but it is definitely possible to do this is Python.
To achieve this behaviour you can subclass int and override its __call__ method to return a new instance of the same class with updated value each time:
class Add(int):
def __call__(self, val):
return type(self)(self + val)
Demo:
>>> Add(5)(10)
15
>>> Add(5)(10)(15)
30
>>> Add(5)
5
# Can be used to perform other arithmetic operations as well
>>> Add(5)(10)(15) * 100
3000
If you want to support floats as well then subclass from float instead of int.

The sort of "currying" you're looking for is not possible.
Imagine that add(5)(10) is 15. In that case, add(5)(10)(20) needs to be equivalent to 15(20). But 15 is not callable, and in particular is not the same thing as the "add 15" operation.
You can certainly say lambda *args: sum(args), but you would need to pass that its arguments in the usual way: add(5,10,20,93)
[EDITED to add:] There are languages in which functions with multiple arguments are handled in this sort of way; Haskell, for instance. But those are functions with a fixed number of multiple arguments, and the whole advantage of doing it that way is that if e.g. add 3 4 is 7 then add 3 is a function that adds 3 to things -- which is exactly the behaviour you're wanting not to get, if you want something like this to take a variable number of arguments.
For a function of fixed arity you can get Haskell-ish behaviour, though the syntax doesn't work so nicely in Python, just by nesting lambdas: after add = lambda x: lambda y: x+y you can say add(3)(4) and get 7, or you can say add(3) and get a function that adds 3 to things.
[EDITED again to add:] As Ashwini Chaudhary's ingenious answer shows, you actually can kinda do what you want by arranging for add(5)(10) to be not the actual integer 15 but another object that very closely resembles 15 (and will just get displayed as 15 in most contexts). For me, this is firmly in the category of "neat tricks you should know about but never ever actually do", but if you have an application that really needs this sort of behaviour, that's one way to do it.
(Why shouldn't you do this sort of thing? Mostly because it's brittle and liable to produce unexpected results in edge cases. For instance, what happens if you ask for add(5)(10.5)? That will fail with A.C.'s approach; PM 2Ring's approach will cope OK with that but has different problems; e.g., add(2)(3)==5 will be False. The other reason to avoid this sort of thing is because it's ingenious and rather obscure, and therefore liable to confuse other people reading your code. How much this matters depends on who else will be reading your code. I should add for the avoidance of doubt that I'm quite sure A.C. and PM2R are well aware of this, and that I think their answers are very clever and elegant; I am not criticizing them but offering a warning about what to do with what they've told you.)

You can kind of do this with a class, but I really wouldn't advise using this "party trick" in real code.
class add(object):
def __init__(self, arg):
self.arg = arg
def __call__(self, arg):
self.arg += arg
return self
def __repr__(self):
return repr(self.arg)
# Test
print(add(1)(15)(20)(4))
output
40
Initially, add(1) creates an add instance, setting its .args attribute to 1. add(1)(15) invokes the .call method, adding 15 to the current value of .args and returning the instance so we can call it again. This same process is repeated for the subsequent calls. Finally, when the instance is passed to print its __repr__ method is invoked, which passes the string representation of .args back to print.

How to pass tuple to function as separate variables

When returning multiple variables from a Python function, returning a tuple is the most convenient solution to me. However, it lacks convenience when chaining the return value with another function taking separate variables as input. For example, say we have
def foo():
return (1,2)
def sum(a, b):
return a+b
And in this case, the following chaining doesn't work
sum(foo())
I know one solution, namely to change the function sum to take a tuple as input rather than 2 vars, i.e.
def sum2((a, b)):
return a+b
Now the chaining sum2(foo()) works, but it's a bit awkward, isn't it? I wound rather change foo() because sum() might has been used by others. What is a nice solution to this?

You are looking for unpacking. You should try -
sum(*foo())

explicitly passing functions in python

Out of curiosity is more desirable to explicitly pass functions to other functions, or let the function call functions from within. is this a case of Explicit is better than implicit?
for example (the following is only to illustrate what i mean)
def foo(x,y):
return 1 if x > y else 0
partialfun = functools.partial(foo, 1)
def bar(xs,ys):
return partialfun(sum(map(operator.mul,xs,ys)))
>>> bar([1,2,3], [4,5,6])
--or--
def foo(x,y):
return 1 if x > y else 0
partialfun = functools.partial(foo, 1)
def bar(fn,xs,ys):
return fn(sum(map(operator.mul,xs,ys)))
>>> bar(partialfun, [1,2,3], [4,5,6])

There's not really any difference between functions and anything else in this situation. You pass something as an argument if it's a parameter that might vary over different invocations of the function. If the function you are calling (bar in your example) is always calling the same other function, there's no reason to pass that as an argument. If you need to parameterize it so that you can use many different functions (i.e., bar might need to call many functions besides partialfun, and needs to know which one to call), then you need to pass it as an argument.

Generally, yes, but as always, it depends. What you are illustrating here is known as dependency injection. Generally, it is a good idea, as it allows separation of variability from the logic of a given function. This means, for example, that it will be extremely easy for you to test such code.
# To test the process performed in bar(), we can "inject" a function
# which simply returns its argument
def dummy(x):
return x
def bar(fn,xs,ys):
return fn(sum(map(operator.mul,xs,ys)))
>>> assert bar(dummy, [1,2,3], [4,5,6]) == 32

It depends very much on the context.
Basically, if the function is an argument to bar, then it's the responsibility of the caller to know how to implement that function. bar doesn't have to care. But consequently, bar's documentation has to describe what kind of function it needs.
Often this is very appropriate. The obvious example is the map builtin function. map implements the logic of applying a function to each item in a list, and giving back a list of results. map itself neither knows nor cares about what the items are, or what the function is doing to them. map's documentation has to describe that it needs a function of one argument, and each caller of map has to know how to implement or find a suitable function. But this arrangement is great; it allows you to pass a list of your custom objects, and a function which operates specifically on those objects, and map can go away and do its generic thing.
But often this arrangement is inappropriate. A function gives a name to a high level operation and hides the internal implementation details, so you can think of the operation as a unit. Allowing part of its operation to be passed in from outside as a function parameter exposes that it works in a way that uses that function's interface.
A more concrete (though somewhat contrived) example may help. Lets say I've implemented data types representing Person and Job, and I'm writing a function name_and_title for formatting someone's full name and job title into a string, for client code to insert into email signatures or on letterhead or whatever. It's obviously going to take a Person and Job. It could potentially take a function parameter to let the caller decide how to format the person's name: something like lambda firstname, lastname: lastname + ', ' + firstname. But to do this is to expose that I'm representing people's names with a separate first name and last name. If I want to change to supporting a middle name, then either name_and_title won't be able to include the middle name, or I have to change the type of the function it accepts. When I realise that some people have 4 or more names and decide to change to storing a list of names, then I definitely have to change the type of function name_and_title accepts.
So for your bar example, we can't say which is better, because it's an abstract example with no meaning. It depends on whether the call to partialfun is an implementation detail of whatever bar is supposed to be doing, or whether the call to partialfun is something that the caller knows about (and might want to do something else). If it's "part of" bar, then it shouldn't be a parameter. If it's "part of" the caller, then it should be a parameter.
It's worth noting that bar could have a huge number of function parameters. You call sum, map, and operator.mul, which could all be parameterised to make bar more flexible:
def bar(fn, xs,ys, g, h, i):
return fn(g(h(i,xs,ys))
And the way in which g is called on the output of h could be abstracted too:
def bar(fn, xs, ys, g, h, i, j):
return fn(j(g, h(i, xs, ys)))
And we can keep going on and on, until bar doesn't do anything at all, and everything is controlled by the functions passed in, and the caller might as well have just directly done what they want done rather than writing 100 functions to do it and passing those to bar to execute the functions.
So there really isn't a definite answer one way or the other that applies all the time. It depends on the particular code you're writing.

Defining "overloaded" functions in python

I really like the syntax of the "magic methods" or whatever they are called in Python, like
class foo:
def __add__(self,other): #It can be called like c = a + b
pass
The call
c = a + b
is then translated to
a.__add__(b)
Is it possible to mimic such behaviour for "non-magic" functions? In numerical computations I need the Kronecker product, and am eager to have "kron" function such that
kron(a,b)
is in fact
a.kron(b)?
The use case is: I have two similar classes, say, matrix and vector, both having Kronecker product. I would like to call them
a = matrix()
b = matrix()
c = kron(a,b)
a = vector()
b = vector()
c = kron(a,b)
matrix and vector classes are defined in one .py file, thus share the common namespace. So, what is the best (Pythonic?) way to implement functions like above? Possible solutions:
1) Have one kron() functions and do type check
2) Have different namespaces
3) ?

The python default operator methods (__add__ and such) are hard-wired; python will look for them because the operator implementations look for them.
However, there is nothing stopping you from defining a kron function that does the same thing; look for __kron__ or __rkron__ on the objects passed to it:
def kron(a, b):
if hasattr(a, '__kron__'):
return a.__kron__(b)
if hasattr(b, '__rkron__'):
return b.__rkron__(a)
# Default kron implementation here
return complex_operation_on_a_and_b(a, b)

What you're describing is multiple dispatch or multimethods. Magic methods is one way to implement them, but it's actually more usual to have an object that you can register type-specific implementations on.
For example, http://pypi.python.org/pypi/multimethod/ will let you write
#multimethod(matrix, matrix)
def kron(lhs, rhs):
pass
#multimethod(vector, vector)
def kron(lhs, rhs):
pass
It's quite easy to write a multimethod decorator yourself; the BDFL describes a typical implementation in an article. The idea is that the multimethod decorator associates the type signature and method with the method name in a registry, and replaces the method with a generated method that performs type lookup to find the best match.

Technically speaking, implementing something similar to the "standard" operator (and operator-like - think len() etc) behaviour is not difficult:
def kron(a, b):
if hasattr(a, '__kron__'):
return a.__kron__(b)
elif hasattr(b, '__kron__'):
return b.__kron__(a)
else:
raise TypeError("your error message here")
Now you just have to add a __kron__(self, other) method on the relevant types (assuming you have control over these types or they don't use slots or whatever else that would prevent adding methods outside the class statement's body).
Now I'd not use a __magic__ naming scheme as in my above snippet since this is supposed to be reserved for the language itself.
Another solution would be to maintain a type:specifici function mapping and have the "generic" kron function looking up the mapping, ie:
# kron.py
from somewhere import Matrix, Vector
def matrix_kron(a, b):
# code here
def vector_kron(a, b):
# code here
KRON_IMPLEMENTATIONS = dict(
Matrix=matrix_kron,
Vector=vector_kron,
)
def kron(a, b):
for typ in (type(a), type(b)):
implementation = KRON_IMPLEMENTATION.get(typ, None)
if implementation:
return implementation(a, b)
else:
raise TypeError("your message here")
This solution doesn't work well with inheritance but it "less surprinsing" - doesn't require monkeypatching nor __magic__ name etc.

I think having one single function that delegate the actual computation is a nice way to do it. If the Kronecker product only works on two similar classes, you can even do the type checking in the function :
def kron(a, b):
if type(a) != type(b):
raise TypeError('expected two instances of the same class, got %s and %s'%(type(a), type(b)))
return a._kron_(b)
Then, you just need to define a _kron_ method on the class. This is only some basic example, you might want to improve it to handle more gracefully the cases where a class doesn't have the _kron_ method, or to handle subclasses.
Binary operations in the standart libary usually have a reverse dual (__add__ and __radd__), but since your operator only work for same type objects, it isn't useful here.

Is nested function a good approach when required by only one function? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
Let's say that a function A is required only by function B, should A be defined inside B?
Simple example. Two methods, one called from another:
def method_a(arg):
some_data = method_b(arg)
def method_b(arg):
return some_data
In Python we can declare def inside another def. So, if method_b is required for and called only from method_a, should I declare method_b inside method_a? like this :
def method_a(arg):
def method_b(arg):
return some_data
some_data = method_b(arg)
Or should I avoid doing this?

>>> def sum(x, y):
... def do_it():
... return x + y
... return do_it
...
>>> a = sum(1, 3)
>>> a
<function do_it at 0xb772b304>
>>> a()
4
Is this what you were looking for? It's called a closure.

You don't really gain much by doing this, in fact it slows method_a down because it'll define and recompile the other function every time it's called. Given that, it would probably be better to just prefix the function name with underscore to indicate it's a private method -- i.e. _method_b.
I suppose you might want to do this if the nested function's definition varied each time for some reason, but that may indicate a flaw in your design. That said, there is a valid reason to do this to allow the nested function to use arguments that were passed to the outer function but not explicitly passed on to them, which sometimes occurs when writing function decorators, for example. It's what is being shown in the accepted answer although a decorator is not being defined or used.
Update:
Here's proof that nesting them is slower (using Python 3.6.1), although admittedly not by much in this trivial case:
setup = """
class Test(object):
def separate(self, arg):
some_data = self._method_b(arg)
def _method_b(self, arg):
return arg+1
def nested(self, arg):
def method_b2(self, arg):
return arg+1
some_data = method_b2(self, arg)
obj = Test()
"""
from timeit import Timer
print(min(Timer(stmt='obj.separate(42)', setup=setup).repeat())) # -> 0.24479823284461724
print(min(Timer(stmt='obj.nested(42)', setup=setup).repeat())) # -> 0.26553459700452575
Note I added some self arguments to your sample functions to make them more like real methods (although method_b2 still isn't technically a method of the Test class). Also the nested function is actually called in that version, unlike yours.

Generally, no, do not define functions inside functions.
Unless you have a really good reason. Which you don't.
Why not?
It prevents easy hooks for unit testing. You are unit testing, aren't you?
It doesn't actually obfuscate it completely anyway, it's safer to assume nothing in python ever is.
Use standard Python automagic code style guidelines to encapsulate methods instead.
You will be needlessly recreating a function object for the identical code every single time you run the outer function.
If your function is really that simple, you should be using a lambda expression instead.
What is a really good reason to define functions inside functions?
When what you actually want is a dingdang closure.

A function inside of a function is commonly used for closures.
(There is a lot of contention over what exactly makes a closure a closure.)
Here's an example using the built-in sum(). It defines start once and uses it from then on:
def sum_partial(start):
def sum_start(iterable):
return sum(iterable, start)
return sum_start
In use:
>>> sum_with_1 = sum_partial(1)
>>> sum_with_3 = sum_partial(3)
>>>
>>> sum_with_1
<function sum_start at 0x7f3726e70b90>
>>> sum_with_3
<function sum_start at 0x7f3726e70c08>
>>> sum_with_1((1,2,3))
7
>>> sum_with_3((1,2,3))
9
Built-in python closure
functools.partial is an example of a closure.
From the python docs, it's roughly equivalent to:
def partial(func, *args, **keywords):
def newfunc(*fargs, **fkeywords):
newkeywords = keywords.copy()
newkeywords.update(fkeywords)
return func(*(args + fargs), **newkeywords)
newfunc.func = func
newfunc.args = args
newfunc.keywords = keywords
return newfunc
(Kudos to #user225312 below for the answer. I find this example easier to figure out, and hopefully will help answer #mango's comment.)

It's actually fine to declare one function inside another one. This is specially useful creating decorators.
However, as a rule of thumb, if the function is complex (more than 10 lines) it might be a better idea to declare it on the module level.

I found this question because I wanted to pose a question why there is a performance impact if one uses nested functions. I ran tests for the following functions using Python 3.2.5 on a Windows Notebook with a Quad Core 2.5 GHz Intel i5-2530M processor
def square0(x):
return x*x
def square1(x):
def dummy(y):
return y*y
return x*x
def square2(x):
def dummy1(y):
return y*y
def dummy2(y):
return y*y
return x*x
def square5(x):
def dummy1(y):
return y*y
def dummy2(y):
return y*y
def dummy3(y):
return y*y
def dummy4(y):
return y*y
def dummy5(y):
return y*y
return x*x
I measured the following 20 times, also for square1, square2, and square5:
s=0
for i in range(10**6):
s+=square0(i)
and got the following results
>>>
m = mean, s = standard deviation, m0 = mean of first testcase
[m-3s,m+3s] is a 0.997 confidence interval if normal distributed
square? m s m/m0 [m-3s ,m+3s ]
square0 0.387 0.01515 1.000 [0.342,0.433]
square1 0.460 0.01422 1.188 [0.417,0.503]
square2 0.552 0.01803 1.425 [0.498,0.606]
square5 0.766 0.01654 1.979 [0.717,0.816]
>>>
square0 has no nested function, square1 has one nested function, square2 has two nested functions and square5 has five nested functions. The nested functions are only declared but not called.
So if you have defined 5 nested funtions in a function that you don't call then the execution time of the function is twice of the function without a nested function. I think should be cautious when using nested functions.
The Python file for the whole test that generates this output can be found at ideone.

So in the end it is largely a question about how smart the python implementation is or is not, particularly in the case of the inner function not being a closure but simply an in function needed helper only.
In clean understandable design having functions only where they are needed and not exposed elsewhere is good design whether they be embedded in a module, a class as a method, or inside another function or method. When done well they really improve the clarity of the code.
And when the inner function is a closure that can also help with clarity quite a bit even if that function is not returned out of the containing function for use elsewhere.
So I would say generally do use them but be aware of the possible performance hit when you actually are concerned about performance and only remove them if you do actual profiling that shows they best be removed.
Do not do premature optimization of just using "inner functions BAD" throughout all python code you write. Please.

It's just a principle about exposure APIs.
Using python, It's a good idea to avoid exposure API in outer space(module or class), function is a good encapsulation place.
It could be a good idea. when you ensure
inner function is ONLY used by outer function.
insider function has a good name to explain its purpose because the code talks.
code cannot directly understand by your colleagues(or other code-reader).
Even though, Abuse this technique may cause problems and implies a design flaw.
Just from my exp, Maybe misunderstand your question.

It's perfectly OK doing it that way, but unless you need to use a closure or return the function I'd probably put in the module level. I imagine in the second code example you mean:
...
some_data = method_b() # not some_data = method_b
otherwise, some_data will be the function.
Having it at the module level will allow other functions to use method_b() and if you're using something like Sphinx (and autodoc) for documentation, it will allow you to document method_b as well.
You also may want to consider just putting the functionality in two methods in a class if you're doing something that can be representable by an object. This contains logic well too if that's all you're looking for.

You can use it to avoid defining global variables. This gives you an alternative for other designs. 3 designs presenting a solution to a problem.
A) Using functions without globals
def calculate_salary(employee, list_with_all_employees):
x = _calculate_tax(list_with_all_employees)
# some other calculations done to x
pass
y = # something
return y
def _calculate_tax(list_with_all_employees):
return 1.23456 # return something
B) Using functions with globals
_list_with_all_employees = None
def calculate_salary(employee, list_with_all_employees):
global _list_with_all_employees
_list_with_all_employees = list_with_all_employees
x = _calculate_tax()
# some other calculations done to x
pass
y = # something
return y
def _calculate_tax():
return 1.23456 # return something based on the _list_with_all_employees var
C) Using functions inside another function
def calculate_salary(employee, list_with_all_employees):
def _calculate_tax():
return 1.23456 # return something based on the list_with_a--Lemployees var
x = _calculate_tax()
# some other calculations done to x
pass
y = # something
return y
Solution C) allows to use variables in the scope of the outer function without having the need to declare them in the inner function. Might be useful in some situations.

Do something like:
def some_function():
return some_other_function()
def some_other_function():
return 42
if you were to run some_function() it would then run some_other_function() and returns 42.
EDIT: I originally stated that you shouldn't define a function inside of another but it was pointed out that it is practical to do this sometimes.

Function In function python
def Greater(a,b):
if a>b:
return a
return b
def Greater_new(a,b,c,d):
return Greater(Greater(a,b),Greater(c,d))
print("Greater Number is :-",Greater_new(212,33,11,999))