Better Function Composition in Python - python

I work in Python. Recently, I discovered a wonderful little package called fn. I've been using it for function composition.
For example, instead of:
baz(bar(foo(x))))
with fn, you can write:
(F() >> foo >> bar >> baz)(x) .
When I saw this, I immediately thought of Clojure:
(-> x foo bar baz) .
But notice how, in Clojure, the input is on the left. I wonder if this possible in python/fn.

You can't replicate the exact syntax, but you can make something similar:
def f(*args):
result = args[0]
for func in args[1:]:
result = func(result)
return result
Seems to work:
>>> f('a test', reversed, sorted, ''.join)
' aestt'

You can't get that exact syntax, although you can get something like F(x)(foo, bar, baz). Here's a simple example:
class F(object):
def __init__(self, arg):
self.arg = arg
def __call__(self, *funcs):
arg = self.arg
for f in funcs:
arg = f(arg)
return arg
def a(x):
return x+2
def b(x):
return x**2
def c(x):
return 3*x
>>> F(2)(a, b, c)
48
>>> F(2)(c, b, a)
38
This is a bit different from Blender's answer since it stores the argument, which can later be re-used with different functions.
This is sort of like the opposite of normal function application: instead of specifying the function up front and leaving some arguments to be specified later, you specify the argument and leave the function(s) to be specified later. It's an interesting toy but it's hard to think why you'd really want this.

If you want to use fn, with a little hack you can get a bit closer to Clojure syntax:
>>> def r(x): return lambda: x
>>> (F() >> r(x) >> foo >> bar >> baz)()
See how I added another function at the beginning of the composition chain that will just return x when called. The problem with this is that you still have to call your composed function, just without any arguments.
I think #Blender's answer is your best bet trying to emulate Clojure's thread function in Python.

I came up with this
def _composition(arg, *funcs_and_args):
"""
_composition(
[1,2,3],
(filter, lambda x: x % 2 == 1),
(map, lambda x: x+3)
)
#=> [4, 6]
"""
for func_and_args in funcs_and_args:
func, *b = func_and_args
arg = func(*b, arg)
return(arg)

This seems to work for simple input. Not sure it is worth the effort for complex input, e.g., ((42, 'spam'), {'spam': 42}).
def compose(function, *functions):
return function if not functions else \
lambda *args, **kwargs: function(compose(*functions)(*args, **kwargs))
def rcompose(*functions):
return compose(*reversed(functions))
def postfix(arg, *functions):
return rcompose(*functions)(arg)
Example:
>>> postfix(1, str, len, hex)
'0x1'
>>> postfix(1, hex, len)
3

My compose function that returns a function
def compose(*args):
length = len(args)
def _composeInner(lastResult, index):
if ((length - 1) < index):
return lastResult
return _composeInner(args[index](lastResult), index + 1)
return (lambda x: _composeInner(x, 0))
Usage:
fn = compose(
lambda x: x * 2,
lambda x: x + 2,
lambda x: x + 1,
lambda x: x / 3
)
result = fn(6) # -> 5

I understand what you mean. It doesn't make sense. In my opinion this python library
does it better.
>>> from compositions.compositions import Compose
>>> foo = Compose(lambda x:x)
>>> foo = Compose(lambda x:x**2)
>>> foo = Compose(lambda x:sin(x))
>>> (baz*bar*foo)(x)

Related

Prevent a function from being called twice in a row

I'm learning decorators and I have a task asking me to create a decorator to prevent a function from being called twice in a row. If the same function is called again, it should return None
I can't seem to understand how it really works, so far I've achieved this:
def dont_run_twice(f):
global counter
def wrapper(*args, **kwargs):
counter += 1
if counter == 2:
return None
else:
result = f(*args, **kwargs)
return result
counter = 0
(I know that it's a really bad attempt, but I just don't know a way to keep track of a function called with specific arguments in order to check if it was already before)
the output should be something like:
#dont_run_twice
def myPrint(*args):
print(*args)
myPrint("Hello")
myPrint("Hello") #won't do anything (only return None)
myPrint("Hello") #still does nothing.
myPrint("Goodbye") #will work
myPrint("Hello") #will work
Seems this can help you:
import functools
def do_not_run_twice(func):
prev_call = None
#functools.wraps(func) # It is good practice to use this decorator for decorators
def wrapper(*args, **kwargs):
nonlocal prev_call
if (args, kwargs) == prev_call:
return None
prev_call = args, kwargs
return func(*args, **kwargs)
return wrapper
Try this:
my_print("Hello")
my_print("Hello") # won't do anything (only return None)
my_print("Hello") # still does nothing.
my_print("Goodbye") # will work
my_print("Hello") # will work.
Here's a solution which is very similar to Andrey Berenda's solution, but which works by assigning an attribute to the function object, rather than using a non-local variable. The practical difference is that the function's previous arguments become available externally, which might help for debugging purposes.
from functools import wraps
def dont_run_twice(func):
#wraps(func)
def wrapper(*args, **kwargs):
if (args, kwargs) == wrapper._prev_args:
return None
wrapper._prev_args = args, kwargs
return func(*args, **kwargs)
wrapper._prev_args = None
return wrapper
Example:
>>> #dont_run_twice
... def f(x, y):
... return x + y
...
>>> f(1, 2)
3
>>> f(3, 4)
7
>>> f(3, 4) # returns None
>>> f(1, 2)
3
>>> f._prev_args
((1, 2), {})
Note that both solutions have a slight flaw: you can call with the same argument values if you provide them as positional arguments then keyword arguments (or vice-versa):
>>> f(5, 6)
11
>>> f(x=5, y=6)
11
As a workaround, you can declare the wrapped function with positional-only (or keyword-only) arguments:
# positional-only, requires Python 3.8+
#dont_run_twice
def f(x, y, /):
return x + y
# keyword-only
#dont_run_twice
def g(*, x, y):
return x + y
Note also that if the previous args are mutable, then strange things can happen:
>>> a = [1, 2]
>>> b = [3, 4]
>>> f(a, b)
[1, 2, 3, 4]
>>> a[:] = [5, 6]
>>> b[:] = [7, 8]
>>> f([5, 6], [7, 8]) # returns None
The second function call here returns None despite the new arguments not being equal to the original arguments by either value or identity; they are equal to the original arguments' current values which were changed after they were used as arguments. This could lead to rather subtle bugs, but unfortunately there's no easy way to fix it.

how to store functions inside an array and call one single function as array[index] [duplicate]

How can I bind arguments to a Python function so that I can call it later without arguments (or with fewer additional arguments)?
For example:
def add(x, y):
return x + y
add_5 = magic_function(add, 5)
assert add_5(3) == 8
What is the magic_function I need here?
It often happens with frameworks and libraries that people accidentally call a function immediately when trying to give arguments to a callback: for example on_event(action(foo)). The solution is to bind foo as an argument to action, using one of the techniques described here. See for example How to pass arguments to a Button command in Tkinter? and Using a dictionary as a switch statement in Python.
Some APIs, however, allow you to pass the to-be-bound arguments separately, and will do the binding for you. Notably, the threading API in the standard library works this way. See thread starts running before calling Thread.start. If you are trying to set up your own API like this, see How can I write a simple callback function?.
Explicitly binding arguments is also a way to avoid problems caused by late binding when using closures. This is the problem where, for example, a lambda inside a for loop or list comprehension produces separate functions that compute the same result. See What do lambda function closures capture? and Creating functions (or lambdas) in a loop (or comprehension).
functools.partial returns a callable wrapping a function with some or all of the arguments frozen.
import sys
import functools
print_hello = functools.partial(sys.stdout.write, "Hello world\n")
print_hello()
Hello world
The above usage is equivalent to the following lambda.
print_hello = lambda *a, **kw: sys.stdout.write("Hello world\n", *a, **kw)
Using functools.partial:
>>> from functools import partial
>>> def f(a, b):
... return a+b
...
>>> p = partial(f, 1, 2)
>>> p()
3
>>> p2 = partial(f, 1)
>>> p2(7)
8
If functools.partial is not available then it can be easily emulated:
>>> make_printer = lambda s: lambda: sys.stdout.write("%s\n" % s)
>>> import sys
>>> print_hello = make_printer("hello")
>>> print_hello()
hello
Or
def partial(func, *args, **kwargs):
def f(*args_rest, **kwargs_rest):
kw = kwargs.copy()
kw.update(kwargs_rest)
return func(*(args + args_rest), **kw)
return f
def f(a, b):
return a + b
p = partial(f, 1, 2)
print p() # -> 3
p2 = partial(f, 1)
print p2(7) # -> 8
d = dict(a=2, b=3)
p3 = partial(f, **d)
print p3(), p3(a=3), p3() # -> 5 6 5
lambdas allow you to create a new unnamed function with fewer arguments and call the function:
>>> def foobar(x, y, z):
... print(f'{x}, {y}, {z}')
...
>>> foobar(1, 2, 3) # call normal function
1, 2, 3
>>> bind = lambda x: foobar(x, 10, 20) # bind 10 and 20 to foobar
>>> bind(1)
1, 10, 20
>>> bind = lambda: foobar(1, 2, 3) # bind all elements
>>> bind()
1, 2, 3
You can also use functools.partial. If you are planning to use named argument binding in the function call this is also applicable:
>>> from functools import partial
>>> barfoo = partial(foobar, x=10)
>>> barfoo(y=5, z=6)
10, 5, 6
Note that if you bind arguments from the left you need to call the arguments by name. If you bind from the right it works as expected.
>>> barfoo(5, 6)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foobar() got multiple values for argument 'x'
>>> f = partial(foobar, z=20)
>>> f(1, 1)
1, 1, 20
This would work, too:
def curry(func, *args):
def curried(*innerargs):
return func(*(args+innerargs))
curried.__name__ = "%s(%s, ...)" % (func.__name__, ", ".join(map(str, args)))
return curried
>>> w=curry(sys.stdout.write, "Hey there")
>>> w()
Hey there
Functors can be defined this way in Python. They're callable objects. The "binding" merely sets argument values.
class SomeFunctor( object ):
def __init__( self, arg1, arg2=None ):
self.arg1= arg1
self.arg2= arg2
def __call___( self, arg1=None, arg2=None ):
a1= arg1 or self.arg1
a2= arg2 or self.arg2
# do something
return
You can do things like
x= SomeFunctor( 3.456 )
x( arg2=123 )
y= SomeFunctor( 3.456, 123 )
y()
The question asks generally about binding arguments, but all answers are about functions. In case you are wondering, partial also works with class constructors (i.e. using a class instead of a function as a first argument), which can be useful for factory classes. You can do it as follows:
from functools import partial
class Animal(object):
def __init__(self, weight, num_legs):
self.weight = weight
self.num_legs = num_legs
animal_class = partial(Animal, weight=12)
snake = animal_class(num_legs = 0)
print(snake.weight) # prints 12

Function with Lambda in Python

I have the following straightforward function in Python 3:
def func(i,j):
return lambda i,j: (i*j)
Here's an example of what this function should do:
IN: func(4,'Hello')
OUT: ('Hello' 'Hello' 'Hello' 'Hello')
However, the actual output is an address in memory where the result is stored. What modification do I need to make?
If you want to return the value of i * j, then go ahead and return it. A lambda means it returns a function that acts as you want it to. Consider this:
def mul(x, y):
return x * y
def func(x, y):
return lambda x, y: x*y
Now let's take a look at a little shell session:
>>> mul(4, 'hello')
'hellohellohellohello'
>>> func(4, 'hello')
<function 'lambda' at ...>
>>> f = func(4, 'hello')
>>> f(4, 'hello')
'hellohellohellohello'
As you can see, when you use lambda, your function returns a function which, in turn, needs to be called. The arguments x and y have no correspondence to the arguments in the lambda function.
Since your expected output is a tuple of x lots of y, use a tuple in your function:
def func(i, j):
return (j,) * i

Python one-liner to call list of functions

I've got some old code where I stored lists of functions in Python as class attributes. These lists are used as a sort of event hook.
To call each function in the list with appropriate arguments, I've used one-liners, mixing map with lambda expressions. I'm now concerned that there is unnecessary overhead in using lambda expressions like this.. I guess the recommended way would be to drop both map and lambda and just use a standard for loop, for readability.
Is there a better (read faster) one-liner to do this, though?
For example:
class Foo:
"""Dummy class demonstrating event hook usage."""
pre = [] # list of functions to call before entering loop.
mid = [] # list of functions to call inside loop, with value
post = [] # list of functions to call after loop.
def __init__(self, verbose=False, send=True):
"""Attach functions when initialising class."""
self._results = []
if verbose:
self.mid.append( self._print )
self.mid.append( self._store )
if send:
self.post.append( self._send )
def __call__(self, values):
# call each function in self.pre (no functions there)
map( lambda fn: fn(), self.pre )
for val in values:
# call each function in self.mid, with one passed argument
map( lambda fn: fn(val), self.mid )
# call each fn in self.post, with no arguments
map( lambda fn: fn(), self.post )
def _print(self, value):
"""Print argument, when verbose=True."""
print value
def _store(self, value):
"""Store results"""
self._results.append(value)
def _send(self):
"""Send results somewhere"""
# create instance of Foo
foo = Foo(verbose=True)
# equivalent to: foo.__call__( ... )
foo( [1, 2, 3, 4] )
Is there a better way to write those one-liner map calls?
The recommended way is definitely to use for loops, however, if you insist on using map, then operator.methodcaller might be just what you need:
>>> def foo(*args):
... print 'foo',args
...
>>> def bar(*args):
... print 'bar',args
...
>>> from operator import methodcaller
>>>
>>> map(methodcaller('__call__',1,2,3),[foo,bar])
foo (1, 2, 3)
bar (1, 2, 3)
[None, None]
A word of caution about using map for this -- It won't work if you port your code to python 3 since map became lazy.
You could also use list comprehensions pretty trivially (and that works on python3 also):
[fn() for fn in self.pre]
[fn(val) for fn in self.mid]
etc.
First of all "I'm concerned that there is unnecessary overhead" is no way to optimise your code. Use a profiler to find the hotspots.
Secondly, your code could do with comments to let the reader know what is going on.
Finally, until proven otherwise, the following is a fine way to accomplish the task:
for func in self.pre: func()
#apply every function in self.mid to every value in values
for func,val in itertools.product(self.mid, values):
func(val)
If you wanted to capture the values, you could use a list comprehension; if you wanted to delay evaluation, you could use a generator expression.
>>> def chain(*fn):
>>> return lambda *args, **kwargs: [_(*args, **kwargs) for _ in fn]
>>>
>>> def add(x, y):
>>> return(x + y)
>>>
>>> def multiply(x, y):
>>> return(x * y)
>>>
>>> chained = chain(add, multiply)
>>> chained(2, 6)
[8, 12]

Implementing pointwise arithmetic with implicit type conversion

Suppose I have class Function, whose instances are callables that take one argument. I defined pointwise arithmetic for these classes in the straightforward way. Here's a simplified version of my code (I actually have more complex behavior in __init__ and __call__ but it's irrelevant for this question):
class Function:
'''
>>> f = Function(lambda x : x**2)
>>> g = Function(lambda x : x + 4)
>>> h = f/g
>>> h(6)
3.6
'''
def __init__(self, func):
self.func = func
def __call__(self, value):
return self.func(value)
def __truediv__(self, other):
if isinstance(other, Function):
return Function(lambda x:self(x)/other(x))
else:
return NotImplemented
# ...
I'm stuck when I try to allow implicit type conversions. For example, I want to be able to write:
>>> f = Function(lambda x : x ** 2)
>>> g = f+1
>>> g(5)
26
In other words, whenever I see a numeric object v in an arithmetic expression next to a Function instance, I want to convert v to Function(lambda x : v).
In addition, I want to achieve similar behavior for some of my user-defined types (again, whenever I see them in the same binary arithmetic expression with a Function object).
While I can certainly code this logic with a brute force assortment of regular and reflected binary arithmetic operators, each checking isinstance(v, numbers.Number), and isinstance(v, MyUserDefinedType), I feel there might be a more elegant way.
Also, if there are any other improvements possible with my design, please let me know. (Function objects are created rarely, but called very often, so performance is of some interest.)
EDIT:
To address #Eric's comment, I should clarify that I have another user-defined class Functional:
class Functional:
'''
>>> c = [1, 2, 3]
>>> f = Functional(lambda x : x + 1)
>>> f(c)
[2, 3, 4]
>>> g = Functional(lambda x : x ** 2)
>>> h = f + g
>>> h(c)
[3, 7, 13]
'''
def __init__(self, func):
self.func = func
#staticmethod
def from_function(self, function):
return Functional(function.func)
def __call__(self, container):
return type(container)(self.func(c) for c in container)
def __add__(self, other):
if isinstance(other, Functional):
return Functional(lambda x : self.func(x) + other.func(x))
else:
return NotImplemented
When I see both a Function and a Functional instance in the same arithmetic expression, I want Function to be implicitly converted to Functional using Functional.from_function method.
So, implicit type conversion hierarchy goes like this:
Functional
Function
anything else
And I'd like to implicitly convert to the highest type in this hierarchy seen in a given arithmetic expression.
Something like this for all operators would work well:
def __truediv__(self, other):
if callable(other):
return Function(lambda x:self(x)/other(x))
else:
return Function(lambda x:self(x)/other)
One option is to make all the operators in your Function class accept arbitrary values, which will be applied to the result of the underlying function if they're not functions themselves. For example, to extend allow f / 5, when f is a Function, simply modify the __truediv__ implementation you have to:
def __truediv__(self, other):
if isinstance(other, Function):
return Function(lambda x:self(x)/other(x))
else:
return Function(lambda x:self(x)/other)
You can optionally do some type checking to make sure that's sane (and raise errors early rather than later on), but it works without that.
After reading the comments and the other answers, I tried this approach. I'm posting it to ask for feedback. I like that I can handle both Function and Functional in one swoop, but I'm afraid it might be very expensive in terms of performance:
class Function:
'''
>>> f = Function(lambda x : x**2)
>>> g = Function(lambda x : x + 4)
>>> h = f/g
>>> h(6)
3.6
>>> k = f + 1
>>> k(5)
26
>>> m = f + (lambda x : x + 1)
>>> m(5)
31
'''
def __init__(self, arg):
if isinstance(arg, Function):
self.func = arg.func
elif callable(arg):
self.func = arg
else:
self.func = lambda x : arg
def __call__(self, value):
return self.func(value)
def __truediv__(self, other):
return self.__class__(lambda x:Function(self)(x)/Function(other)(x))
def __rtruediv__(self, other):
return self.__class__(lambda x:Function(other)(x)/Function(self)(x))
def __add__(self, other):
return self.__class__(lambda x:Function(self)(x)+Function(other)(x))
def __radd__(self, other):
return self.__class__(lambda x:Function(other)(x)+Function(self)(x))
# ...
class Functional(Function):
'''
>>> c = [1, 2, 3]
>>> f = Functional(lambda x : x + 1)
>>> f(c)
[2, 3, 4]
>>> g = Functional(lambda x : x ** 2)
>>> h = f + g
>>> h(c)
[3, 7, 13]
'''
def __call__(self, container):
return type(container)(self.func(c) for c in container)

Categories

Resources