I am using a function in TensorFlow which maps a set of tensors to another arrangement of tensors. For example, you might write:
data = data.map(_function)
def _function(a, b, c):
return (a + 1, b, c)
So here, you pass _function as a function variable to map, and map passes it three tensors, which are mutated in some way (here, just adding one) and returned.
My question is: Is there a way to pass in additional variables to _function?
If I want to perform a + x, and not a + 1, then how could I pass in the additional variable?
You can't do something like: data.map(_function(x)) because then you're passing the result of a function, not the function itself.
I've experimented with *arg, but I can't find a way. Any help is greatly appreciated.
You can do sth like
def extra_func(x):
def _function(a, b, c):
return (a + x, b, c)
return _function
So you can do data.map(extra_func(x))
or you can use functools.partial to fix some of a function params
Related
Mock version of the problem
For a function
def f(a,b,c):
return a+b+c
The function
def fix(func, **kwargs):
fa = kwargs.get('a')
fb = kwargs.get('b')
if fa is not None and fb is not None:
def f(*args):
return func(a=fa, b=fb, c=args[0])
elif fa is not None:
def f(*args):
return func(a=fa, b=args[0], c=args[1])
elif fb is not None:
def f(*args):
return func(a=args[0],b=fb, c=args[1])
else:
def f(*args):
return func(args)
return f
allows to obtain a new function by fixing some of the parameters of func.
For example: fix(g, b=3) would give us a function like
def fixed_b_in_g(a,c):
return g(a,3,c)
Question: I would like to see if there is some trick to use fix in such a way that produces a function like
def fix_a_equal_b_in_g(a,c):
return g(a,a,c)
Concrete problem
The function scipy.stats.rv_continuous.fit allows to fit parameters of a distribution to an input sample. It allows to input some keyword arguments (like fix above does) to tell it to keep some of the parameters fixed to values that the user inputs. Internally scipy.stats.rv_continuous.fit has a function, scipy.stats.rv_continuous._reduce_func, that does more or less what dix does (better implemented than my fix for example).
In my case, rather than fixing some parameters to values, I would like to fit to keep two parameters (say a and b) equal to each other, but still free during the fitting.
We can use this function to copy a keyword argument whose name is base_kwarg_name to added_kwarg_name:
def with_copied_kwargs(func, added_kwarg_names_by_base):
def fixed_func(*args, **base_kwargs):
added_kwargs = {
added_kwarg_name: base_kwargs[base_kwarg_name]
for base_kwarg_name, added_kwarg_name in added_kwarg_names_by_base.items()
}
return func(*args, **base_kwargs, **added_kwargs)
return fixed_func
Given:
def add(*, a, b, c):
return a + b + c
then modified_add = with_copied_kwargs(add, {"b": "c"}) is equivalent to:
def modified_add(*, a, b):
return add(a=a, b=b, c=b)
with_copied_kwargs can then be used along with functools.partial to both both copy keyword arguments and provide values incrementally. modified_add = functools.partial(with_copied_kwargs(add, {"b": "c"}), a=1) is equivalent to:
def modified_add(*, b):
return add(a=1, b=b, c=b)
Note that I add * (see PEP 3102) before all parameters in functions I then apply with_copied_kwargs to because the minute people start using positional arguments, things would get messy. So better to restrict it to keyword-only arguments.
I am trying to pass a list into the definition of a function in order to create new variables. The use case here is to run scipy's curve fit to find optimal parameters of the function. I want this function to be able to take any number of variables dynamically without specifically typing in all the variables I want it to optimize/solve for (b1, rate_1, etc.). Right now I have a list of the variables to include but can't seem to get the function to create them as new parameters in the function definition, which it looks like I need to do.
I'm familiar with using * in a function as seen below, but it seems that is for when the function is already defined and you're calling it. I want to do something similar but in the definition of the function so the function itself recognizes b1, rate_1, etc. as parameters that it can solve for using curve_fit.
My starter code:
def get_optimal_adstock_multivariate(x_var_names):
y = np.array(final_df['Count Of Solutions'])
# make list of coefficient variables (b1, b2, etc.) and make new variables for each rate (rate_1, rate_2, etc.)
coef_vars = []
rates = []
for i in range(0, len(x_var_names)):
coef_vars.append("b" + str(i+1))
rates.append("rate_" + str(i+1))
coef_vars_rates = coef_vars + rates
def f(final_df, b0, *coef_vars_rates): # using * to pass b1, rate_1, b2, rate_2, etc. as parameters (unpacking the list)
# need this function to recognize final_df, b0, b1, rate_1, etc. as variables
You cannot directly make the function recognize these variables unfortunately.
You can use keyword arguments with a double-* and address them from the resulting dictionary object:
def f(a, b, **kwargs):
... kwargs["b0"] ...
Posting this in hope that it might help you, though I am aware it's not a full solution to your problem.
I don't understand what you are trying to do with x_var_names.
We can define equivalent functions:
In [260]: a, b, c = 1,2,3
In [261]: def foo0(x, *args):
...: print(x, args)
...:
In [262]: def foo1(x, a,b,c):
...: print(x, a,b,c)
...:
and call them a list of variables or the actual variables:
In [263]: alist=[a,b,c]
In [264]: foo0(100,*alist)
100 (1, 2, 3)
In [265]: foo1(100,*alist)
100 1 2 3
In [266]: foo0(100,a,b,c)
100 (1, 2, 3)
In [267]: foo1(100,a,b,c)
100 1 2 3
Or if I refine the print in foo0 I get the same display in both:
In [268]: def foo2(x, *args):
...: print(x, *args)
...:
In [269]: foo2(100,a,b,c)
100 1 2 3
Look at curve_fit. It will work with either foo1 or foo2 signatures.
f(xdata, *params)
The number of params is determined by p0, though it also talks about determining them by introspection. By that I think it can deduce that foo1 expects 3 values.
Don't confuse variable names in the global environment with the variables, or tuple of values passed via curve_fit to your function. Local names can also be different. For example
def foo3(x, *args):
a,b,c = args
print(x, a,b,c)
uses local unpacking, where the a,b,c are just convenient ways of referencing the values in the function.
Is it possible to define a closure for a function which is already defined?
For example I'd like to have a "raw" function and a function which already has some predefined values set by a surrounding closure.
Here is some code showing what I can do with a closure to add predefined variables to a function definition:
def outer(a, b, c):
def fun(d):
print(a + b + c - d)
return fun
foo = outer(4, 5, 6)
foo(10)
Now I want to have a definition of fun outside of a wrapping closure function, to be able to call fun either with variables from a closure or by passing variables directly. I know that I need to redefine a function to make it usable in a closure, thus I tried using lambda for it:
def fun(a, b, c, d): # raw function
print(a + b + c - d)
def clsr(func): # make a "closure" decorator
def wrap(*args):
return lambda *args: func(*args)
return wrap
foo = clsr(fun)(5, 6, 7) # make a closure with values already defined
foo(10) # raises TypeError: fun() missing 3 required positional arguments: 'a', 'b', and 'c'
fun(5, 6, 7, 10) # prints 8
What I also tried is using wraps from functools, but I was not able to make it work.
But is this even possible? And if yes: Is there any module which already implements decorators for this?
You can just define the wrap on the fly:
def fun(a, b, c, d): # raw function
print(a + b + c - d)
def closed(d): fun(5,6,7,d)
closed(10)
You can use this with lambda, but #juanpa points out you should not if there is no reason to. The above code will result in 8. This method by the way is not Python specific, most languages would support this.
But if you need a closure in a sense that it relies on the wrapper variables, than no, and there is good reason not to. This will create essentially a non-working function, that relies on wrapping. In this case using a class maybe better:
class fun:
def __init__(self,*args): #Can use specific things, not just *args.
self.args = args #Or meaningful names
def __call__(self,a, b, c, d): # raw function
print(a + b + c - d,self.args)
def closed(d):
fun("some",3,"more",['args'])(5,6,7,d)
closed(10)
or using *args/**kwargs directly and passing extra variables through that. Otherwise I am not familiar with a "inner function" construct that only works after wrapping.
Lets say I have three functions in one module as defined below:
def add_nums(a, b):
return a + b
def sum_and_square_one(add_nums, a, b):
result = add_nums(a,b)
return result*result
def sum_and_square_two(a, b):
result = add_nums(a,b)
return result*result
Both functions sum_and_square_one and sum_and_square_two do the same task. But the former takes add_nums as an argument while the latter calls add_nums inside. My question is which one is the better way. Passing a function as an argument or calling inside a function?
If you always want to call the same function then there is no point in passing it as an argument every time. (this is the case in your example)
If you, however, want to dynamically call different (maybe similar) functions, then passing the appropiate function as an argument would make sense.
Depends on your use case. Do you need the function to change the way it works depending on that argument?
Or, rewriting your meaningless example into a bit less meaningless example (but still quite):
def add_nums(a, b):
return a + b
def do_something_then_square(a, b, what_to_do):
result = what_to_do(a, b)
return result * result
def sum_then_square(a, b):
result = add_nums(a, b)
return result * result
# Then you do:
sum_then_square(2, 5) # 49
do_something_then_square(2, 5, what_to_do=add_nums) # 49
do_something_then_square(2, 5, what_to_do=lambda a, b: a + b) # 49
do_something_then_square(2, 5, what_to_do=lambda a, b: a * b) # 100
do_something_then_square(2, 5, what_to_do=min) # 4
do_something_then_square(2, 5, what_to_do=complex) # (-21+20j)
Only question is: do you need that added flexibility? Otherwise it's just useless additional typing. That question must be answered on a case by case basis; it needs a complete example to give a useful answer.
As part of a script I am making, I want to sort a series of points in a counter-clockwise order around a central point, which we will call 'a'.
I have a function that determines, for two points 'b' and 'c', if c is to the right of or left of the ray a->b. This function is right_of(a, b, c), and it is tested and works.
I want to use this function to sort a list of tuples with 2-d coordinates e.g. [(0, 0), (0, 1), (1, 1),...]. However, each time I sort, there will be a different point 'a' to pass to the function right_of(). What I want is a 'function' returnSortFunction(a) that will return a function with two arguments, f(b, c), and and when f(b, c) is called on each pair of coordinates as I sort, it should return the result of right_of(a, b, c) with 'a' already filled in.
I have tried to implement this using a factory, but I don't think I understand factories well enough to do it correctly, or determine if that is not what a factory is for. How can I build this feature?
You can have a function return a function, no problem. A simple way to do it is something like
def returnSortFunction(a):
return lambda b,c: right_of(a,b,c)
You need a wrapper function around your right_of function. You could use a lambda, but I think your logic is going to be more complicated than that. Assuming you want to pass in a function as a comparator to your sorting method, it's going to look something like this:
def returnSortFunction(a):
def comparator(p1, p2, a = a):
if p1 == p2:
return 0
elif right_of(a, p1, p2):
return 1
else:
return -1
return comparator
Functions are first class objects in python, so you can do something like this:
def prepare_funcs(number):
def inc(a):
return number + a
def mult(a):
return number * a
return inc, mult
inc5, mult5 = prepare_funcs(5)
inc2, mult2 = prepare_funcs(2)
inc5(2) #Out: 7
mult2(10) #Out: 20
For your specific context you should also check out functools module, specifically partial function. With it, you can 'partially' prepare arguments to your function like this:
right_of_5 = functools(right_of, 5)
right_of_5(b, c)
That will work, because right_of_5 will automatically fill right_of first argument - a - with number 5.