I want to pass a function, f(a=1,b=2) into g and use the 'a' value in g
def f(a,b): pass
def g(f): #print f.a
g(f(1,2)) should result in an output of 1
I looked into the inspect module but can't seem to get hold of f in g
This is as far as my programming knowledge has got me :
def g(f):
print(list(inspect.signature(f).parameters.keys()))
g(f(1,2)) results in: TypeError: None is not a callable object
This is not possible. When you call g(f(1,2)), f(1,2) is finished before g runs. The only way this would be possible would be for f to return its arguments.
You need to call g appropriately, so the f, as a function, is an argument:
g(f, 1, 2)
Now work that into your function:
def g(f, *args):
print(list(inspect.signature(f).parameters.keys()))
... and from here, you can iterate through args to make the proper call to f.
Does that get you moving?
You could do something like this:
def f(a, b):
print(a)
print(b)
Then define wrapper as:
def wrapper(func, a, b):
func(a, b)
Or if you want more flexibility use *args:
def wrapper(func, *args):
func(*args)
That flexibility comes with some risk if the number of arguments don't match. Which means you'll need to take care that all func passed to wrapper have a consistent signature.
You could use **kwargs which would help with the above, then:
def wrapper(func, **kwargs):
func(**kwargs)
Then calls to wrapper would look like:
wrapper(f, a=1, b=2)
This would allow for more flexibility and signature of func could vary as needed.
You could turn wrapper into a decorator but that's another question.
Related
Mock version of the problem
For a function
def f(a,b,c):
return a+b+c
The function
def fix(func, **kwargs):
fa = kwargs.get('a')
fb = kwargs.get('b')
if fa is not None and fb is not None:
def f(*args):
return func(a=fa, b=fb, c=args[0])
elif fa is not None:
def f(*args):
return func(a=fa, b=args[0], c=args[1])
elif fb is not None:
def f(*args):
return func(a=args[0],b=fb, c=args[1])
else:
def f(*args):
return func(args)
return f
allows to obtain a new function by fixing some of the parameters of func.
For example: fix(g, b=3) would give us a function like
def fixed_b_in_g(a,c):
return g(a,3,c)
Question: I would like to see if there is some trick to use fix in such a way that produces a function like
def fix_a_equal_b_in_g(a,c):
return g(a,a,c)
Concrete problem
The function scipy.stats.rv_continuous.fit allows to fit parameters of a distribution to an input sample. It allows to input some keyword arguments (like fix above does) to tell it to keep some of the parameters fixed to values that the user inputs. Internally scipy.stats.rv_continuous.fit has a function, scipy.stats.rv_continuous._reduce_func, that does more or less what dix does (better implemented than my fix for example).
In my case, rather than fixing some parameters to values, I would like to fit to keep two parameters (say a and b) equal to each other, but still free during the fitting.
We can use this function to copy a keyword argument whose name is base_kwarg_name to added_kwarg_name:
def with_copied_kwargs(func, added_kwarg_names_by_base):
def fixed_func(*args, **base_kwargs):
added_kwargs = {
added_kwarg_name: base_kwargs[base_kwarg_name]
for base_kwarg_name, added_kwarg_name in added_kwarg_names_by_base.items()
}
return func(*args, **base_kwargs, **added_kwargs)
return fixed_func
Given:
def add(*, a, b, c):
return a + b + c
then modified_add = with_copied_kwargs(add, {"b": "c"}) is equivalent to:
def modified_add(*, a, b):
return add(a=a, b=b, c=b)
with_copied_kwargs can then be used along with functools.partial to both both copy keyword arguments and provide values incrementally. modified_add = functools.partial(with_copied_kwargs(add, {"b": "c"}), a=1) is equivalent to:
def modified_add(*, b):
return add(a=1, b=b, c=b)
Note that I add * (see PEP 3102) before all parameters in functions I then apply with_copied_kwargs to because the minute people start using positional arguments, things would get messy. So better to restrict it to keyword-only arguments.
suppose I define a function of an unpackable array (e.g. list)
def f(params):
a,b,c = params
return a+b**2+c**2
is there a way to integrate over only the first parameter? I tried
import scipy.integrate.quad as quad
def integrate(b,c):
return quad(f,0,1,args=(b,c))
but it returns an error because the function f only takes one positional argument. I guess I have to unpack it in the integration step, but unsure how.
Thanks!
If these are the only two use cases for using this function you can do something like the following:
def f(*args):
if len(args) == 1:
a,b,c = args[0]
else:
a, b,c = args
return a+b**2+c**2
This assumes that if you pass one argument to the function in can be unpacked and if you pass more than one you pass a, b, c directly.
In a more generic setting, you can define different behavior of a function depending on the input type using functools.singledispatch. In this case this would look as follows:
from functools import singledispatch
#singledispatch
def f(arg1, arg2=None, arg3=None):
a, b, c = arg1
return a+b**2+c**2
#f.register(int)
#f.register(float)
def _(arg1, arg2, arg3):
return arg1+arg2**2+arg3**2
The advantage of this would be that you can extend it for different input types if that is needed.
I would like to use functools.partial to set a certain argument to a constant and at the same time remove the argument altogether.
Let me explain it using a simple example.
from functools import partial
def f(a, b):
return a * b
g = partial(f, b=2)
However, this function g still has the following calling signature:
g?
Signature: g(a, *, b=1)
Call signature: g(*args, **kwargs)
Type: partial
String form: functools.partial(<function f at 0x7ff7045289d8>, b=1)
File: /opt/conda/envs/dev/lib/python3.6/functools.py
Docstring:
partial(func, *args, **keywords) - new function with partial application
of the given arguments and keywords.
I could of course do this with a lambda function like:
def f(a, b):
return a * b
g = lambda a: f(a, b=2)
with the correct calling signature:
g?
Signature: g(a)
Docstring: <no docstring>
File: ~/Work/<ipython-input-7-fc5f3f492590>
Type: function
The downside of using lamdba functions is that I would need to write down all the arguments again. In my simple example this doesn't matter, but look at this:
phase_func = lambda site1, site2, B_x, B_y, B_z, orbital, e, hbar: \
phase_func(site1, site2, B_x, B_y, B_z, orbital, e, hbar, xyz_offset=(0,0,0))
# or this
phase_func = partial(phase_func, xyz_offset=(0,0,0)
Why would I even want this?
Later in my code I use wrappers that can generate a new function by multiplying two other functions, like combine(function1, function2, operator.mul). This combine function looks at all the arguments, therefore I need to remove the argument after setting it.
Consider the following code:
args, varargs, varkw, defaults = inspect.getargspec(method)
if inspect.ismethod(method):
args = args[1:] # Skip 'self'
When running this on Python 2 and adding something with self, self is skipped (as mentioned in the comment). On Python 3, however I got trouble when using the code on Class.method (i.e. not instance.method). The problem is similar to Detecting bound method in classes (not instances) in Python 3, but non of the answers there works. Using inspect.isroutine() or inspect.isfunction() breaks the code for non-methods (no self). Using hasattr(method, '__self__') doesn't work on Class.method.
I've written a small testscript for this:
from __future__ import print_function
import inspect
def args_without_self(method):
args, varargs, varkw, defaults = inspect.getargspec(method)
if inspect.ismethod(method):
args = args[1:] # Skip 'self'
return args
class Class(object):
def method(self, a, b, c):
pass
#staticmethod
def static(a, b, c):
pass
#classmethod
def classmethod(cls, a, b, c):
pass
def function(a, b, c):
pass
instance = Class()
print(args_without_self(Class.method))
print(args_without_self(instance.method))
print(args_without_self(Class.static))
print(args_without_self(instance.static))
print(args_without_self(Class.classmethod))
print(args_without_self(instance.classmethod))
print(args_without_self(function))
The code works with both Python 2 and 3. However args_without_self(Class.method) also has self in Python 3 (and I would like to avoid that, but not break the others). Everythign should print ['a', 'b', 'c'].
You can't, on Python 3, detect methods on the class, as they are never bound. They are just regular functions.
At most you can look at their qualified name and guess if they might be methods, then see if the first argument is named self. Heuristics and guesses, in other words:
if inspect.isfunction(method) and `.` in method.__qualname__ and args[0] == 'self':
args = args[1:] # Skip 'self'
I'm very new to Python and I don't understand how functions themselves can seemingly have attributes. In the code below, there is a function called f, and later in the code, something by the name of f.count is referenced. How can a function, namely f, have a .count? I'm getting an error message of: 'NoneType' object has no attribute 'count' on that line, so it obviously doesn't have that attribute yet. How do I give it that attribute?
def fcount(n):
print n.__name__
#fcount
def f(n):
return n+2
for n in range(5):
print n
#print f(n)
print 'f count =',f.count #THE LINE CAUSING THE ERROR MENTIONED ABOVE
#fcount
def g(n):
return n*n
print 'g count =',g.count
print g(3)
print 'g count =',g.count
Edit: Added fcount(), which doesn't do anything much, and details about error.
Let’s start with the definition of f:
#fcount
def f(n):
return n+2
This defines a f as the return value of a call to the function fcount, which is used as a decorator (the leading #) here.
This code is roughly equivalent with
def f(n):
return n+2
f = fcount(f)
since the decorator – fcount – does not return anything, f is None and not a function at the call site.
In your case fcount should return some function and add a count attribute to that returned function. Something useful (?) might be
def fcount(fn):
def wrapper(n):
wrapper.count += 1
return fn(n)
wrapper.count = 0
return wrapper
EDIT
As #jonrsharpe pointed out, a generalized decorator can forward positional and keyword arguments by capturing them with *args and **kwargs in the signature and expanding them in the same way when calling another function. The names args and kwargs are used by convention.
Python also has a helper function (a decorator itself) that can transfer information (name, docstring and signature information) from one function to another: functools.wraps. A complete example looks like this:
from functools import wraps
def decorator(func):
#wraps(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
#decorator
def f(a, b, c=None):
"The useful f function"
pass
print f.__name__ # `f` rather than `wrapper`
print help(f) # `f(*args, **kwargs) The useful f function` rather than `wrapper(*args, **kwargs)`
The problem here is with your "decorator function", fcount. A Python decorator function should return a function:
#decorates
def func(...):
...
is effectively:
func = decorates(func)
In your case, the "decorator" fcount only prints, and won't return anything; hence using it will assign f = None.
The answer to your more general question is that functions, being Python objects like more-or-less everything else, certainly can have attributes. To actually implement what you want, a decorator that counts how many times the decorated function is called, you could do:
def fcount(f):
"""Decorator function to count how many times f is called."""
def func(*args, **kwargs):
func.count += 1
return f(*args, **kwargs)
func.count = 0
return func
Python functions are first-class objects.
They can have attributes.
This feature is rarely seen in the literature, but it has its uses, such as a simplified closure.
By the way, the kind of counter code you asked about is explained in a Pycon 2014 video about decorators.
Reference
PEP 232 -- Function Attributes
Colton Myers: Decorators: A Powerful Weapon in your Python Arsenal - PyCon 2014