I am looking at an existing code base that is using a bool variable like a method, for example:
class Manager(object):
def __init__(self):
self._on_elect_callback = None
self._on_revoke_callback = None
self.... = ... (..., event_listener = self._event)
def _event(self, type):
on_elect_callback = self._on_elect_callback
if type == SOME_CONSTANT:
....
if on_elect_callback:
on_elect_callback()
def do_this(self, on_elect_function):
self._on_elect_callback = on_elect_function
if self....:
on_elect_function()
Questions:
I am curious how that on_elect_callback is being used like a function with () after if condition on the last line. Isn't that some boolean variable? I searched the repo and there is no definition for that. What is it doing?
Also, I would like to set a variable in __init__ that the callback function of event can use like "hey this event type IS SOME_CONSTANT, so set the variable in __init__ to "ABCD" (or True), how can I achieve it? Is the way in the code above the way to do it?
The self._on_elect_callback was probably going to be assigned a function as in self._on_elect_callback = lambda: "I'm a function". I think the reason you think it was meant to be a variable is because it is used in an if expression. The reason is that if self._on_elect_callback gets assigned a function the expression will evaluate to True, as functions support a method bool, which returns True by default.
About Q2, sure you can, functions are objects in python, so you can pass them around to higher order functions, or assign them as variables as in the example
class Manager(object):
def __init__(self, function):
self._on_elect_callback = function # here we assigned the given function
if not function:
# here we define a default attribute using an anonymous function
self._on_elect_callback = lambda: "Default Function"
def _event(self, type):
if self._on_elect_callback:
self._on_elect_callback()
In python, is it possible to chain together class methods and functions together? For example, if I want to instantiate a class object and call a method on it that affects an instance variable's state, could I do that? Here is an example:
class Test(object):
def __init__(self):
self.x = 'Hello'
#classmethod
def make_upper(y):
y.x = y.x.upper()
What I'm wanting to do is this:
h = Test().make_upper()
I want to instantiate a class object and affect the state of a variable in one line of code, but I would also like to be able to chain together multiple functions that can affect state or do something else on the object. Is this possible in python like it is in jQuery?
Yes, sure. Just return self from the instance methods you are interested in:
class Test(object):
def __init__(self):
self.x = 'Hello'
def make_upper(self):
self.x = self.x.upper()
return self
def make_lower(self):
self.x = self.x.lower()
return self
h = Test().make_upper()
print(h.x)
Output:
HELLO
Yes and no. The chaining certainly works, but h is the return value of make_upper(), not the object returned by Test(). You need to write this as two lines.
h = Test()
h.make_upper()
However, PEP-572 was recently accepted for inclusion in Python 3.8, which means someday you could write
(h := Test()).make_upper()
The return value of Test() is assigned to h in the current scope and used as the value of the := expression, which then invokes its make_upper method. I'm not sure I would recommend using := in this case, though; the currently required syntax is much more readable.
I have a class (list of dicts) and I want it to sort itself:
class Table(list):
…
def sort (self, in_col_name):
self = Table(sorted(self, key=lambda x: x[in_col_name]))
but it doesn't work at all. Why? How to avoid it? Except for sorting it externally, like:
new_table = Table(sorted(old_table, key=lambda x: x['col_name'])
Isn't it possible to manipulate the object itself? It's more meaningful to have:
class Table(list):
pass
than:
class Table(object):
l = []
…
def sort (self, in_col_name):
self.l = sorted(self.l, key=lambda x: x[in_col_name])
which, I think, works.
And in general, isn't there any way in Python which an object is able to change itself (not only an instance variable)?
You can't re-assign to self from within a method and expect it to change external references to the object.
self is just an argument that is passed to your function. It's a name that points to the instance the method was called on. "Assigning to self" is equivalent to:
def fn(a):
a = 2
a = 1
fn(a)
# a is still equal to 1
Assigning to self changes what the self name points to (from one Table instance to a new Table instance here). But that's it. It just changes the name (in the scope of your method), and does affect not the underlying object, nor other names (references) that point to it.
Just sort in place using list.sort:
def sort(self, in_col_name):
super(Table, self).sort(key=lambda x: x[in_col_name])
Python is pass by value, always. This means that assigning to a parameter will never have an effect on the outside of the function. self is just the name you chose for one of the parameters.
I was intrigued by this question because I had never thought about this. I looked for the list.sort code, to see how it's done there, but apparently it's in C. I think I see where you're getting at; what if there is no super method to invoke? Then you can do something like this:
class Table(list):
def pop_n(self, n):
for _ in range(n):
self.pop()
>>> a = Table(range(10))
>>> a.pop_n(3)
>>> print a
[0, 1, 2, 3, 4, 5, 6]
You can call self's methods, do index assignments to self and whatever else is implemented in its class (or that you implement yourself).
So i have a relatively convoluted setup for something I'm working on explained as follows:
This is is python. and more of a rough outline, but it covers everything I need. Though the process next function is the same so feel free to clean that up if you want.
#timer event that runs every .1 second and processes events in a queue
some_event_timer():
events.process_next()
class Event_queue:
def __init__(self):
self.events = []
def push(self, event, parameters):
self.events.insert(len(self.events), event, parameters)
def process_next(self):
event = self.pop(0)
event[0](event[1])
class Foo:
def __init__(self, start_value = 1):
self.value = start_value
def update_value(self, multiple):
self.value *= multiple
def return_bah(self)
return self.value + 3
class Bar:
def __init__(self, number1, number2):
self.init = number1
self.add = number2
def print_alt_value(self, in_value):
print in_value * (self.init + self.add)
That is a barebones of what I have, but it illustrates my problem:
Doing the below
events2 = Event_queue2()
foo1 = Foo(4) ----> foo1.value = 4 here
bar1 = Bar(4, 2)
events2.push(foo1.update_value,1.5)
events2.push(bar1.print_alt_value,foo1.value)
events2.push(bar.print_alt_value,foo1.return_bah())
events2.process_next() ----> should process update_value to change foo.value to 6
events2.process_next() ----> should process print_alt_value in bar class - expected 36
events2.process_next() ----> should process print_alt_value - expected 54
I initially expected my output to be 36 6 * (4 + 2)
I know why its not, foo1.value and foo1.return_bah() gets passed as an evaluated parameter (correct term?).
What I really want is to pass the reference to the variable or the reference to the method, rather than having it evaluate when I put it in my event queue.
Can anyone help me.
I tried searching, but I couldn't piece together what I wanted exactly.
TO get what I have now I initially looked at these threads:
Calling a function of a module from a string with the function's name in Python
Use a string to call function in Python
But I don't see how to support parameters from that properly or how to support passing another function or reference to a variable from those.
I suppose at least for the method call, I could perhaps pass the parameter as foo1.return.bah and evaluate in the process_next method, but I was hoping for a general way that would accept both standard variables and method calls, as the event_queue will take both.
Thank you for the help
Update edit:
So I following the suggestion below, and got really close, but:
Ok, so I followed your queue suggestion and got really close to what I want, but I don't completely understand the first part about multiple functions.
I want to be able to call a dictionary of objects with this as well.
for example:
names = ["test1", "test2"]
for name in names:
names_objs[name] = Foo(4)
Then when attempting to push via lambda
for name in names_list:
events2.push(lambda: names_objs[name].update_value(2))
doesn't work. When teh event actually gets processed it only runs on whatever name_objs[name] references, and if the name variable is no longer valid or has been modified outside the function, it is wrong.
This actually wasn't surprising, but adding a:
name_obj_hold = name_objs[name]
then pushing that didn't either. it again only operates on whatever name_obj_hold last referenced.
Can someone clarify the multiple funcs thing. I'm afraid I'm having trouble wrapping my head around it.
basically I need the initial method call evaluated, so something like:
names_objs[name].some_func(#something in here#)
gets the proper method and associated with the right class object instance, but the #something in here# doesn't get evaluated (whether it is a variable or another function) until it actually gets called from the event queue.
Instead of passing in the function to call func1 and the arguments that should be passed to the function, pass in a function func2 that calls func1 with the arguments that should be passed in.
d = {"a":1}
def p(val):
print val
def func1():
p(d["a"])
def call_it(func):
func()
call_it(func1)
d["a"] = 111
call_it(func1)
Within func1, d["a"] is not evaluated until func1 actually executes.
For your purposes, your queue would change to:
class EventQueue(object):
def __init__(self):
self.events = deque()
def push(self, callable):
self.events.append(callable)
def process_next(self):
self.events.popleft()()
collections.deque will be faster at popping from the front of the queue than a list.
And to use the EventQueue, you can use lambdas for quick anonymous function.
events2 = EventQueue()
foo1 = Foo(4)
bar1 = Bar(4, 2)
events2.push(lambda: foo1.update_value(1.5))
events2.push(lambda: bar1.print_alt_value(foo1.value))
events2.push(lambda: bar1.print_alt_value(foo1.return_bah()))
events2.process_next()
events2.process_next() # 36.0
events2.process_next() # 54.0
For Edit:
In this case you need to "capture" the value in a variable that is more tightly scoped than the loop. You can use a normal function and partial() to achieve this.
for name in names_list:
def update(name):
names_objs[name].update_value(2)
events2.push(partial(update, name))
Suppose I have a generic function f. I want to programmatically create a function f2 that behaves the same as f, but has a customized signature.
More detail
Given a list l and and dictionary d I want to be able to:
Set the non-keyword arguments of f2 to the strings in l
Set the keyword arguments of f2 to the keys in d and the default values to the values of d
ie. Suppose we have
l = ["x", "y"]
d = {"opt": None}
def f(*args, **kwargs):
# My code
Then I would want a function with signature:
def f2(x, y, opt=None):
# My code
A specific use case
This is just a simplified version of my specific use case. I am giving this as an example only.
My actual use case (simplified) is as follows. We have a generic initiation function:
def generic_init(self, *args, **kwargs):
"""Function to initiate a generic object"""
for name, arg in zip(self.__init_args__, args):
setattr(self, name, arg)
for name, default in self.__init_kw_args__.items():
if name in kwargs:
setattr(self, name, kwargs[name])
else:
setattr(self, name, default)
We want to use this function in a number of classes. In particular, we want to create a function __init__ that behaves like generic_init, but has the signature defined by some class variables at creation time:
class my_class:
__init_args__ = ["x", "y"]
__kw_init_args__ = {"my_opt": None}
__init__ = create_initiation_function(my_class, generic_init)
setattr(myclass, "__init__", __init__)
We want create_initiation_function to create a new function with the signature defined using __init_args__ and __kw_init_args__. Is it possible to write create_initiation_function?
Please note:
If I just wanted to improve the help, I could set __doc__.
We want to set the function signature on creation. After that, it doesn't need to be changed.
Instead of creating a function like generic_init, but with a different signature we could create a new function with the desired signature that just calls generic_init
We want to define create_initiation_function. We don't want to manually specify the new function!
Related
Preserving signatures of decorated functions: This is how to preserve a signature when decorating a function. We need to be able to set the signature to an arbitrary value
From PEP-0362, there actually does appear to be a way to set the signature in py3.3+, using the fn.__signature__ attribute:
from inspect import signature
from functools import wraps
def shared_vars(*shared_args):
"""Decorator factory that defines shared variables that are
passed to every invocation of the function"""
def decorator(f):
#wraps(f)
def wrapper(*args, **kwargs):
full_args = shared_args + args
return f(*full_args, **kwargs)
# Override signature
sig = signature(f)
sig = sig.replace(parameters=tuple(sig.parameters.values())[1:])
wrapper.__signature__ = sig
return wrapper
return decorator
Then:
>>> #shared_vars({"myvar": "myval"})
>>> def example(_state, a, b, c):
>>> return _state, a, b, c
>>> example(1,2,3)
({'myvar': 'myval'}, 1, 2, 3)
>>> str(signature(example))
'(a, b, c)'
Note: the PEP is not exactly right; Signature.replace moved the params from a positional arg to a kw-only arg.
For your usecase, having a docstring in the class/function should work -- that will show up in help() okay, and can be set programmatically (func.__doc__ = "stuff").
I can't see any way of setting the actual signature. I would have thought the functools module would have done it if it was doable, but it doesn't, at least in py2.5 and py2.6.
You can also raise a TypeError exception if you get bad input.
Hmm, if you don't mind being truly vile, you can use compile()/eval() to do it. If your desired signature is specified by arglist=["foo","bar","baz"], and your actual function is f(*args, **kwargs), you can manage:
argstr = ", ".join(arglist)
fakefunc = "def func(%s):\n return real_func(%s)\n" % (argstr, argstr)
fakefunc_code = compile(fakefunc, "fakesource", "exec")
fakeglobals = {}
eval(fakefunc_code, {"real_func": f}, fakeglobals)
f_with_good_sig = fakeglobals["func"]
help(f) # f(*args, **kwargs)
help(f_with_good_sig) # func(foo, bar, baz)
Changing the docstring and func_name should get you a complete solution. But, uh, eww...
I wrote a package named forge that solves this exact problem for Python 3.5+:
With your current code looking like this:
l=["x", "y"]
d={"opt":None}
def f(*args, **kwargs):
#My code
And your desired code looking like this:
def f2(x, y, opt=None):
#My code
Here is how you would solve that using forge:
f2 = forge.sign(
forge.arg('x'),
forge.arg('y'),
forge.arg('opt', default=None),
)(f)
As forge.sign is a wrapper, you could also use it directly:
#forge.sign(
forge.arg('x'),
forge.arg('y'),
forge.arg('opt', default=None),
)
def func(*args, **kwargs):
# signature becomes: func(x, y, opt=None)
return (args, kwargs)
assert func(1, 2) == ((), {'x': 1, 'y': 2, 'opt': None})
Have a look at makefun, it was made for that (exposing variants of functions with more or less parameters and accurate signature), and works in python 2 and 3.
Your example would be written like this:
try: # python 3.3+
from inspect import signature, Signature, Parameter
except ImportError:
from funcsigs import signature, Signature, Parameter
from makefun import create_function
def create_initiation_function(cls, gen_init):
# (1) check which signature we want to create
params = [Parameter('self', kind=Parameter.POSITIONAL_OR_KEYWORD)]
for mandatory_arg_name in cls.__init_args__:
params.append(Parameter(mandatory_arg_name, kind=Parameter.POSITIONAL_OR_KEYWORD))
for default_arg_name, default_arg_val in cls.__opt_init_args__.items():
params.append(Parameter(default_arg_name, kind=Parameter.POSITIONAL_OR_KEYWORD, default=default_arg_val))
sig = Signature(params)
# (2) create the init function dynamically
return create_function(sig, generic_init)
# ----- let's use it
def generic_init(self, *args, **kwargs):
"""Function to initiate a generic object"""
assert len(args) == 0
for name, val in kwargs.items():
setattr(self, name, val)
class my_class:
__init_args__ = ["x", "y"]
__opt_init_args__ = {"my_opt": None}
my_class.__init__ = create_initiation_function(my_class, generic_init)
and works as expected:
# check
o1 = my_class(1, 2)
assert vars(o1) == {'y': 2, 'x': 1, 'my_opt': None}
o2 = my_class(1, 2, 3)
assert vars(o2) == {'y': 2, 'x': 1, 'my_opt': 3}
o3 = my_class(my_opt='hello', y=3, x=2)
assert vars(o3) == {'y': 3, 'x': 2, 'my_opt': 'hello'}
You can't do this with live code.
That is, you seem to be wanting to take an actual, live function that looks like this:
def f(*args, **kwargs):
print args[0]
and change it to one like this:
def f(a):
print a
The reason this can't be done--at least without modifying actual Python bytecode--is because these compile differently.
The former results in a function that receives two parameters: a list and a dict, and the code you're writing operates on that list and dict. The second results in a function that receives one parameter, and which is accessed as a local variable directly. If you changed the function "signature", so to speak, it'd result in a function like this:
def f(a):
print a[0]
which obviously wouldn't work.
If you want more detail (though it doesn't really help you), a function that takes an *args or *kwargs has one or two bits set in f.func_code.co_flags; you can examine this yourself. The function that takes a regular parameter has f.func_code.co_argcount set to 1; the *args version is 0. This is what Python uses to figure out how to set up the function's stack frame when it's called, to check parameters, etc.
If you want to play around with modifying the function directly--if only to convince yourself that it won't work--see this answer for how to create a code object and live function from an existing one to modify bits of it. (This stuff is documented somewhere, but I can't find it; it's nowhere in the types module docs...)
That said, you can dynamically change the docstring of a function. Just assign to func.__doc__. Be sure to only do this at load time (from the global context or--most likely--a decorator); if you do it later on, tools that load the module to examine docstrings will never see it.
Maybe I didn't understand the problem well, but if it's about keeping the same behavior while changing the function signature, then you can do something like :
# define a function
def my_func(name, age) :
print "I am %s and I am %s" % (name, age)
# label the function with a backup name
save_func = my_func
# rewrite the function with a different signature
def my_func(age, name) :
# use the backup name to use the old function and keep the old behavior
save_func(name, age)
# you can use the new signature
my_func(35, "Bob")
This outputs :
I am Bob and I am 35
We want create_initiation_function to change the signature
Please don't do this.
We want to use this function in a number of classes
Please use ordinary inheritance.
There's no value in having the signature "changed" at run time.
You're creating a maintenance nightmare. No one else will ever bother to figure out what you're doing. They'll simply rip it out and replace it with inheritance.
Do this instead. It's simple and obvious and makes your generic init available in all subclasses in an obvious, simple, Pythonic way.
class Super( object ):
def __init__( self, *args, **kwargs ):
# the generic __init__ that we want every subclass to use
class SomeSubClass( Super ):
def __init__( self, this, that, **kwdefaults ):
super( SomeSubClass, self ).__init__( this, that, **kwdefaults )
class AnotherSubClass( Super ):
def __init__( self, x, y, **kwdefaults ):
super( AnotherSubClass, self ).__init__( x, y, **kwdefaults )
Edit 1: Answering new question:
You ask how you can create a function with this signature:
def fun(a, b, opt=None):
pass
The correct way to do that in Python is thus:
def fun(a, b, opt=None):
pass
Edit 2: Answering explanation:
"Suppose I have a generic function f. I want to programmatically create a function f2 that behaves the same as f, but has a customised signature."
def f(*args, **kw):
pass
OK, then f2 looks like so:
def f2(a, b, opt=None):
f(a, b, opt=opt)
Again, the answer to your question is so trivial, that you obviously want to know something different that what you are asking. You really do need to stop asking abstract questions, and explain your concrete problem.