Is it possible to do something like the following:
class foo():
def bar(): # a method that doesn't take any args
# slow calculation
return somefloat
b = bar # bar is a function but b just gives you the float attribute
f = foo()
f.b # returns somefloat but doesn't require the empty parentheses
I hope the example is clear since I'm not super clear on what the terminology is for what I want to do. My basic goal is to remove a bunch of parentheses for methods that don't have arguments to make the code cleaner to read.
The function is slow and rarely used so it would be easiest to calculate it real time rather than calculate it once ahead of time and store the variable.
Is this possible? Is it good practice? Is there a better way?
The standard way to achieve this is to use property, which is a decorator:
class Foo():
#property
def bar(self):
# slow calculation
return somefloat
f = Foo()
f.bar # returns somefloat but doesn't require the empty parentheses
A couple of things to notice:
You still need self in the method signature as usual, because sometimes you're going to need to refer to e.g. self.some_attribute inside the method. As you can see, that doesn't affect the use of the property at all.
There's no need to clutter your API with both a f.bar() method and a f.b property - it's better to decide what makes most sense for your class than offer a heap of different ways to do the same thing.
b = bar obviously wouldn't work. However a property would for the simplest "doesn't require the empty parentheses" ask of yours:
b = property(bar)
Now every access to f.b will call f.bar() "behind the curtains".
However this means that if you access f.b twice, f.bar() gets called twice, repeating the computation. If the repetition is irrelevant (i.e if the result doesn't change for repeated computations on the same object) you can do better ("caching" the result in f.b forever once it's first been computed) -- something like:
class foo(object):
def bar(self): # a method that doesn't take any args
# slow calculation
return somefloat
def _cache_bar(self):
result = self.bar()
setattr(self, 'b', result)
return result
b = property(_cache_bar)
By static method, but need to call by parentheses.
class foo(object):
#staticmethod
def bar(): # a method that doesn't take any args
# slow calculation
return "abc"
b = bar # bar is a function but b just gives you the float attribute
f = foo()
print f.b()
output:
$ python test.py
abc
Related
I am confused with following difference. Say I have this class with some use case:
class C:
def f(self, a, b, c=None):
print(f"Real f called with {a=}, {b=} and {c=}.")
my_c = C()
my_c.f(1, 2, c=3) # Output: Real f called with a=1, b=2 and c=3.
I can monkey patch it for purpose of testing like this:
class C:
def f(self, a, b, c=None):
print(f"Real f called with {a=}, {b=} and {c=}.")
def f_monkey_patched(self, *args, **kwargs):
print(f"Patched f called with {args=} and {kwargs=}.")
C.f = f_monkey_patched
my_c = C()
my_c.f(1, 2, c=3) # Output: Patched f called with args=(1, 2) and kwargs={'c': 3}.
So far so good. But I would like to patch only one single instance and it somehow consumes first argument:
class C:
def f(self, a, b, c=None):
print(f"Real f called with {a=}, {b=} and {c=}.")
def f_monkey_patched(self, *args, **kwargs):
print(f"Patched f called with {args=} and {kwargs=}.")
my_c = C()
my_c.f = f_monkey_patched
my_c.f(1, 2, c=3) # Output: Patched f called with args=(2,) and kwargs={'c': 3}.
Why has been first argument consumed as self instead of the instance itself?
Functions in Python are descriptors; when they're attached to a class, but looked up on an instance of the class, the descriptor protocol gets invoked, producing a bound method on your behalf (so my_c.f, where f is defined on the class, is distinct from the actual function f you originally defined, and implicitly passes my_c as self).
If you want to make a replacement that shadows the class f only for a specific instance, but still passes along the instance as self like you expect, you need to manually bind the instance to the function to create the bound method using the (admittedly terribly documented) types.MethodType:
from types import MethodType # The class implementing bound methods in Python 3
# ... Definition of C and f_monkey_patched unchanged
my_c = C()
my_c.f = MethodType(f_monkey_patched, my_c) # Creates a pre-bound method from the function and
# the instance to bind to
Being bound, my_c.f will now behave as a function that does not accept self from the caller, but when called self will be received as the instance bound to my_c at the time the MethodType was constructed.
Update with performance comparisons:
Looks like, performance-wise, all the solutions are similar enough as to be irrelevant performance-wise (Kedar's explicit use of the descriptor protocol and my use of MethodType are equivalent, and the fastest, but the percentage difference over functools.partial is so small that it won't matter under the weight of any useful work you're doing):
>>> # ... define C as per OP
>>> def f_monkey_patched(self, a): # Reduce argument count to reduce unrelated overhead
... pass
>>> from types import MethodType
>>> from functools import partial
>>> partial_c, mtype_c, desc_c = C(), C(), C()
>>> partial_c.f = partial(f_monkey_patched, partial_c)
>>> mtype_c.f = MethodType(f_monkey_patched, mtype_c)
>>> desc_c.f = f_monkey_patched.__get__(desc_c, C)
>>> %%timeit x = partial_c # Swapping in partial_c, mtype_c or desc_c
... x.f(1)
...
I'm not even going to give exact timing outputs for the IPython %%timeit magic, as it varied across runs, even on a desktop without CPU throttling involved. All I could say for sure is that partial was reliably a little slower, but only by a matter of ~1 ns (the other two typically ran in 56-56.5 ns, the partial solution typically took 56.5-57.5), and it took quite a lot of paring of extraneous stuff (e.g. switching from %timeit reading the names from global scope causing dict lookups to caching to a local name in %%timeit to use simple array lookups) to even get the differences that predictable.
Point is, any of them work, performance-wise. I'd personally recommend either my MethodType or Kedar's explicit use of descriptor protocol approach (they are identical in end result AFAICT; both produce the same bound method class), whichever one looks prettier to you, as it means the bound method is actually a bound method (so you can extract .__self__ and .__func__ like you would on any bound method constructed the normal way, where partial requires you to switch to .args[0] and .func to get the same info).
You can convert the function to bound method by calling its __get__ method (since all function as descriptors as well, thus have this method)
def t(*args, **kwargs):
print(args)
print(kwargs)
class Test():
pass
Test.t = t.__get__(Test(), Test) # binding to the instance of Test
For example
Test().t(1,2, x=1, y=2)
(<__main__.Test object at 0x7fd7f6d845f8>, 1, 2)
{'y': 2, 'x': 1}
Note that the instance is also passed as an positional argument. That is if you want you function to be instance method, the function should have been written in such a way that first argument behaves as instance of the class. Else, you can bind the function to None instance and the class, which will be like staticmethod.
Test.tt = t.__get__(None, Test)
Test.tt(1,2,x=1, y=2)
(1, 2)
{'y': 2, 'x': 1}
Furthermore, to make it a classmethod (first argument is class):
Test.ttt = t.__get__(Test, None) # bind to class
Test.ttt(1,2, x=1, y=2)
(<class '__main__.Test'>, 1, 2)
{'y': 2, 'x': 1}
When you do C.f = f_monkey_patched, and later instantiate an object of C, the function is bound to that object, effectively doing something like
obj.f = functools.partial(C.f, obj)
When you call obj.f(...), you are actually calling the partially bound function, i.e. f_monkey_patched(obj, ...)
On the other hand, doing my_c.f = f_monkey_patched, you assign the function as-is to the attribute my_c.f. When you call my_c.f(...), those arguments are passed to the function as-is, so self is the first argument you passed, i.e. 1, and the remaining arguments go to *args
I am just wondering what the actual explanation is as to why this is valid python.
foo.py
class Foo:
def __init__(self):
if type(self).__name__ in MAP.keys():
print('WOO HOOO')
self.Bar = True
MAP = {'Foo': Foo}
test
>>> from foo import *
>>> x = Foo()
WOO HOOO
I would have thought that the MAP dict when referenced in the constructor would be invalid.
Is this due to the fact that the constructor is only checked at run-time?
You're right, MAP is only looked up when __init__ is run.
If it helps you understand, let's strip away all the non-essential info:
It's irrelevant that __init__ is a method. You can get the same behaviour with a function.
The value of MAP is not important; we just want to check whether it exists
Let's also see what happens when MAP doesn't exist, using a try-except-else.
Put everything in one script
self.Bar and x are unused
def foo():
try:
MAP
except NameError:
print('No!')
else:
print('Yes!')
foo() # -> No!
MAP = 0
foo() # -> Yes!
I'm developing a bot on Python (2.7, 3.4). I defined a about 30+ dynamic functions which to be used based on bot commands. While development, since not all functions are done, I have to define for them an empty functions (if I not define then code won't run) like this:
def c_about():
return
def c_events():
return
def c_currentlocation():
return
etc. many dummy functions.
Question:
it is somehow possible in Python to define same function but with multiple names?
Something like this:
def c_about(), c_events(), c_currentlocation():
return
Yes, it's perfectly possible since defined functions are stored in variables like everything else.
def foo():
pass
baz = bar = foo
There is still some metadata relating to the original function (help(bar) will still mention foo), but it doesn't affect functionality.
Another option is to use lambdas for one-liners:
foo = bar = baz = lambda: None
Functions do not intern in Python (i.e., automatically share multiple references to the same immutable object), but can share the same name:
>>> def a(): pass
...
>>> a
<function a at 0x101c892a8>
>>> def b(): pass
...
>>> b
<function b at 0x101c89320>
>>> c=a
>>> c
<function a at 0x101c892a8> # note the physical address is the same as 'a'
So clearly you can do:
>>> c=d=e=f=g=a
>>> e
<function a at 0x101c892a8>
For the case of functions not yet defined, you can use a try/catch block by catching either a NameError:
def default():
print "default called"
try:
not_defined()
except NameError:
default()
Or use a dict of funcs and catch the KeyError:
funcs={"default": default}
try:
funcs['not_defined']()
except KeyError:
funcs['default']()
Or, you can do funcs.get(not_defined, default)() if you prefer that syntax with a dict of funcs.
Yes, it is possible, and it is even possible to store them in lists using loops. For instance:
l = []
for i in range(10):
l.append(lambda: None)
And you can reference any of them through indices like l[index]()
For example:
c_events = 0
c_about = 1
l[c_events]()
I have one class A which extends B, and B has one method count(). Now I want to allow user call both A.count and A.count(). A.count means count is one field of A while A.count() means it is method derived from B.
This is impossible in Python, and here's why:
You can always assign a method (or really any function) to a variable and call it later.
hello = some_function
hello()
is semantically identical to
some_function()
So what would happen if you had an object of your class A called x:
x = A()
foo = x.count
foo()
The only way you could do this is by storing a special object in x.count that is callable and also turns into e.g. an integer when used in that way, but that is horrible and doesn't actually work according to specification.
As i said, it's not exactly impossible, as told by other answers. Lets see a didactic example:
class A(object):
class COUNT(object):
__val = 12345
def __call__(self, *args, **kwargs):
return self.__val
def __getattr__(self, item):
return self.__val
def __str__(self):
return str(self.__val)
count = COUNT()
if __name__ == '__main__':
your_inst = A()
print(your_inst.count)
# outputs: 12345
print(your_inst.count())
# outputs: 12345
As you may notice, you need to implement a series of things to accomplish that kind of behaviour. First, your class will need to implement the attribute count not as the value type that you intent, but as an instance of another class, which will have to implement, among other things (to make that class behave, by duck typing, as the type you intent) the __call__ method, that should return the same as you A class __getattr__, that way, the public attribute count will answer as a callable (your_inst.count()) or, as you call, a field (your_inst.count), the same way.
By the way, i don't know if the following is clear to you or not, but it may help you understand why it isn't as trivial as one may think it is to make count and count() behave the same way:
class A(object):
def count(self):
return 123
if __name__ == '__main__':
a = A()
print(type(a.count))
# outputs: <class 'method'>
print(type(a.count()))
# outputs: <class 'int'>
. invokes the a class __getattr__ to get the item count. a.count will return the referente to that function (python's function are first class objects), the second one, will do the same, but the parentheses will invoke the __call__ method from a.count.
If I have this function, what should I do to replace the inner function with my own custom version?
def foo():
def bar():
# I want to change this
pass
# here starts a long list of functions I want to keep unchanged
def baz():
pass
Using classes this would be easily done overriding the method. Though, I can't figure out how to do that with nested functions. Changing foo to be a class (or anything else) is not an option because it comes from a given imported module I can't modify.
Here's one way of doing it, creating a new foo that "does the right thing" by hacking the function internals. ( As mentioned by #DSM ). Unfortunately we cant just jump into the foo function and mess with its internals, as they're mostly marked read only, so what we have to do is modify a copy we construct by hand.
# Here's the original function
def foo():
def bar():
print(" In bar orig")
def baz():
print(" Calling bar from baz")
bar()
print("Foo calling bar:")
bar()
print("Foo calling baz:")
baz()
# Here's using it
foo()
# Now lets override the bar function
import types
# This is our replacement function
def my_bar():
print(" Woo hoo I'm the bar override")
# This creates a new code object used by our new foo function
# based on the old foo functions code object.
foocode = types.CodeType(
foo.func_code.co_argcount,
foo.func_code.co_nlocals,
foo.func_code.co_stacksize,
foo.func_code.co_flags,
foo.func_code.co_code,
# This tuple is a new version of foo.func_code.co_consts
# NOTE: Don't get this wrong or you will crash python.
(
foo.func_code.co_consts[0],
my_bar.func_code,
foo.func_code.co_consts[2],
foo.func_code.co_consts[3],
foo.func_code.co_consts[4]
),
foo.func_code.co_names,
foo.func_code.co_varnames,
foo.func_code.co_filename,
foo.func_code.co_name,
foo.func_code.co_firstlineno,
foo.func_code.co_lnotab,
foo.func_code.co_freevars,
foo.func_code.co_cellvars )
# This is the new function we're replacing foo with
# using our new code.
foo = types.FunctionType( foocode , {})
# Now use it
foo()
I'm pretty sure its not going to catch all cases. But it works for the example (for me on an old python 2.5.1 )
Ugly bits that could do with some tidy up are:
The huge argument list being passed to CodeType
The ugly tuple constructed from co_consts overriding only one member. All the info is in co_consts to determine which to replace - so a smarter function could do this. I dug into the internals by hand using print( foo.func_code.co_consts ).
You can find some information about the CodeType and FunctionType by using the interpreter
command help( types.CodeType ).
UPDATE:
I thought this was too ugly so I built a helper function to make it prettier. With the helper you can write:
# Use our function to get a new version of foo with "bar" replaced by mybar
foo = monkey_patch_fn( foo, "bar", my_bar )
# Check it works
foo()
Here's the implementation of monkey_patch_fn:
# Returns a copy of original_fn with its internal function
# called name replaced with new_fn.
def monkey_patch_fn( original_fn, name, new_fn ):
#Little helper function to pick out the correct constant
def fix_consts(x):
if x==None: return None
try:
if x.co_name == name:
return new_fn.func_code
except AttributeError, e:
pass
return x
original_code = original_fn.func_code
new_consts = tuple( map( fix_consts, original_code.co_consts ) )
code_type_args = [
"co_argcount", "co_nlocals", "co_stacksize", "co_flags", "co_code",
"co_consts", "co_names", "co_varnames", "co_filename", "co_name",
"co_firstlineno", "co_lnotab", "co_freevars", "co_cellvars" ]
new_code = types.CodeType(
*[ ( getattr(original_code,x) if x!="co_consts" else new_consts )
for x in code_type_args ] )
return types.FunctionType( new_code, {} )
You can pass it in as an optional parameter
def foo(bar=None):
def _bar():
# I want to change this
pass
if bar is None:
bar = _bar