Let's say I have this class:
class A:
def __init__(self, a):
self.a = a
#classmethod
def foo(self):
return 'hello world!'
I use #classmethod, so that I can directly call the function without calling the class:
>>> A.foo()
'hello world!'
>>>
But now I am wondering, since I still can access it with calling the class:
>>> A(1).foo()
'hello world!'
>>>
Would I be able to make it that it would raise an error if the function foo is called from a called class. And only let it to be called without calling the class, like A.foo().
So if I do:
A(1).foo()
It should give an error.
The functionality of how classmethod, staticmethod and in fact normal methods are lookedup / bound is implemented via descriptors. Similarly, one can define a descriptor that forbids lookup/binding on an instance.
A naive implementation of such a descriptor checks whether it is looked up via an instance and raises an error in this case:
class NoInstanceMethod:
"""Descriptor to forbid that other descriptors can be looked up on an instance"""
def __init__(self, descr, name=None):
self.descr = descr
self.name = name
def __set_name__(self, owner, name):
self.name = name
def __get__(self, instance, owner):
# enforce the instance cannot look up the attribute at all
if instance is not None:
raise AttributeError(f"{type(instance).__name__!r} has no attribute {self.name!r}")
# invoke any descriptor we are wrapping
return self.descr.__get__(instance, owner)
This can be applied on top of other descriptors to prevent them from being looked up on an instance. Prominently, it can be combined with classmethod or staticmethod to prevent using them on an instance:
class A:
def __init__(self, a):
self.a = a
#NoInstanceMethod
#classmethod
def foo(cls):
return 'hello world!'
A.foo() # Stdout: hello world!
A(1).foo() # AttributeError: 'A' object has no attribute 'foo'
The above NoInstanceMethod is "naive" in that it does not take care of propagating descriptor calls other than __get__ to its wrapped descriptor. For example, one could propagate __set_name__ calls to allow the wrapped descriptor to know its name.
Since descriptors are free to (not) implement any of the descriptor methods, this can be supported but needs appropriate error handling. Extend the NoInstanceMethod to support whatever descriptor methods are needed in practice.
A workaround is to override its value upon initialization of a class object to make sure it wouldn't be called from self.
def raise_(exc):
raise exc
class A:
STRICTLY_CLASS_METHODS = [
"foo",
]
def __init__(self, a):
self.a = a
for method in self.STRICTLY_CLASS_METHODS:
# Option 1: Using generator.throw() to raise exception. See https://www.python.org/dev/peps/pep-0342/#new-generator-method-throw-type-value-none-traceback-none
# setattr(self, method, lambda *args, **kwargs: (_ for _ in ()).throw(AttributeError(method)))
# Option 2: Using a function to raise exception
setattr(self, method, lambda *args, **kwargs: raise_(AttributeError(method)))
#classmethod
def foo(cls):
return 'hello world!'
def bar(self):
return 'hola mundo!', self.a
Output
>>> A.foo()
'hello world!'
>>> a = A(123)
>>> a.bar()
('hola mundo!', 123)
>>> a.foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 11, in <lambda>
File "<stdin>", line 2, in raise_
AttributeError: foo
>>> a.bar()
('hola mundo!', 123)
>>> A(45).bar()
('hola mundo!', 45)
>>> A(6789).foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 11, in <lambda>
File "<stdin>", line 2, in raise_
AttributeError: foo
>>> A.foo()
'hello world!'
Related
for the function,
def add(a, b):
return a + b
when I do,
add.__get__(object(), object)
<bound method add of <object object at 0x7f3c7af42b90>>
add.__get__(list(), list)
<bound method add of []>
add.__get__(bool(), bool)
<bound method add of False>
what is happening here?
Functions are descriptors and use the descriptor protocol to bind the instance the method is called on to the first argument. This is how "self works". So, if you look at the Descriptor HOWTO it gives you an example of how it would be implemented in Python:
class Function:
...
def __get__(self, obj, objtype=None):
"Simulate func_descr_get() in Objects/funcobject.c"
if obj is None:
return self
return MethodType(self, obj)
So, here are examples of calling the returned callable with different types of objects:
>>> def frobnicate(first):
... return first.foo + 1
...
>>> class Foo:
... foo = 42
...
>>> class Bar:
... foo = "bar"
...
>>> class Baz:
... bar = 42
...
>>> frobnicate.__get__(Foo())()
43
>>> frobnicate.__get__(Bar())()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in frobnicate
TypeError: can only concatenate str (not "int") to str
>>> frobnicate.__get__(Baz())()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in frobnicate
AttributeError: 'Baz' object has no attribute 'foo'
And of course, putting the function in some class namespace makes it act like any other method automatically:
>>> Foo.frobnicate = frobnicate
>>> Foo().frobnicate()
43
Note, you can mentally think of MethodType as just something like:
def bind_first(func, obj):
def bound(*args, **kwargs):
return func(obj, *args, **kwargs)
return bound
Although, if you check the link, it shows a closer implementation (in Python, instead of C):
Methods can be created manually with types.MethodType which is roughly equivalent to:
class MethodType:
"Emulate PyMethod_Type in Objects/classobject.c"
def __init__(self, func, obj):
self.__func__ = func
self.__self__ = obj
def __call__(self, *args, **kwargs):
func = self.__func__
obj = self.__self__
return func(obj, *args, **kwargs)
Which is basically doing the same thing as the bind_first function I gave as an example... (although, again, it's using a class and it also creates the __func__ and __self__ arguments that bound-methods have).
it appears that,
def f():
pass
when I do,
hasattr(f, '__get__')
then, it returns True
when I do,
class A:
pass
a = A()
f.__get__(a, A) # I think the second argument is optional here
# so, even, f.__get__(a) would give same result
it returns,
<bound method f of <__main__.A object at 0x7f9c7568dd10>>
which means that our function is similar to something like this,
class B:
def func_(self):
pass
b = B()
b.func_
<bound method B.func_ of <__main__.B object at 0x7f9c7563cc50>>
and then when I call the function on top,
f.__get__(a)()
then, it gives an error,
TypeError: f() takes 0 positional arguments but 1 was given
which means, that I am passing the instance here,
similar to func_, where,
the first argument self refers to the instance of the class B.
so, the function we have on top, becomes a bound method, and if I change it to,
def f(self):
print(self)
and then run,
f.__get__(a)()
it gives me,
<__main__.A object at 0x7f9c7568dd10>
one more thing to notice here is that we could bind our function to a class also, which would make it a class method,
f.__get__(A)
<bound method func of <class '__main__.A'>>
where,
f.__get__(A)()
gives,
<class '__main__.A'>
which is similar to,
class C:
#classmethod
def x(cls):
print(cls)
C.x
<bound method C.x of <class '__main__.C'>>
or we could keep our function as a static method, without binding it to an instance or a class.
f.__get__(1)
<bound method func of 1>
where,
f.__get__(1)()
gives,
1
here, f would be a function with the first argument equal to 1.
I'm working with Python3, and I have a really heavy class with many functions as attributes:
Class A (object):
def __init__(self):
...
def method1(self):
...
def method2(self):
...
...
def methodN(self):
...
I would like to create an instance of class A that only has method1, for example. How could I do this?
Using inheritance, though it might be the most technically correct way, is not an option in my case - I can't modify the codebase so much.
I thought about decorating the class and deleting its attributes before __init__ is called, but I'm not even sure where to start tackling this. Any ideas?
You can modify the __getattribute__ method of the class to disallow access to those attributes (via normal instance.attribute access)
class A (object):
def __init__(self, x):
self.x = x
def method1(self):
...
def method2(self):
...
def __getattribute__(self, name):
if object.__getattribute__(self, 'x'):
if name == 'method2':
raise AttributeError("Cannot access method2 is self.x is True")
return object.__getattribute__(self, name)
>>> a = A(False)
>>> a.method1
<bound method A.method1 of <__main__.A object at 0x000001E25992F248>>
>>> a.method2
<bound method A.method2 of <__main__.A object at 0x000001E25992F248>>
>>> b = A(True)
>>> b.method1
<bound method A.method1 of <__main__.A object at 0x000001E25992F2C8>>
>>> b.method2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 11, in __getattribute__
AttributeError: Cannot access method2 is self.x is True
Obviously, this gets pretty unwieldy and violates a lot of assumptions about what it means to be an instance of a class. I can't think of a good reason to do this in real code, as you can still access the methods through object.__getattribute__(b, 'method2')
When I try to monkey-patch a class with a method from another class, it doesn't work because the argument self isn't of the right type.
For example, let's like the result of the method __str__ created by the fancy class A:
class A:
def __init__(self, val):
self.val=val
def __str__(self):
return "Fancy formatted %s"%self.val
and would like to reuse it for a boring class B:
class B:
def __init__(self, val):
self.val=val
That means:
>>> b=B("B")
>>> #first try:
>>> B.__str__=A.__str__
>>> str(b)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unbound method __str__() must be called with A instance as first argument (got nothing instead)
>>> #second try:
>>> B.__str__= lambda self: A.__str__(self)
>>> str(b)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 1, in <lambda>
TypeError: unbound method __str__() must be called with A instance as first argument (got B instance instead)
So in both cases the it doesn't work because the argument self should be an instance of class A, but evidently isn't.
It would be nice to find a way to do the monkey-patching, but my actual question is, why it is necessary for the implicit parameter self to be an instance of the "right" class and not just depend on the duck-typing?
Because of the way methods are contributed to class objects in Python 2, the actual function object is hidden behind an unbound method, but you can access it using the im_func aka __func__ attribute:
>>> B.__str__ = A.__str__.__func__
>>> str(B('stuff'))
'Fancy formatted stuff'
Arguably, a better way to do this is using new-style classes and inheritance.
class MyStrMixin(object):
def __str__(self):
return "Fancy formatted %s" % self.val
Then inherit from MyStrMixin in both A and B, and just let the MRO do its thing.
I want to provide a method that can be used on a Python 2.7 class object, but does not pollute the attribute namespace of its instances. Is there any way to do this?
>>> class Foo(object):
... #classmethod
... def ugh(cls):
... return 33
...
>>> Foo.ugh()
33
>>> foo = Foo()
>>> foo.ugh()
33
You could subclass the classmethod descriptor:
class classonly(classmethod):
def __get__(self, obj, type):
if obj: raise AttributeError
return super(classonly, self).__get__(obj, type)
This is how it would behave:
class C(object):
#classonly
def foo(cls):
return 42
>>> C.foo()
42
>>> c=C()
>>> c.foo()
AttributeError
This desugars to the descriptor call (rather, it is invoked by the default implementation of __getattribute__):
>>> C.__dict__['foo'].__get__(None, C)
<bound method C.foo of <class '__main__.C'>>
>>> C.__dict__['foo'].__get__(c, type(c))
AttributeError
Required reading: Data Model — Implementing Descriptors and Descriptor HowTo Guide.
ugh is not in the namespace:
>>> foo.__dict__
{}
but the rules for attribute lookup fall back to the type of the instance for missing names. You can override Foo.__getattribute__ to prevent this.
class Foo(object):
#classmethod
def ugh(cls):
return 33
def __getattribute__(self, name):
if name == 'ugh':
raise AttributeError("Access to class method 'ugh' block from instance")
return super(Foo,self).__getattribute__(name)
This produces:
>>> foo = Foo()
>>> foo.ugh()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "tmp.py", line 8, in __getattribute__
raise AttributeError("Access to class method 'ugh' block from instance")
AttributeError: Access to class method 'ugh' block from instance
>>> Foo.ugh()
33
You must use __getattribute__, which is called unconditionally on any attribute access, rather than __getattr__, which is only called after the normal lookup (which includes checking the type's namespace) fails.
Python has quasi-private variables that use name-munging to reduce accidental access. Methods and object variables of the form __name are converted to _ClassName__name. Python automatically changes the name when compiling methods on the class but doesn't change the name for subclasses.
I can use the private method in a class
>>> class A(object):
... def __private(self):
... print('boo')
... def hello(self):
... self.__private()
...
>>>
>>> A().hello()
boo
But not outside the class
>>> A().__private()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'A' object has no attribute '__private'
>>>
Or in subclasses
>>> class B(A):
... def hello2(self):
... self.__private()
...
>>>
>>> B().hello()
boo
>>> B().hello2()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in hello2
AttributeError: 'B' object has no attribute '_B__private'
Yes, you can create the method in the metaclass.
class FooMeta(type):
# No #classmethod here
def ugh(cls):
return 33
class Foo(object):
__metaclass__ = FooMeta
Foo.ugh() # returns 33
Foo().ugh() # AttributeError
Note that metaclasses are a power feature, and their use is discouraged if unnecessary. In particular, multiple inheritance requires special care if the parent classes have different metaclasses.
I wish to create a class in Python that I can add and remove attributes and methods. How can I acomplish that?
Oh, and please don't ask why.
This example shows the differences between adding a method to a class and to an instance.
>>> class Dog():
... def __init__(self, name):
... self.name = name
...
>>> skip = Dog('Skip')
>>> spot = Dog('Spot')
>>> def talk(self):
... print 'Hi, my name is ' + self.name
...
>>> Dog.talk = talk # add method to class
>>> skip.talk()
Hi, my name is Skip
>>> spot.talk()
Hi, my name is Spot
>>> del Dog.talk # remove method from class
>>> skip.talk() # won't work anymore
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: Dog instance has no attribute 'talk'
>>> import types
>>> f = types.MethodType(talk, skip, Dog)
>>> skip.talk = f # add method to specific instance
>>> skip.talk()
Hi, my name is Skip
>>> spot.talk() # won't work, since we only modified skip
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: Dog instance has no attribute 'talk'
I wish to create a class in Python that I can add and remove attributes and methods.
import types
class SpecialClass(object):
#classmethod
def removeVariable(cls, name):
return delattr(cls, name)
#classmethod
def addMethod(cls, func):
return setattr(cls, func.__name__, types.MethodType(func, cls))
def hello(self, n):
print n
instance = SpecialClass()
SpecialClass.addMethod(hello)
>>> SpecialClass.hello(5)
5
>>> instance.hello(6)
6
>>> SpecialClass.removeVariable("hello")
>>> instance.hello(7)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'SpecialClass' object has no attribute 'hello'
>>> SpecialClass.hello(8)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: type object 'SpecialClass' has no attribute 'hello'
A possibly interesting alternative to using types.MethodType in:
>>> f = types.MethodType(talk, puppy, Dog)
>>> puppy.talk = f # add method to specific instance
would be to exploit the fact that functions are descriptors:
>>> puppy.talk = talk.__get__(puppy, Dog)
I wish to create a class in Python that I can add and remove attributes and methods. How can I acomplish that?
You can add and remove attributes and methods to any class, and they'll be available to all instances of the class:
>>> def method1(self):
pass
>>> def method1(self):
print "method1"
>>> def method2(self):
print "method2"
>>> class C():
pass
>>> c = C()
>>> c.method()
Traceback (most recent call last):
File "<pyshell#62>", line 1, in <module>
c.method()
AttributeError: C instance has no attribute 'method'
>>> C.method = method1
>>> c.method()
method1
>>> C.method = method2
>>> c.method()
method2
>>> del C.method
>>> c.method()
Traceback (most recent call last):
File "<pyshell#68>", line 1, in <module>
c.method()
AttributeError: C instance has no attribute 'method'
>>> C.attribute = "foo"
>>> c.attribute
'foo'
>>> c.attribute = "bar"
>>> c.attribute
'bar'
you can just assign directly to the class (either by accessing the original class name or via __class__ ):
class a : pass
ob=a()
ob.__class__.blah=lambda self,k: (3, self,k)
ob.blah(5)
ob2=a()
ob2.blah(7)
will print
(3, <__main__.a instance at 0x7f18e3c345f0>, 5)
(3, <__main__.a instance at 0x7f18e3c344d0>, 7)
Simply:
f1 = lambda:0 #method for instances
f2 = lambda _:0 #method for class
class C: pass #class
c1,c2 = C(),C() #instances
print dir(c1),dir(c2)
#add to the Instances
c1.func = f1
c1.any = 1.23
print dir(c1),dir(c2)
print c1.func(),c1.any
del c1.func,c1.any
#add to the Class
C.func = f2
C.any = 1.23
print dir(c1),dir(c2)
print c1.func(),c1.any
print c2.func(),c2.any
which results in:
['__doc__', '__module__'] ['__doc__', '__module__']
['__doc__', '__module__', 'any', 'func'] ['__doc__', '__module__']
0 1.23
['__doc__', '__module__', 'any', 'func'] ['__doc__', '__module__', 'any', 'func']
0 1.23
0 1.23
another alternative, if you need to replace the class wholesale is to modify the class attribute:
>>> class A(object):
... def foo(self):
... print 'A'
...
>>> class B(object):
... def foo(self):
... print 'Bar'
...
>>> a = A()
>>> a.foo()
A
>>> a.__class__ = B
>>> a.foo()
Bar
Does the class itself necessarily need to be modified? Or is the goal simply to replace what object.method() does at a particular point during runtime?
I ask because I sidestep the problem of actually modifying the class to monkey patch specific method calls in my framework with getattribute and a Runtime Decorator on my Base inheritance object.
Methods retrieved by a Base object in getattribute are wrapped in a Runtime_Decorator that parses the method calls keyword arguments for decorators/monkey patches to apply.
This enables you to utilize the syntax object.method(monkey_patch="mypatch"), object.method(decorator="mydecorator"), and even object.method(decorators=my_decorator_list).
This works for any individual method call (I leave out magic methods), does so without actually modifying any class/instance attributes, can utilize arbitrary, even foreign methods to patch, and will work transparently on sublcasses that inherit from Base (provided they don't override getattribute of course).
import trace
def monkey_patched(self, *args, **kwargs):
print self, "Tried to call a method, but it was monkey patched instead"
return "and now for something completely different"
class Base(object):
def __init__(self):
super(Base, self).__init__()
def testmethod(self):
print "%s test method" % self
def __getattribute__(self, attribute):
value = super(Base, self).__getattribute__(attribute)
if "__" not in attribute and callable(value):
value = Runtime_Decorator(value)
return value
class Runtime_Decorator(object):
def __init__(self, function):
self.function = function
def __call__(self, *args, **kwargs):
if kwargs.has_key("monkey_patch"):
module_name, patch_name = self._resolve_string(kwargs.pop("monkey_patch"))
module = self._get_module(module_name)
monkey_patch = getattr(module, patch_name)
return monkey_patch(self.function.im_self, *args, **kwargs)
if kwargs.has_key('decorator'):
decorator_type = str(kwargs['decorator'])
module_name, decorator_name = self._resolve_string(decorator_type)
decorator = self._get_decorator(decorator_name, module_name)
wrapped_function = decorator(self.function)
del kwargs['decorator']
return wrapped_function(*args, **kwargs)
elif kwargs.has_key('decorators'):
decorators = []
for item in kwargs['decorators']:
module_name, decorator_name = self._resolve_string(item)
decorator = self._get_decorator(decorator_name, module_name)
decorators.append(decorator)
wrapped_function = self.function
for item in reversed(decorators):
wrapped_function = item(wrapped_function)
del kwargs['decorators']
return wrapped_function(*args, **kwargs)
else:
return self.function(*args, **kwargs)
def _resolve_string(self, string):
try: # attempt to split the string into a module and attribute
module_name, decorator_name = string.split(".")
except ValueError: # there was no ".", it's just a single attribute
module_name = "__main__"
decorator_name = string
finally:
return module_name, decorator_name
def _get_module(self, module_name):
try: # attempt to load the module if it exists already
module = modules[module_name]
except KeyError: # import it if it doesn't
module = __import__(module_name)
finally:
return module
def _get_decorator(self, decorator_name, module_name):
module = self._get_module(module_name)
try: # attempt to procure the decorator class
decorator_wrap = getattr(module, decorator_name)
except AttributeError: # decorator not found in module
print("failed to locate decorators %s for function %s." %\
(kwargs["decorator"], self.function))
else:
return decorator_wrap # instantiate the class with self.function
class Tracer(object):
def __init__(self, function):
self.function = function
def __call__(self, *args, **kwargs):
tracer = trace.Trace(trace=1)
tracer.runfunc(self.function, *args, **kwargs)
b = Base()
b.testmethod(monkey_patch="monkey_patched")
b.testmethod(decorator="Tracer")
#b.testmethod(monkey_patch="external_module.my_patch")
The downside to this approach is getattribute hooks all access to attributes, so the checking of and potential wrapping of methods occurs even for attributes that are not methods + won't be utilizing the feature for the particular call in question. And using getattribute at all is inherently somewhat complicated.
The actual impact of this overhead in my experience/for my purposes has been negligible, and my machine runs a dual core Celeron. The previous implementation I used introspected methods upon object init and bound the Runtime_Decorator to methods then. Doing things that way eliminated the need to utilize getattribute and reduced the overhead mentioned previously... however, it also breaks pickle (maybe not dill) and is less dynamic then this approach.
The only use cases I have actually come across "in the wild" with this technique were with timing and tracing decorators. However, the possibilities it opens up are extremely wide ranging.
If you have a preexisting class that cannot be made to inherit from a different base (or utilize the technique it's own class definition or in it's base class'), then the whole thing simply does not apply to your issue at all unfortunately.
I don't think setting/removing non-callable attributes on a class at runtime is necessarily so challenging? unless you want classes that inherit from the modified class to automatically reflect the changes in themselves as well... That'd be a whole 'nother can o' worms by the sound of it though.