What's the easiest way to determine which Python class defines an attribute when inheriting? For example, say I have:
class A(object):
defined_in_A = 123
class B(A):
pass
a = A()
b = B()
and I wanted this code to pass:
assert hasattr(a, 'defined_in_A')
assert hasattr(A, 'defined_in_A')
assert hasattr(b, 'defined_in_A')
assert hasattr(B, 'defined_in_A')
assert defines_attribute(A, 'defined_in_A')
assert not defines_attribute(B, 'defined_in_A')
How would I implement the fictional defines_attribute function? My first thought would be to walk through the entire inheritance chain, and use hasattr to check for the attribute's existence, with the deepest match assumed to be the definer. Is there a simpler way?
(Almost) Every python object is defined with it's own instance variables (instance variables of a class object we usually call class variables) to get this as a dictionary you can use the vars function and check for membership in it:
>>> "defined_in_A" in vars(A)
True
>>> "defined_in_A" in vars(B)
False
>>> "defined_in_A" in vars(a) or "defined_in_A" in vars(b)
False
the issue with this is that it does not work when a class uses __slots__ or builtin objects since it changes how the instance variables are stored:
class A(object):
__slots__ = ("x","y")
defined_in_A = 123
>>> A.x
<member 'x' of 'A' objects>
>>> "x" in vars(a)
Traceback (most recent call last):
File "<pyshell#5>", line 1, in <module>
"x" in vars(a)
TypeError: vars() argument must have __dict__ attribute
>>> vars(1) #or floats or strings will raise the same error
Traceback (most recent call last):
...
TypeError: vars() argument must have __dict__ attribute
I'm not sure there is a simple workaround for this case.
Related
I'm trying to write function which creates classes from classes without modifying original one.
Simple solution (based on this answer)
def class_operator(cls):
namespace = dict(vars(cls))
... # modifying namespace
return type(cls.__qualname__, cls.__bases__, namespace)
works fine except type itself:
>>> class_operator(type)
Traceback (most recent call last):
File "<input>", line 1, in <module>
TypeError: type __qualname__ must be a str, not getset_descriptor
Tested on Python 3.2-Python 3.6.
(I know that in current version modification of mutable attributes in namespace object will change original class, but it is not the case)
Update
Even if we remove __qualname__ parameter from namespace if there is any
def class_operator(cls):
namespace = dict(vars(cls))
namespace.pop('__qualname__', None)
return type(cls.__qualname__, cls.__bases__, namespace)
resulting object doesn't behave like original type
>>> type_copy = class_operator(type)
>>> type_copy is type
False
>>> type_copy('')
Traceback (most recent call last):
File "<input>", line 1, in <module>
TypeError: descriptor '__init__' for 'type' objects doesn't apply to 'type' object
>>> type_copy('empty', (), {})
Traceback (most recent call last):
File "<input>", line 1, in <module>
TypeError: descriptor '__init__' for 'type' objects doesn't apply to 'type' object
Why?
Can someone explain what mechanism in Python internals prevents copying type class (and many other built-in classes).
The problem here is that type has a __qualname__ in its __dict__, which is a property (i.e. a descriptor) rather than a string:
>>> type.__qualname__
'type'
>>> vars(type)['__qualname__']
<attribute '__qualname__' of 'type' objects>
And trying to assign a non-string to the __qualname__ of a class throws an exception:
>>> class C: pass
...
>>> C.__qualname__ = 'Foo' # works
>>> C.__qualname__ = 3 # doesn't work
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can only assign string to C.__qualname__, not 'int'
This is why it's necessary to remove the __qualname__ from the __dict__.
As for the reason why your type_copy isn't callable: This is because type.__call__ rejects anything that isn't a subclass of type. This is true for both the 3-argument form:
>>> type.__call__(type, 'x', (), {})
<class '__main__.x'>
>>> type.__call__(type_copy, 'x', (), {})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: descriptor '__init__' for 'type' objects doesn't apply to 'type' object
As well as the single-argument form, which actually only works with type as its first argument:
>>> type.__call__(type, 3)
<class 'int'>
>>> type.__call__(type_copy, 3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: type.__new__() takes exactly 3 arguments (1 given)
This isn't easy to circumvent. Fixing the 3-argument form is simple enough: We make the copy an empty subclass of type.
>>> type_copy = type('type_copy', (type,), {})
>>> type_copy('MyClass', (), {})
<class '__main__.MyClass'>
But the single-argument form of type is much peskier, since it only works if the first argument is type. We can implement a custom __call__ method, but that method must be written in the metaclass, which means type(type_copy) will be different from type(type).
>>> class TypeCopyMeta(type):
... def __call__(self, *args):
... if len(args) == 1:
... return type(*args)
... return super().__call__(*args)
...
>>> type_copy = TypeCopyMeta('type_copy', (type,), {})
>>> type_copy(3) # works
<class 'int'>
>>> type_copy('MyClass', (), {}) # also works
<class '__main__.MyClass'>
>>> type(type), type(type_copy) # but they're not identical
(<class 'type'>, <class '__main__.TypeCopyMeta'>)
There are two reasons why type is so difficult to copy:
It's implemented in C. You'll run into similar problems if you try to copy other builtin types like int or str.
The fact that type is an instance of itself:
>>> type(type)
<class 'type'>
This is something that's usually not possible. It blurs the line between class and instance. It's a chaotic accumulation of instance and class attributes. This is why __qualname__ is a string when accessed as type.__qualname__ but a descriptor when accessed as vars(type)['__qualname__'].
As you can see, it's not possible to make a perfect copy of type. Each implementation has different tradeoffs.
The easy solution is to make a subclass of type, which doesn't support the single-argument type(some_object) call:
import builtins
def copy_class(cls):
# if it's a builtin class, copy it by subclassing
if getattr(builtins, cls.__name__, None) is cls:
namespace = {}
bases = (cls,)
else:
namespace = dict(vars(cls))
bases = cls.__bases__
cls_copy = type(cls.__name__, bases, namespace)
cls_copy.__qualname__ = cls.__qualname__
return cls_copy
The elaborate solution is to make a custom metaclass:
import builtins
def copy_class(cls):
if cls is type:
namespace = {}
bases = (cls,)
class metaclass(type):
def __call__(self, *args):
if len(args) == 1:
return type(*args)
return super().__call__(*args)
metaclass.__name__ = type.__name__
metaclass.__qualname__ = type.__qualname__
# if it's a builtin class, copy it by subclassing
elif getattr(builtins, cls.__name__, None) is cls:
namespace = {}
bases = (cls,)
metaclass = type
else:
namespace = dict(vars(cls))
bases = cls.__bases__
metaclass = type
cls_copy = metaclass(cls.__name__, bases, namespace)
cls_copy.__qualname__ = cls.__qualname__
return cls_copy
I want to provide a method that can be used on a Python 2.7 class object, but does not pollute the attribute namespace of its instances. Is there any way to do this?
>>> class Foo(object):
... #classmethod
... def ugh(cls):
... return 33
...
>>> Foo.ugh()
33
>>> foo = Foo()
>>> foo.ugh()
33
You could subclass the classmethod descriptor:
class classonly(classmethod):
def __get__(self, obj, type):
if obj: raise AttributeError
return super(classonly, self).__get__(obj, type)
This is how it would behave:
class C(object):
#classonly
def foo(cls):
return 42
>>> C.foo()
42
>>> c=C()
>>> c.foo()
AttributeError
This desugars to the descriptor call (rather, it is invoked by the default implementation of __getattribute__):
>>> C.__dict__['foo'].__get__(None, C)
<bound method C.foo of <class '__main__.C'>>
>>> C.__dict__['foo'].__get__(c, type(c))
AttributeError
Required reading: Data Model — Implementing Descriptors and Descriptor HowTo Guide.
ugh is not in the namespace:
>>> foo.__dict__
{}
but the rules for attribute lookup fall back to the type of the instance for missing names. You can override Foo.__getattribute__ to prevent this.
class Foo(object):
#classmethod
def ugh(cls):
return 33
def __getattribute__(self, name):
if name == 'ugh':
raise AttributeError("Access to class method 'ugh' block from instance")
return super(Foo,self).__getattribute__(name)
This produces:
>>> foo = Foo()
>>> foo.ugh()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "tmp.py", line 8, in __getattribute__
raise AttributeError("Access to class method 'ugh' block from instance")
AttributeError: Access to class method 'ugh' block from instance
>>> Foo.ugh()
33
You must use __getattribute__, which is called unconditionally on any attribute access, rather than __getattr__, which is only called after the normal lookup (which includes checking the type's namespace) fails.
Python has quasi-private variables that use name-munging to reduce accidental access. Methods and object variables of the form __name are converted to _ClassName__name. Python automatically changes the name when compiling methods on the class but doesn't change the name for subclasses.
I can use the private method in a class
>>> class A(object):
... def __private(self):
... print('boo')
... def hello(self):
... self.__private()
...
>>>
>>> A().hello()
boo
But not outside the class
>>> A().__private()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'A' object has no attribute '__private'
>>>
Or in subclasses
>>> class B(A):
... def hello2(self):
... self.__private()
...
>>>
>>> B().hello()
boo
>>> B().hello2()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in hello2
AttributeError: 'B' object has no attribute '_B__private'
Yes, you can create the method in the metaclass.
class FooMeta(type):
# No #classmethod here
def ugh(cls):
return 33
class Foo(object):
__metaclass__ = FooMeta
Foo.ugh() # returns 33
Foo().ugh() # AttributeError
Note that metaclasses are a power feature, and their use is discouraged if unnecessary. In particular, multiple inheritance requires special care if the parent classes have different metaclasses.
Let's say a function looks at an object and checks if it has a function a_method:
def func(obj):
if hasattr(obj, 'a_method'):
...
else:
...
I have an object whose class defines a_method, but I want to hide it from hasattr. I don't want to change the implementation of func to achieve this hiding, so what hack can I do to solve this problem?
If the method is defined on the class you appear to be able to remove it from the __dict__ for the class. This prevents lookups (hasattr will return false). You can still use the function if you keep a reference to it when you remove it (like the example) - just remember that you have to pass in an instance of the class for self, it's not being called with the implied self.
>>> class A:
... def meth(self):
... print "In method."
...
>>>
>>> a = A()
>>> a.meth
<bound method A.meth of <__main__.A instance at 0x0218AB48>>
>>> fn = A.__dict__.pop('meth')
>>> hasattr(a, 'meth')
False
>>> a.meth
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: A instance has no attribute 'meth'
>>> fn()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: meth() takes exactly 1 argument (0 given)
>>> fn(a)
In method.
You could redefine the hasattr function. Below is an example.
saved_hasattr = hasattr
def hasattr(obj, method):
if method == 'MY_METHOD':
return False
else:
return saved_hasattr(obj, method)
Note that you probably want to implement more detailed checks than just checking the method name. For example checking the object type might be beneficial.
Try this:
class Test(object):
def __hideme(self):
print 'hidden'
t = Test()
print hasattr(t,"__hideme") #prints False....
I believe this works b/c of the double underscore magic of hiding members (owning to name mangling) of a class to outside world...Unless someone has a strong argument against this, I'd think this is way better than popping stuff off from __dict__? Thoughts?
I know with getattr() you can call the method, but I need to overwrite it, so myInstance.mymethod will be overwritten.
I have the method's name as a string and the instance's reference.
You can overwrite it with setattr
>>> class Foo(object):
... def method(self): pass
...
>>> a = Foo()
>>> a.method()
>>> setattr(a,'method',1)
>>> a.method()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callable
To replace with another method:
>>> import types
>>> setattr(a,'method',types.MethodType(lambda self: self.__class__.__name__,a))
>>> a.method()
'Foo'
Where the lambda stuff is just fancy shorthand for defining a function:
def func(self):
return self.__class__.__name__
setattr(a,'method',types.MethodType(func,a))
My question is pretty simple, I have:
class upperstr(str):
def __new__(cls, arg):
return str.__new__(cls, str(arg).upper())
Why, if my __new__() method is directly using an instance of an inmutable type (str), instances of my new type (upperstr) are mutable?
>>> s = str("text")
>>> "__dict__" in dir(s)
False
>>> s = upperstr("text")
>>> "__dict__" in dir(s)
True
In what stage does the interpreter sets the __dict__ attribute to upperstr intances if I'm only overriding the __new__() method?
Thanks!
All user-defined classes in Python have a __dict__() attribute by default, even if you don't overwrite anything at all:
>>> x = object()
>>> x.__dict__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'object' object has no attribute '__dict__'
>>> class MyObject(object):
... pass
...
>>> x = MyObject()
>>> x.__dict__
{}
If you don't want a new-style class to have a __dict__, use __slots__ (documentation, related SO thread):
>>> class MyObject(object):
... __slots__ = []
...
>>> x = MyObject()
>>> x.__dict__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'MyObject' object has no attribute '__dict__'