Develop Reference

Remove a property (getter/setter) from an attribute in subclass

This may have been answered somewhere else, but I was wondering if there was any way to remove an attribute/method decorated with #property in a subclass.
Example:
from datetime import datetime
class A():
def __init__(self, num):
self._num = num
#property
def id(self):
return self._num * datetime.now().timestamp()
class B(A):
def __init__(self, id, num):
super().__init__(num)
self.id = id
The above code does not run if you attempt to create an instance of class B. AttributeError: can't set attribute
The base class uses a property because it needs to evaluate its ID on the fly, while my sub class is able to know its ID when it is created. The id attribute is accessed OFTEN, and I am seeing a significant performance hit because I have to use a property to serve this attribute, instead of just accessing it directly. (From what I have read, properties increase time-to-access by 5x). My application is currently spending around 10% of runtime getting this property.
Is there any way I can short-circuit the property in a sub class?

I'm going to go through several possibilities here. Some of them do what you literally asked. Some of them don't, but they may be better options anyway.
First, your example base class changes the value of obj.id on every access due to the passage of time. That's really bizarre and doesn't seem like a useful concept of "ID". If your real use case has a stable obj.id return value, then you can cache it to avoid the expense of recomputation:
def __init__(self):
...
self._id = None
#property
def id(self):
if self._id is not None:
return self._id
retval = self._id = expensive_computation()
return retval
This may mitigate the expense of the property. If you need more mitigation, look for places where you access id repeatedly, and instead, access it once and save it in a variable. Local variable lookup outperforms attribute access no matter how the attribute is implemented. (Of course, if you actually do have weird time-variant IDs, then this sort of refactoring may not be valid.)
Second, you can't override a property with a "regular" attribute, but you can create your own version of property that can be overridden this way. Your property blocks attribute setting, and takes priority over "regular" attributes even if you force an entry into the instance __dict__, because property has a __set__ method (even if you don't write a setter). Writing your own descriptor without a __set__ would allow overriding. You could do it with a generic LowPriorityProperty:
class LowPriorityProperty(object):
"""
Like #property, but no __set__ or __delete__, and does not take priority
over the instance __dict__.
"""
def __init__(self, fget):
self.fget = fget
def __get__(self, instance, owner=None):
if instance is None:
return self
return self.fget(instance)
class Foo(object):
...
#LowPriorityProperty
def id(self):
...
class Bar(Foo):
def __init__(self):
super(Bar, self).__init__()
self.id = whatever
...
Or with a role-specific descriptor class:
class IDDescriptor(object):
def __get__(self, instance, owner=None):
if instance is None:
return self
# Remember, self is the descriptor. instance is the object you're
# trying to compute the id attribute of.
return whatever(instance)
class Foo(object):
id = IDDescriptor()
...
class Bar(Foo):
def __init__(self):
super(Bar, self).__init__()
self.id = whatever
...
The role-specific descriptor performs better than the generic LowPriorityProperty, but both perform worse than property due to implementing more logic in Python instead of C.
Finally, you can't override a property with a "regular" attribute, but you can override it with another descriptor, such as another property, or such as the descriptors created for __slots__. If you're really, really pressed for performance, __slots__ is probably more performant than any descriptor you could implement manually, but the interaction between __slots__ and the property is weird and obscure and you'll probably want to leave a comment explaining what you're doing.
class Foo(object):
#property
def id(self):
...
class Bar(Foo):
__slots__ = ('id',)
def __init__(self):
super(Bar, self).__init__()
self.id = whatever
...

add a class C as common ancestor, without id. inherit A and B from it and implement id there as needed. Python wont care that id doesn’t exist on C.
refactor non-id code/attributes from A to C.
Suitability depends on whether OP controls class hierarchy and instantiation mechanisms.
I also found a workaround to get it working as is:
from datetime import datetime
class A():
def __init__(self, num):
self._num = num
#property
def id(self):
return self._num * datetime.now().timestamp()
class B(A):
#this fixes the problem
id = None
def __init__(self, id, num):
super().__init__(num)
self.id = id
b = B("id", 3)
print(vars(b))
This will output:
{'_num': 3, 'id': 'id'}
The trick is id = None on class B. Basically, Python's attribute/method lookup mechanism will stop at the first class with id as an attribute in the MRO. With id = None on class B, the lookup stops there and it never gets as far as that pesky #property on A.
If I comment it back out, as per the OP:
self.id = id
AttributeError: can't set attribute

A good practice to implement with python multiple inheritance class?

The Scenario:
class A:
def __init__(self, key, secret):
self.key = key
self.secret = secret
def same_name_method(self):
do_some_staff
def method_a(self):
pass
class B:
def __init__(self, key, secret):
self.key = key
self.secret = secret
def same_name_method(self):
do_another_staff
def method_b(self):
pass
class C(A,B):
def __init__(self, *args, **kwargs):
# I want to init both class A and B's key and secret
## I want to rename class A and B's same method
any_ideas()
...
What I Want:
I want the instance of class C initialize both class A and B, because they are different api key.
And I want rename class A and B's same_name_method, so I will not confused at which same_name_method.
What I Have Done:
For problem one, I have done this:
class C(A,B):
def __init__(self, *args, **kwargs):
A.__init__(self, a_api_key,a_api_secret)
B.__init__(self, b_api_key,b_api_secret)
Comment: I know about super(), but for this situation I do not know how to use it.
For problem two, I add a __new__ for class C
def __new__(cls, *args, **kwargs):
cls.platforms = []
cls.rename_method = []
for platform in cls.__bases__:
# fetch platform module name
module_name = platform.__module__.split('.')[0]
cls.platforms.append(module_name)
# rename attr
for k, v in platform.__dict__.items():
if not k.startswith('__'):
setattr(cls, module_name+'_'+k, v)
cls.rename_method.append(k)
for i in cls.rename_method:
delattr(cls, i) ## this line will raise AttributeError!!
return super().__new__(cls)
Comment: because I rename the new method names and add it to cls attr. I need to delete the old method attr, but do not know how to delattr. Now I just leave them alone, did not delete the old methods.
Question:
Any Suggestions?

So, you want some pretty advanced things, some complicated things, and you don't understand well how classes behave in Python.
So, for your first thing: initializing both classes, and every other method that should run in all classes: the correct solution is to make use of cooperative calls to super() methods.
A call to super() in Python returns you a very special proxy objects that reflects all methods available in the next class, obeying the proper method Resolution Order.
So, if A.__init__ and B.__init__ have to be called, both methods should include a super().__init__ call - and one will call the other's __init__ in the appropriate order, regardless of how they are used as bases in subclasses. As object also have __init__, the last super().__init__ will just call it that is a no-op. If you have more methods in your classes that should be run in all base classes, you'd rather build a proper base class so that the top-most super() call don't try to propagate to a non-existing method.
Otherwise, it is just:
class A:
def __init__(self, akey, asecret, **kwargs):
self.key = akey
self.secret = asecret
super().__init__(**kwargs)
class B:
def __init__(self, bkey, bsecret, **kwargs):
self.key = bkey
self.secret = bsecret
super().__init__(**kwargs)
class C(A,B):
# does not even need an explicit `__init__`.
I think you can get the idea. Of course, the parameter names have to differ - ideally, when writing C you don't have to worry about parameter order - but when calling C you have to worry about suplying all mandatory parameters for C and its bases. If you can't rename the parameters in A or B to be distinct, you could try to use the parameter order for the call, though, with each __init__ consuming two position-parameters - but that will require some extra care in inheritance order.
So - up to this point, it is basic Python multiple-inheritance "howto", and should be pretty straightforward. Now comes your strange stuff.
As for the auto-renaming of methods: first things first -
are you quite sure you need inheritance? Maybe having your granular classes for each external service, and a registry and dispatch class that call the methods on the others by composition would be more sane. (I may come back to this later)
Are you aware that __new__ is called for each instantiation of the class, and all class-attribute mangling you are performing there happens at each new instance of your classes?
So, if the needed method-renaming + shadowing needs to take place at class creation time, you can do that using the special method __init_subclass__ that exists from Python 3.6. It is a special class method that is called once for each derived class of the class it is defined on. So, just create a base class, from which A and B themselves will inherit, and move a properly modified version the thing you are putting in __new__ there. If you are not using Python 3.6, this should be done on the __new__ or __init__ of a metaclass, not on the __new__ of the class itself.
Another approach would be to have a custom __getattribute__ method - this could be crafted to provide namespaces for the base classes. It would owrk ony on instances, not on the classes themselves (but could be made to, again, using a metaclass). __getattribute__ can even hide the same-name-methods.
class Base:
#classmethod
def _get_base_modules(cls):
result = {}
for base in cls.__bases__:
module_name = cls.__module__.split(".")[0]
result[module_name] = base
return result
#classmethod
def _proxy(self, module_name):
class base:
def __dir__(base_self):
return dir(self._base_modules[module_name])
def __getattr__(base_self, attr):
original_value = self._base_modules[module_name].__dict__[attr]
if hasattr(original_value, "__get__"):
original_value = original_value.__get__(self, self.__class__)
return original_value
base.__name__ = module_name
return base()
def __init_subclass__(cls):
cls._base_modules = cls._get_base_modules()
cls._shadowed = {name for module_class in cls._base_modules.values() for name in module_class.__dict__ if not name.startswith("_")}
def __getattribute__(self, attr):
if attr.startswith("_"):
return super().__getattribute__(attr)
cls = self.__class__
if attr in cls._shadowed:
raise AttributeError(attr)
if attr in cls._base_modules:
return cls._proxy(attr)
return super().__getattribute__(attr)
def __dir__(self):
return super().dir() + list(self._base_modules)
class A(Base):
...
class B(Base):
...
class C(A, B):
...
As you can see - this is some fun, but starts getting really complicated - and all the hoola-boops that are needed to retrieve the actual attributes from the superclasses after ading an artificial namespace seem to indicate your problem is not calling for using inheritance after all, as I suggested above.
Since you have your small, functional, atomic classes for each "service" , you could use a plain, simple, non-meta-at-all class that would work as a registry for the various services - and you can even enhance it to call the equivalent method in several of the services it is handling with a single call:
class Services:
def __init__(self):
self.registry = {}
def register(self, cls, key, secret):
name = cls.__module__.split(".")[0]
service= cls(key, secret)
self.registry[name] = service
def __getattr__(self, attr):
if attr in self.registry:
return self.registry[attr]

python: how to access class.__dict__ from class variable?

I need to define a class variable, named "class". I want to do this directly in class namespace, not in a class method. Obviously, I cannot directly say:
class MyClass(object):
a = 1
b = 2
class = 3
So, I want to do something like:
class MyClass(object):
a = 1
b = 2
self.__dict__["class"] = 3
Where "self" should be replaced with a reference to the class. So, how do I refer to a class from class namespace?
NOTE: This question might seem contrived, but it stems from a practical goal.
In fact, MyClass is a Django REST Framework serializer and I need a "class" field to be defined on it, because this REST endpoint has to follow a certain protocol.
There's a metaclass defined for Serializers, which calls __new__() upon class creation and that __new__() aggregates all the fields, defined on class and populates a registry of fields with them. So, I have to define my variable class before the class is created. Also see: Django REST Framework: how to make verbose name of field differ from its field_name?

You could do:
class MyClass(object):
a = 1
b = 2
vars()['class'] = 3
But since class is a reserved keyword, then you have to access the variable using getattr and setattr, so that class remains a string.
>>> m = MyClass()
>>> getattr(m, 'class')
3

You can create your class from type and add the attribute class to the class dictionary:
>>> MyClass = type('MyClass', (), {'class': 3, 'a':1, 'b':2})
>>> getattr(MyClass, 'class')
3
You can't directly access the name class with a dot reference, you'll need to use getattr:
>>> MyClass.class
File "<stdin>", line 1
MyClass.class
^
SyntaxError: invalid syntax
FWIW, you can define the class methods like you would do conventionally and then bind them to the class later on.
Caveat: While this works, I wouldn't use this hack myself as the keyword class is too much of a keyword to tamper with.

You don't need to name the attribute class, which can lead to all kinds of problems. You can name the attribute class_, but still have it pull from a source attribute named class and render out to JSON as class.
You can do this by overriding the metaclass for Serializers. Here is an example of a serializers.py file (the models and classes are largely pulled straight from the tutorial).
The main magic is this section of the metaclass
# Remap fields (to use class instead of class_)
fields_ = []
for name, field in fields:
if name.endswith('_'):
name = name.rstrip('_')
fields_.append((name, field))
This takes any field you define in the serializer that ends in an underscore (ie. field_) and removes the underscore from the name when it binds the Fields and sets the _declared_fields attribute on the serializer.
from collections import OrderedDict
from rest_framework import serializers
from rest_framework.fields import Field
from snippets.models import Snippet, LANGUAGE_CHOICES, STYLE_CHOICES
class MyMeta(serializers.SerializerMetaclass):
#classmethod
def _get_declared_fields(cls, bases, attrs):
fields = [(field_name, attrs.pop(field_name))
for field_name, obj in list(attrs.items())
if isinstance(obj, Field)]
fields.sort(key=lambda x: x[1]._creation_counter)
# If this class is subclassing another Serializer, add that Serializer's
# fields. Note that we loop over the bases in *reverse*. This is necessary
# in order to maintain the correct order of fields.
for base in reversed(bases):
if hasattr(base, '_declared_fields'):
fields = list(base._declared_fields.items()) + fields
# Remap fields (to use class instead of class_)
fields_ = []
for name, field in fields:
if name.endswith('_'):
name = name.rstrip('_')
fields_.append((name, field))
return OrderedDict(fields_)
class SnippetSerializer(serializers.Serializer):
__metaclass__ = MyMeta
pk = serializers.IntegerField(read_only=True)
title = serializers.CharField(required=False, allow_blank=True, max_length=100)
class_ = serializers.CharField(source='klass', label='class', default='blah')
def create(self, validated_data):
"""
Create and return a new `Snippet` instance, given the validated data.
"""
return Snippet.objects.create(**validated_data)
def update(self, instance, validated_data):
"""
Update and return an existing `Snippet` instance, given the validated data.
"""
instance.title = validated_data.get('title', instance.title)
instance.class_ = validated_data.get('class', instance.class_)
instance.save()
return instance
Here is the models.py file for reference (django doesn't allow field names to end in an underscore)
from django.db import models
class Snippet(models.Model):
title = models.CharField(max_length=100, blank=True, default='')
klass = models.CharField(max_length=100, default='yo')
This is how it looks from the django shell
$ python manage.py shell
>>> from snippets.models import Snippet
>>> from snippets.serializers import SnippetSerializer
>>> from rest_framework.renderers import JSONRenderer
>>> from rest_framework.parsers import JSONParser
>>> snippet = Snippet(title='test')
>>> snippet.save()
>>> serializer = SnippetSerializer(snippet)
>>> serializer.data
{'title': u'test', 'pk': 6, 'class': u'yo'}

You cannot it while creating class - technically that object does not exist yet.
You could consider:
class MyClass(object):
a = 1
b = 2
# class is already created
MyClass.__dict__["class"] = 3
But MyClass.__dict__ is not a dict, but a dictproxy, and 'dictproxy' object does not support item assignment, so there would be TypeError raised.

Use '''setattr''' to set a class attribute immediately after you finish the class definition. Outside the definition, of course. Pass '''MyClass''' for parameter, and it will create an attribute of your class.
Dict of members should not be used, especially for modifying an object. In fact, it is rarely needed. Most of things (though not all) people usually intend it to do are better done by '''setattr''' and '''getattr'''.
Finally, as one of those who answered noticed, you do not really need a field named '''class''', but that's another story, different from your original question.

Why don't you have to initialize properties in Django models?

I'm starting off with Django, and I'm sort of confused about how the models work. I've searched for a bit and can't find an answer. When creating classes in Python, we have to initialize the object properties, for example:
class Contact(Object):
def __init__(self, name, number):
self.name = name
self.number = number
And if we create a subclass, for example, coworker:
Class Coworker(Contact):
def __init__(self, name, number, title):
Contact.__init__(self, name, number)
self.title = title
So it makes sense that we still initialize the properties from the superclass, but in Django, why don't we do any initialization? We inherit from the models.Model class:
class Poll(models.Model):
question = models.CharField(max_length=200)
Why don't we have to initialize CharField from Model before we use it? I hope I'm not being too cryptic with my question. Like I said, I'm just getting started with Django, so any help is appreciated.

Because they're stored in the database. Either the program or the manager will instantiate the model with the correct arguments, therefore it is inappropriate for the initializer to blatantly override them.

def __init___(self, **kwargs):
cls_ = type(self)
for k in kwargs:
if not hasattr(cls_, k):
raise TypeError(
"%r is an invalid keyword argument for %s" %
(k, cls_.__name__))
setattr(self, k, kwargs[k])
There should be an universal __init__ method in models.Model like this (This is a snippet from SQLAlchemy but I believe that Django uses some similar methods)

Two Python classes identical except for superclass - how to avoid duplication?

I have some functionality wrapped up in a Python class (classa). classa inherits from another class supera.
I want exactly the same functionality as classa except that I want to inherit from superb.
I could just copy the class classa to a new class classb and then change the superclass for classb but obviously this very tacky, a maintenance headache and and I'm quite sure there's much better way - can anyone tell me what it is ?
EDIT: Thanks for answers so far. I should have said initially the my classa invokes super in its methods in order to invoke supera methods. It seems that this has some significance when looking at mixins as an option

This can be done with Python's multiple inheritance if none of the methods need to invoke super().
class Dog(object):
name = "Spot"
class Cat(object):
name = "Whiskers"
class SpeakingAnimalMixin(object):
def speak(self):
print "My name is", self.name, "and I can speak!"
class SpeakingDog(SpeakingAnimalMixin, Dog):
pass
class SpeakingCat(SpeakingAnimalMixin, Cat):
pass
SpeakingDog().speak()
My name is Spot and I can speak!
If you do need to invoke super() from a method then you need to create the class dynamically. This works fine, but the generated class's name will be less helpful and IDEs and other static analysis tools may be less useful.
You can create the class using a function, passing the superclass as an argument:
def make_speaking_animal_class(SpeechlessAnimal):
class SpeakingAnimal(SpeechlessAnimal):
def get_name(self):
return "Speaking " + super(SpeakingAnimal, self).get_name()
def speak(self):
print "My name is", self.get_name()
return SpeakingAnimal
class Dog(object):
def get_name(self):
return "Spot"
class Cat(object):
def get_name(self):
return "Whiskers"
SpeakingDog = make_speaking_animal_class(Dog)
SpeakingCat = make_speaking_animal_class(Cat)
SpeakingCat().speak()
My name is Speaking Whiskers
However as mentioned, the class's __name__ attribute may not be what you expect.
print SpeakingDog
print SpeakingDog()
<class '__main__.SpeakingAnimal'>
<__main__.SpeakingAnimal object at 0x1004a3b50>
You can fix this by assigning them unique __name__ attributes yourself:
SpeakingDog.__name__ = 'SpeakingDog'
print SpeakingDog
<class '__main__.SpeakingDog'>
(Credit to Andrew Jaffe for suggesting this in an answer, but he deleted it.)
There's another way to create a class dynamically, but I discourage you from using it unless you need to; it's even less clear. The type function has a second use, apart from its main one of determining the class of an object: it can be used to dynamically create a new class.
When used this way, the type function takes three parameters:
name, the __name__ the new class will have.
bases, a tuple of of base classes that the new class will inherit from.
dict, a dictionary containing the methods and attributes the new class will have.
You could use it like this:
def make_speaking_animal_class(SpeechlessAnimal, name):
def get_name(self):
return "Speaking " + super(SpeakingAnimal, self).get_name()
def speak(self):
print "My name is", self.get_name()
bases = (SpeechlessAnimal,)
# We need to define SpeakingAnimal in a variable so that get_name can refer
# to it for the super() call, otherwise we could just return it directly.
SpeakingAnimal = type(name, bases, {
'get_name': get_name,
'speak': speak
})
return SpeakingAnimal
class Dog(object):
def get_name(self):
return "Spot"
class Cat(object):
def get_name(self):
return "Whiskers"
SpeakingDog = make_speaking_animal_class(Dog, 'SpeakingDog')
SpeakingCat = make_speaking_animal_class(Cat, 'SpeakingCat')
SpeakingDog().speak()
SpeakingCat().speak()
My name is Speaking Spot
My name is Speaking Whiskers