Develop Reference

Generic is an abstract class in python?

I'm trying to create a base class that works for any CRUD in applications and I've seen the following implementation:
ModelType = TypeVar("ModelType", bound=Base)
CreateSchemaType = TypeVar("CreateSchemaType", bound=BaseModel)
UpdateSchemaType = TypeVar("UpdateSchemaType", bound=BaseModel)
class CRUDBase(Generic[ModelType, CreateSchemaType, UpdateSchemaType]):
def __init__(self, model: Type[ModelType]):
"""CRUD object with default methods to Create, Read, Update,
Delete (CRUD).
**Parameters**
* `model`: A SQLAlchemy model class
* `schema`: A Pydantic model (schema) class
"""
self.model = model
'...crud methods'`
Generic is a way to define an abstract class, containing the objects that are specified (in this case [ModelType,CreateSchemaType, UpdateSchemaType]) or what is the use of generic?

If your question is about the purpose of typing.Generic I would suggest you read through PEP 484. It has a section dedicated to user defined generic classes with some examples specifically for this, but the entire document is worthwhile reading IMHO. If you are unsure about the entire concept of generic types, the Wikipedia articles on parametric polymorphism and generic programming might even be worth skimming.
In very simple terms, you can use the Generic base class to define your own classes in a generic way, i.e. parameterized with regards to one or more type arguments. These type parameters are constructed via typing.TypeVar. It is worth noting that in most cases these things are only relevant for the purposes of type safety, which is not really enforced by Python itself. Instead, static type checkers (most IDEs have them built-in) deal with those things.
As to your example, it is entirely possible to define your class in a generic way and thus improve your own coding experience because you'll likely get useful auto-suggestions and warnings by your IDE. This only really becomes useful, if you use the type variables somewhere else throughout your class. Example:
from typing import Generic, TypeVar
class Base:
pass
ModelType = TypeVar("ModelType", bound=Base)
class CRUDBase(Generic[ModelType]):
def __init__(self, model: type[ModelType]):
self.model = model
def get_model_instance(self) -> ModelType:
return self.model()
class Sub(Base):
def foo(self) -> None:
print("hi mom")
if __name__ == "__main__":
crud_a = CRUDBase(Sub)
crud_b = CRUDBase(Base)
a = crud_a.get_model_instance()
b = crud_b.get_model_instance()
a.foo()
b.foo() # error
A good type checker like mypy will know, that a is of the Sub type and thus has the foo method, whereas b is of the Base type and therefore does not. And it will know so just from the way we annotated CRUDBase.
PyCharm for example struggles with this (don't know why). Fortunately, we can help it out with explicit type annotations because we defined CRUDBase as a generic type:
...
crud_a: CRUDBase[Sub] = CRUDBase(Sub)
crud_b: CRUDBase[Base] = CRUDBase(Base)
a = crud_a.get_model_instance()
b = crud_b.get_model_instance()
a.foo()
b.foo() # PyCharm marks `foo` and complains
Notice that so far there is nothing particularly "abstract" about our CRUDBase. It is fully self-contained and functioning like this (albeit not very useful yet).
If you want a class to be an abstract base and not be directly instantiated, the abc module provides you with the tools you need. You can define an abstract base class and abstract methods on it that all subclasses must implement. But I think it is clear now that this is a different concept. The idea is that an abstract class is not intended to be instantiated, only its subclasses are.
I hope this helps point you in a few useful directions.

Using ABC, PolymorphicModel, django-models gives metaclass conflict

So far every other answer on SO answers in the exact same way: construct your metaclasses and then inherit the 'joined' version of those metaclasses, i.e.
class M_A(type): pass
class M_B(type): pass
class A(metaclass=M_A): pass
class B(metaclass=M_B): pass
class M_C(M_A, M_B): pass
class C:(A, B, metaclass=M_C): pass
But I don't know what world these people are living in, where they're constructing your own metaclasses! Obviously, one would be using classes from other libraries and unless you have a perfect handle on meta programming, how are you supposed to know whether you can just override a class's metaclass? (Clearly I do not have a handle on them yet).
My problem is:
class InterfaceToTransactions(ABC):
def account(self):
return None
...
class Category(PolymorphicModel, InterfaceToTransactions):
def account(self):
return self.source_account
...
class Income(TimeStampedModel, InterfaceToTransactions):
def account(self):
return self.destination_account
...
Which of course gives me the error: "metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases"
I've tried many variations of the solution given above, the following does not work, gives the same error.
class InterfaceToTransactionsIntermediaryMeta(type(PolymorphicModel), type(InterfaceToTransactions)):
pass
class Category(PolymorphicModel, InterfaceToTransactions):
__metaclass__ = InterfaceToTransactionsIntermediaryMeta
...
Nor does putting anything inside the class Meta function. I've read every single other SO question on this topic, please don't simply mark it as duplicate.
-------------------Edited 1/8/18 after accepting the solution-------
Oddly enough, if I try to makemigrations with this new configuration (the one I accepted), it starts giving the metaclass error again, but it still works during runtime. If I comment out the metaclass parts then makemigrations and migrate, it will do it successfully, but then I have to put it back in there after migrating every time.

If you are using Python 3, you are trying to use your derived metaclass incorrectly.
And since you get "the same error", and not other possible, more subtle, error, I'd say this is what is happening.
Try just changing to:
class IntermediaryMeta(type(InterfaceToTransactions), type(PolymorphicModel)):
pass
class Category(PolymorphicModel, InterfaceToTransactions, metaclass=IntermediaryMeta):
...
(At least the ABCMeta class is guaranteed to work collaboratively using super, that is enough motive to place the classe it first on the bases )
tuple)
If that yields you new and improved errors, this means that one or both of those classes can't really collaborate properly due to one of several motives. Then, the way to go is to force your inheritance tree that depends on ABCMeta not to do so, since its role is almost aesthetical in a language where everything else is for "consenting adults" like Python.
Unfortunatelly, the way to that is to use varying methods of brute-force, from safe "rewritting everything" to monkey patching ABCMeta and abstractmethod on the place were "InterfaceToTransactions" is defined to simply do nothing.
If you need to get there, and need some help, please post another question.
Sorry - this is actually the major drawbacks of using metaclasses.

Unless django-polymorphic decides to inherit from abc.ABC this is going to be very difficult to achieve. A good solution would be to "manually" create your interface. For instance:
class InterfaceToTransactions:
def account(self):
raise NotImplementedError("Account method must be implemented.")
...
class Category(PolymorphicModel, InterfaceToTransactions):
def account(self):
return self.source_account
...
class Income(TimeStampedModel, InterfaceToTransactions):
def account(self):
return self.destination_account
...

How to tell if a class is abstract in Python 3?

I wrote a metaclass that automatically registers its classes in a dict at runtime. In order for it to work properly, it must be able to ignore abstract classes.
The code works really well in Python 2, but I've run into a wall trying to make it compatible with Python 3.
Here's what the code looks like currently:
def AutoRegister(registry, base_type=ABCMeta):
class _metaclass(base_type):
def __init__(self, what, bases=None, attrs=None):
super(_metaclass, self).__init__(what, bases, attrs)
# Do not register abstract classes.
# Note that we do not use `inspect.isabstract` here, as
# that only detects classes with unimplemented abstract
# methods - which is a valid approach, but not what we
# want here.
# :see: http://stackoverflow.com/a/14410942/
metaclass = attrs.get('__metaclass__')
if not (metaclass and issubclass(metaclass, ABCMeta)):
registry.register(self)
return _metaclass
Usage in Python 2 looks like this:
# Abstract classes; these are not registered.
class BaseWidget(object): __metaclass__ = AutoRegister(widget_registry)
class BaseGizmo(BaseWidget): __metaclass__ = ABCMeta
# Concrete classes; these get registered.
class AlphaWidget(BaseWidget): pass
class BravoGizmo(BaseGizmo): pass
What I can't figure out, though, is how to make this work in Python 3.
How can a metaclass determine if it is initializing an abstract class in Python 3?

PEP3119 describes how the ABCMeta metaclass "marks" abstract methods and creates an __abstractmethods__ frozenset that contains all methods of a class that are still abstract. So, to check if a class cls is abstract, check if cls.__abstractmethods__ is empty or not.
I also found this relevant post on abstract classes useful.

I couldn't shake the feeling as I was posting this question that I was dealing with an XY Problem. As it turns out, that's exactly what was going on.
The real issue here is that the AutoRegister metaclass, as implemented, relies on a flawed understanding of what an abstract class is. Python or not, one of the most important criteria of an abstract class is that it is not instanciable.
In the example posted in the question, BaseWidget and BaseGizmo are instanciable, so they are not abstract.
Aren't we just bifurcating rabbits here?
Well, why was I having so much trouble getting AutoRegister to work in Python 3? Because I was trying to build something whose behavior contradicts the way classes work in Python.
The fact that inspect.isabstract wasn't returning the result I wanted should have been a major red flag: AutoRegister is a warranty-voider.
So what's the real solution then?
First, we have to recognize that BaseWidget and BaseGizmo have no reason to exist. They do not provide enough functionality to be instantiable, nor do they declare abstract methods that describe the functionality that they are missing.
One could argue that they could be used to "categorize" their sub-classes, but a) that's clearly not what's going on in this case, and b) quack.
Instead, we could embrace Python's definition of "abstract":
Modify BaseWidget and BaseGizmo so that they define one or more abstract methods.
If we can't come up with any abstract methods, then can we remove them entirely?
If we can't remove them but also can't make them properly abstract, it might be worthwhile to take a step back and see if there are other ways we might solve this problem.
Modify the definition of AutoRegister so that it uses inspect.isabstract to decide if a class is abstract: see final implementation.
That's cool and all, but what if I can't change the base classes?
Or, if you have to maintain backwards compatibility with existing code (as was the case for me), a decorator is probably easier:
#widget_registry.register
class AlphaWidget(object):
pass
#widget_registry.register
class BravoGizmo(object):
pass

Get PyCharm to know what classes are mixin for

Our application has set of complex form wizards. To avoid code duplication I created several mixins.
The problem is that PyCharm highlights mixin methods with Unresolved attribute refference error.
This is correct as object does not have such methods. But I know that this mixin will be used only with special classes. Is there any way to tell this info to PyCharm?
For now I use such approach:
class MyMixin(object):
def get_context_data(self, **kwargs):
assert isinstance(self, (ClassToBeExtended, MyMixin))
# super.get_context_data is still highlighter,
# as super is considered as object
context = super(MyMixin, self).get_context_data(**kwargs)
context.update(self.get_preview_context())
return context
def get_preview_context(self):
# without this line PyCharm highlights the self.initial_data
assert isinstance(self, (ClassToBeExtended, MyMixin))
return {'needs': (self.initial_data['needs']
if 'type' not in self.initial_data
else '%(needs)s %(type)s' % self.initial_data)}
While this works for some cases like autocomplete for self., it fails for other cases like super. Is there a better approach to achieve the desired behavior?
P.S.: I know that I can disable reference check for specific name or whole class, but I don't want to do this as it will not help in typo checks and autocomplete.

You can type-hint to PyCharm what kind of classes to expect.
class DictMixin(object):
def megamethod(
self, # type: dict
key
):
return self.get(key)
It's still not quite comparable to other type handling.
PyCharm is lazy in evaluating it, and only does so when first working on self.
Things are a bit tricky when accessing attributes of the mixin as well - self, # type: dict | DictMixin works for one of my classes, but not in my test code.
In python 3.5, you should be able to use # type: typing.Union[dict, DictMixin].

If you are creating Mixin, for, let's say ClassSub, which is subclass of ClassSuper, you can implement Mixins this way:
class Mixin1(ClassSuper):
pass
class Mixin2(ClassSuper):
pass
and then use them like:
class ClassSub(Mixin1, Mixin2):
pass
That way I use some mixins for models in Django. Also, django-extensions uses similar pattern (gives models that are actually mixins). Basically, this way you don't have to inherit ClassSuper, because it's "included" in every of your mixins.
Most important - PyCharm works like a charm this way.

Dynamic sub-classing in Python

I have a number of atomic classes (Components/Mixins, not really sure what to call them) in a library I'm developing, which are meant to be subclassed by applications. This atomicity was created so that applications can only use the features that they need, and combine the components through multiple inheritance.
However, sometimes this atomicity cannot be ensured because some component may depend on another one. For example, imagine I have a component that gives a graphical representation to an object, and another component which uses this graphical representation to perform some collision checking. The first is purely atomic, however the latter requires that the current object already subclassed this graphical representation component, so that its methods are available to it. This is a problem, because we have to somehow tell the users of this library, that in order to use a certain Component, they also have to subclass this other one. We could make this collision component sub class the visual component, but if the user also subclasses this visual component, it wouldn't work because the class is not on the same level (unlike a simple diamond relationship, which is desired), and would give the cryptic meta class errors which are hard to understand for the programmer.
Therefore, I would like to know if there is any cool way, through maybe metaclass redefinition or using class decorators, to mark these unatomic components, and when they are subclassed, the additional dependency would be injected into the current object, if its not yet available. Example:
class AtomicComponent(object):
pass
#depends(AtomicComponent) # <- something like this?
class UnAtomicComponent(object):
pass
class UserClass(UnAtomicComponent): #automatically includes AtomicComponent
pass
class UserClass2(AtomicComponent, UnAtomicComponent): #also works without problem
pass
Can someone give me an hint on how I can do this? or if it is even possible...
edit:
Since it is debatable that the meta class solution is the best one, I'll leave this unaccepted for 2 days.
Other solutions might be to improve error messages, for example, doing something like UserClass2 would give an error saying that UnAtomicComponent already extends this component. This however creates the problem that it is impossible to use two UnAtomicComponents, given that they would subclass object on different levels.

"Metaclasses"
This is what they are for! At time of class creation, the class parameters run through the
metaclass code, where you can check the bases and change then, for example.
This runs without error - though it does not preserve the order of needed classes
marked with the "depends" decorator:
class AutoSubclass(type):
def __new__(metacls, name, bases, dct):
new_bases = set()
for base in bases:
if hasattr(base, "_depends"):
for dependence in base._depends:
if not dependence in bases:
new_bases.add(dependence)
bases = bases + tuple(new_bases)
return type.__new__(metacls, name, bases, dct)
__metaclass__ = AutoSubclass
def depends(*args):
def decorator(cls):
cls._depends = args
return cls
return decorator
class AtomicComponent:
pass
#depends(AtomicComponent) # <- something like this?
class UnAtomicComponent:
pass
class UserClass(UnAtomicComponent): #automatically includes AtomicComponent
pass
class UserClass2(AtomicComponent, UnAtomicComponent): #also works without problem
pass
(I removed inheritance from "object", as I declared a global __metaclass__ variable. All classs will still be new style class and have this metaclass. Inheriting from object or another class does override the global __metaclass__variable, and a class level __metclass__ will have to be declared)
-- edit --
Without metaclasses, the way to go is to have your classes to properly inherit from their dependencies. Tehy will no longer be that "atomic", but, since they could not work being that atomic, it may be no matter.
In the example bellow, classes C and D would be your User classes:
>>> class A(object): pass
...
>>> class B(A, object): pass
...
>>>
>>> class C(B): pass
...
>>> class D(B,A): pass
...
>>>