Develop Reference

Type Hinting: Use type of a class member as function return type (for inheritance)

What is the correct way to reuse the type of a class member to type hint other items in the class? As an example:
from typing import Type
class Model:
pass
class ChildModel:
childvar = "Child Model"
class Base:
var: Type[Model]
def fn(self) -> ??:
return self.var
class Child(Base):
var = ChildModel
def new_fn(self):
x = self.fn() # Type of x should be "ChildModel"
print(x.childvar)
Child().new_fn() # Prints "Child Model" successfully
I am looking for what would work to replace ?? such that the return type of fn() can be inferred for all child classes.
MyPy does not accept changing ?? to Type[Model] to match Base.var: Incompatible types in assignment (expression has type "Type[ChildModel]", base class "Base" defined the type as "Type[Model]" (though it is possible I made a mistake here). Even if this were allowed, this would allow Base.fn() to return any Model or Model subclass, not strictly the type of var (as defined in a child of Base)
Something like T = TypeVar("T", bound=Type[Model]) seems disallowed without generics, which don't seem quite applicable since the type can be inferred without generic-style specification. I think the solution would likely also work to type hint method arguments, method-local variables, and other class member variables.
What is the best way to do this (if possible)?
Edit: adding clarification, corrected issue with code

This can be accomplished with Generics.
from typing import Generic, TypeVar
T = TypeVar("T", bound="Model")
class Model:
pass
class ChildModel(Model):
childvar = "Child Model"
class Base(Generic[T]):
var: type[T]
def fn(self) -> type[T]:
return self.var
class Child(Base[ChildModel]):
var = ChildModel
def new_fn(self):
x = self.fn() # Type of x is type["ChildModel"]
print(x.childvar)
Child().new_fn()

Though this probably fails the "Explicit is better than Implicit" test, I suppose this will get you what you want while avoiding typing in two places. In this case, rather than defining var on the Child, the var is pulled from the annotation.
Tested on Python 3.10
import typing
from typing import Generic, TypeVar
T = TypeVar("T", bound="Model")
class Model:
pass
class ChildModel(Model):
childvar = "Child Model"
class Base(Generic[T]):
#classmethod
#property
def var(cls) -> type[T]:
for superclass in cls.__orig_bases__:
if getattr(superclass, "__origin__", None) == Base:
return typing.get_args(superclass)[0]
def fn(self) -> type[T]:
return self.var
class Child(Base[ChildModel]):
def new_fn(self):
x = self.fn() # Type of x is type["ChildModel"]
print(x.childvar)

Python init object of generic type

Coming from a C# background and knowing its generic type approaches I'm now trying to implement something similar in Python. I need to serialize and de-serialize classes in a special string format, so I created the following two base classes, the first for single entity serialization and the second one for list serialization of that entity type.
from typing import Any, TypeVar, List, cast, Type, Generic, NewType
import re
T = TypeVar('T')
class Serializable(Generic[T]):
def to_str(self) -> str:
raise NotImplementedError
#classmethod
def from_str(cls, str: str):
raise NotImplementedError
class SerializableList(List[Serializable[T]]):
def __init__(self):
self.separator: str = "\n"
#classmethod
def from_str(cls, str: str):
list = cls()
for match in re.finditer(list.separator, str):
list.append(T().from_str(match)) # <-- PROBLEM: HOW TO INIT A GENERIC ENTITY ???
# list.append(Serializable[T].from_str(match)) <-- Uses base class (NotImplemented) instead of derived class
return list
def to_str(self) -> str:
str = ""
for e in self:
str = str + f"{e.to_str()}{self.separator}"
return str
Then I can derive from those classes and have to implement to_str and from_str. Please see the marker <-- PROBLEM". I have no idea how I can init a new entity of the currently used type for the list. How do we do this in the Python way?

As #user2357112supportsMonica says in the comments, typing.Generic is pretty much only there for static analysis, and has essentially no effect at runtime under nearly all circumstances. From the look of your code, it looks like what you're doing might be better suited to Abstract Base Classes (documentation here, tutorial here), which can be easily combined with Generic.
A class that has ABCMeta as its metaclass is marked as an Abstract Base Class (ABC). A subclass of an ABC cannot be instantiated unless all methods in the ABC marked with the #abstractmethod decorator have been overridden. In my suggested code below, I've explicitly added the ABCMeta metaclass to your Serializable class, and implicitly added it to your SerializableList class by having it inherit from collections.UserList instead of typing.List. (collections.UserList already has ABCMeta as its metaclass.)
Using ABCs, you could define some interfaces like this (you won't be able to instantiate these because of the abstract methods):
### ABSTRACT INTERFACES ###
from abc import ABCMeta, abstractmethod
from typing import Any, TypeVar, Type, Generic
from collections import UserList
import re
T = TypeVar('T')
class AbstractSerializable(metaclass=ABCMeta):
#abstractmethod
def to_str(self) -> str: ...
#classmethod
#abstractmethod
def from_str(cls: Type[T], string: str) -> T: ...
S = TypeVar('S', bound=AbstractSerializable)
class AbstractSerializableList(UserList[S]):
separator = '\n'
#classmethod
#property
#abstractmethod
def element_cls(cls) -> Type[S]: ...
#classmethod
def from_str(cls, string: str):
new_list = cls()
for match in re.finditer(cls.separator, string):
new_list.append(cls.element_cls.from_str(match))
return new_list
def to_str(self) -> str:
return self.separator.join(e.to_str() for e in self)
You could then provide some concrete implementations like this:
class ConcreteSerializable(AbstractSerializable):
def to_str(self) -> str:
# put your actual implementation here
#classmethod
def from_str(cls: Type[T], string: str) -> T:
# put your actual implementation here
class ConcreteSerializableList(AbstractSerializableList[ConcreteSerializable]:
# this overrides the abstract classmethod-property in the base class
element_cls = ConcreteSerializable
(By the way — I changed several of your variable names — str, list, etc — as they were shadowing builtin types and/or functions. This can often lead to annoying bugs, and even if it doesn't, is quite confusing for other people reading your code! I also cleaned up your to_str method, which can be simplified to a one-liner, and moved your separator variable to be a class variable, since it appears to be the same for all class instances and does not appear to ever be altered.)

For now I found a dirty solution - this is to add a Type (constructor) parameter of the list entries like so:
class SerializableList(List[Serializable[T]]):
# This one
# |
# v
def __init__(self, separator: str = "\n", entity_class: Type = None):
self.separator = separator
self.entity_class = entity_class
#classmethod
def from_str(cls, str: str):
list = cls()
for match in re.finditer(list.separator, str):
list.append(list.entity_class.from_str(match))
return list
I wonder if there is a cleaner way to get the correct [T] type constructor from List[T] since it is already provided there?

Change return type of child class

I have a base class (BaseClass) that operates using objects of a base type (BaseType). I want a child class (ChildClass) that inherits from BaseClass but operates using objects of a child type (ChildType).
Is there a way to accomplish this without type hinting errors?
Consider the following code:
from dataclasses import dataclass
#dataclass
class BaseType:
name: str
#dataclass
class ChildType(BaseType):
favoriteColor: str
class BaseClass:
def __init__(self, myThing: BaseType):
self.myThing = myThing
def get_my_thing(self) -> BaseType:
return self.myThing
class ChildClass(BaseClass):
def __init__(self, myThing: ChildType):
self.myThing = myThing
self.do_something_super_complicated()
def do_something_super_complicated():
color = self.myThing.favoriteColor
# do complicated things with color
ChildClass.get_my_thing() now has a return type of ChildType. How can I mark that this is the case? For instance, I don't want to get a type hint error by doing this:
cc = ChildClass(ChildType(name="bob", favoriteColor="red"))
print(cc.get_my_thing().favoriteColor)
# This causes a type hint error because return type of get_my_thing is BaseType which does not have favoriteColor
If I do the following, I still get a type hint error:
class ChildClass(BaseClass):
def __init__(self, myThing: ChildType):
self.myThing = myThing
def get_my_thing(self) -> ChildType:
return super().get_my_thing()
#The return type of super().get_my_thing() is BaseType so this causes a type hint error

There is value in that type hint error - a value of BaseType isn't necessarily a value of ChildType. Concretely, if BaseType is Car and ChildType is Tesla, super().get_my_thing() in ChildClass returns some Car, but you are trying to treat it specifically like a Tesla.
From your example, it seems that you want to model some class that takes in a generic type (that must be a subclass of BaseType) and has a method get_my_thing that has the generic type as its return type. You can consider to define a single class instead, as follow:
from dataclasses import dataclass
from typing import Generic, TypeVar
#dataclass
class BaseType:
name: str
#dataclass
class ChildType(BaseType):
favoriteColor: str
T = TypeVar('T', bound=BaseType)
class MyClass(Generic[T]):
def __init__(self, myThing: T):
self.myThing = myThing
def get_my_thing(self) -> T:
return self.myThing
class BaseClass(MyClass[BaseType]):
pass
class ChildClass(MyClass[ChildType]):
pass
base_type = BaseType("Bob")
base_instance = BaseClass(base_type)
base_instance.get_my_thing().name # OK
child_type = ChildType("Bob", "blue")
child_instance = ChildClass(child_type)
child_instance.get_my_thing().name # OK
child_instance.get_my_thing().favoriteColor # OK
The type hinting works on my editor (VS Code, using the Pylance language server for checking type hints).

Tell methods of base class to use more restricted type hints in a derived class?

Scenario
Lets say I have a generic Store class that implements various methods to retrieve StoreObjects. And to fill the store it defines an abstract method load_object.
Then I create a CarStore. I derive from the Store and overwrite the load_object method to return Car objects.
Now the question is how to add type hints for this. First the code:
from typing import Dict
import weakref
import abc
class StoreObject:
pass
class Car(StoreObject):
def __init__(self, color: str):
self.color = color # type: str
class Store(abc.ABC):
def __init__(self):
self._cache = weakref.WeakValueDictionary() # type: weakref.WeakValueDictionary[int, StoreObject]
def get(self, index: int) -> StoreObject:
try:
return self._cache[index]
except KeyError:
obj = self.load_object(index)
self._cache[index] = obj
return obj
#abc.abstractmethod
def load_object(self, index: int) -> StoreObject:
raise NotImplementedError
class CarStore(Store):
def load_object(self, index: int) -> Car:
if index < 100:
return Car("red")
else:
return Car("blue")
store = CarStore()
car = store.get(10)
print("Your car color is", car.color)
Type Checking Errors
The problem is in the following line:
print("Your car color is", car.color)
Here PyCharm gives the following warning:
Unresolved attribute reference 'color' for class 'StoreObject'
Mypy gives the following error:
development/storetyping.py:39: error: "StoreObject" has no attribute "color"
Also the PyCharm code completion does obviously not include the name method for store.get(10).?.
Question
How can I type the base class such that PyCharm and mypy can successfully check this code?
Is there a way to parameterize the types in Store such that when creating CarStore I can tell it to use Car instead of StoreObject in the annotations?

In more static languages you would create Store as a generic class and use Car as a type parameter when inheriting from Store.
We can actually do that using the typing module in python.
Here is a minimal example:
from typing import Generic, TypeVar
T = TypeVar('T') # this is the generic placeholder for a type
# Store is a generic class with type parameter T
class Store(Generic[T]):
def get(self) -> T: # this returns a T
return self.load_object()
def load_object(self) -> T: # this also returns a T
raise NotImplementedError
class Car:
def __init__(self, color):
self.color = color
# Now we inherit from the Store and use Car as the type parameter
class CarStore(Store[Car]):
def load_object(self):
return Car('red')
s = CarStore()
c = s.get()
print(c.color) # Code completion works and no warnings are shown
Edit:
To address ShadowRanger's note: If you want Car and all the products to have a common base class you can use the bound parameter of the TypeVar. Thank you juanpa.arrivillaga for the hint.
So we create a Product class and bind the TypeVar to it.
class Product:
def get_id(self):
raise NotImplementedError
T = TypeVar('T', bound=Product)
Mypy will now complain about this:
class CarStore(Store[Car]):
def load_object(self):
return Car('red')
because a Car is not a Product. So let' change that, too:
class Car(Product):
def get_id(self):
return ...
def __init__(self, color):
self.color = color
And now, mypy is happy.
Edit2:
Here is the full code with some more annotations, that make even mypy --strict happy.
from typing import Generic, TypeVar
class Product:
def get_id(self) -> int:
raise NotImplementedError
T = TypeVar('T', bound=Product)
class Store(Generic[T]):
def get(self) -> T:
return self.load_object()
def load_object(self) -> T:
raise NotImplementedError
class Car(Product):
def get_id(self) -> int:
return hash(self.color)
def __init__(self, color: str):
self.color = color
class CarStore(Store[Car]):
def load_object(self) -> Car:
return Car('red')
if __name__ == '__main__':
s = CarStore()
c = s.get()
print(c.color)

Your type checking is behaving correctly; get is not overridden in CarStore, so the annotation on it continues to specify that it returns StoreObject. If you want to change the annotation, you'd have to redefine get in CarStore, e.g. by adding:
def get(self, index: int) -> Car:
return typing.cast(Car, super().get(index))
Make sure to import typing to gain access to cast (or use unqualified cast and add it to your from typing import Dict import).
To avoid runtime performance overhead, you could only conditionally define get based on an if typing.TYPE_CHECKING: test (which returns True when static checkers are analyzing the code, and False when running it), so the get overload isn't actually defined at runtime.

Subclass in type hinting

I want to allow type hinting using Python 3 to accept sub classes of a certain class. E.g.:
class A:
pass
class B(A):
pass
class C(A):
pass
def process_any_subclass_type_of_A(cls: A):
if cls == B:
# do something
elif cls == C:
# do something else
Now when typing the following code:
process_any_subclass_type_of_A(B)
I get an PyCharm IDE hint 'Expected type A, got Type[B] instead.'
How can I change type hinting here to accept any subtypes of A?
According to PEP 484 ("Expressions whose type is a subtype of a specific argument type are also accepted for that argument."), I understand that my solution (cls: A) should work?

When you specify cls: A, you're saying that cls expects an instance of type A. The type hint to specify cls as a class object for the type A (or its subtypes) uses typing.Type.
from typing import Type
def process_any_subclass_type_of_A(cls: Type[A]):
pass
From The type of class objects
:
Sometimes you want to talk about class objects that inherit from a
given class. This can be spelled as Type[C] where C is a class. In
other words, when C is the name of a class, using C to annotate an
argument declares that the argument is an instance of C (or of a
subclass of C), but using Type[C] as an argument annotation declares
that the argument is a class object deriving from C (or C itself).

If we look at the Type description from the typing module, then we see these docs:
A special construct usable to annotate class objects.
For example, suppose we have the following classes::
class User: ... # Abstract base for User classes
class BasicUser(User): ...
class ProUser(User): ...
class TeamUser(User): ...
And a function that takes a class argument that's a subclass of
User and returns an instance of the corresponding class::
U = TypeVar('U', bound=User)
def new_user(user_class: Type[U]) -> U:
user = user_class()
# (Here we could write the user object to a database)
return user
joe = new_user(BasicUser)
At this point the type checker knows that joe has type BasicUser.
Based on this, I can imagine a synthetic example that reproduces the problem with type hinting errors in PyCharm.
from typing import Type, Tuple
class BaseClass: ...
class SubClass(BaseClass): ...
class SubSubClass(SubClass): ...
def process(model_instance: BaseClass, model_class: Type[BaseClass]) -> Tuple[BaseClass, BaseClass]:
""" Accepts all of the above classes """
return model_instance, model_class()
class ProcessorA:
#staticmethod
def proc() -> Tuple[SubClass, SubClass]:
""" PyCharm will show an error
`Expected type 'tuple[SubClass, SubClass]', got 'tuple[BaseClass, BaseClass]' instead` """
return process(SubClass(), SubClass)
class ProcessorB:
#staticmethod
def proc() -> Tuple[SubSubClass, SubSubClass]:
""" PyCharm will show an error
`Expected type 'tuple[SubSubClass, SubSubClass]', got 'tuple[BaseClass, BaseClass]' instead` """
return process(SubSubClass(), SubSubClass)
But we see in docs for Type that the situation can be corrected by using TypeVar with the bound argument. Then use it in places where BaseClass is declared as a type.
from typing import TypeVar, Type, Tuple
class BaseClass: ...
B = TypeVar('B', bound=BaseClass)
class SubClass(BaseClass): ...
class SubSubClass(SubClass): ...
def process(model_instance: B, model_class: Type[B]) -> Tuple[B, B]:
""" Accepts all of the above classes """
return model_instance, model_class()
class ProcessorA:
#staticmethod
def proc() -> Tuple[SubClass, SubClass]:
return process(SubClass(), SubClass)
class ProcessorB:
#staticmethod
def proc() -> Tuple[SubSubClass, SubSubClass]:
return process(SubSubClass(), SubSubClass)
Hope this will be helpful.

Type[A] accepts also the class itself, which is not always needed.
If you want your function to accept only subclasses, you should go with NewType, like
class A:
pass
B = NewType('B', A)
def foo(cls: Type[B]):
...