I'm using class decorators in Python and cannot figure out which type annotation to give to my class to make mypy happy.
My code is the following:
from typing import Type
from pprint import pformat
def betterrepr(cls:Type[object]):
"""Improve representation of a class"""
class improved_class(cls): # line 12
def __repr__(self) -> str:
return f"Instance of {cls.__name__}, vars = {pformat(vars(self))}"
return improved_class
I'm currently having the 2 following errors:
myprog.py:12: error: Invalid type "cls"
myprog.py:12: error: Invalid base class
What shall I use for the type of cls (and by the way, is it Pythonic to use this keyword for a class used as argument?)?
Thanks
Using function arguments as base classes is currently not supported by mypy. Your only option is to silence the error, either with a type: ignore comment or a dummy alias like base: Any = cls.
Even without annotating cls, mypy will correctly infer the type of a class decorated with betterrepr. To document that your decorator returns a class similar to the decorated class, use a TypeVar.
from typing import Type, TypeVar
from pprint import pformat
T = TypeVar('T')
def betterrepr(cls: Type[T]) -> Type[T]:
"""Improve representation of a class"""
class IClass(cls): # type: ignore
def __repr__(self) -> str:
return f"Instance of {cls.__name__}, vars = {pformat(vars(self))}"
return IClass
Related
I have a set of unrelated classes (some imported) which all have a common attribute (or property) a of type dict[str, Any].
Within a there should be another dict under the key "b", which I would like to expose on any of these classes as an attribute b to simplify inst.a.get("b", {})[some_key] to inst.b[some_key].
I have made the following subclass factory to work as a class decorator for local classes and a function for imported classes.
But so far I'm failing to type hint its cls argument and return value correctly.
from functools import wraps
def access_b(cls):
#wraps(cls, updated=())
class Wrapper(cls):
#property
def b(self) -> dict[str, bool]:
return self.a.get("b", {})
return Wrapper
MRE of my latest typing attemp (with mypy 0.971 errors):
from functools import wraps
from typing import Any, Protocol, TypeVar
class AProtocol(Protocol):
a: dict[str, Any]
class BProtocol(AProtocol, Protocol):
b: dict[str, bool]
T_a = TypeVar("T_a", bound=AProtocol)
T_b = TypeVar("T_b", bound=BProtocol)
def access_b(cls: type[T_a]) -> type[T_b]:
#wraps(cls, updated=())
class Wrapper(cls): # Variable "cls" is not valid as a type & Invalid base class "cls"
#property
def b(self) -> dict[str, bool]:
return self.a.get("b", {})
return Wrapper
#access_b
class Demo1:
"""Local class."""
def __init__(self, a: dict[str, Any]):
self.a = a.copy()
demo1 = Demo1({"b": {"allow_X": True}})
demo1.b["allow_X"] # "Demo1" has no attribute "b"
class Demo2:
"""Consider me an imported class."""
def __init__(self, a: dict[str, Any]):
self.a = a.copy()
demo2 = access_b(Demo2)({"b": {"allow_X": True}}) # Cannot instantiate type "Type[<nothing>]"
demo2.b["allow_X"]
I do not understand why cls is not valid as a type, even after reading https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases.
I understand I should probably not return a Protocol (I suspect that is the source of Type[<nothing>]), but I don't see how I could specify "returns the original type with an extension".
PS1. I have also tried with a decorator which adds b dynamically, still failed to type it...
PS2. ...and with a decorator which uses a mixin as per #DaniilFajnberg's answer, still failing.
References:
functools.wraps(cls, update=()) from https://stackoverflow.com/a/65470430/17676984
(Type) Variables as base classes?
This is actually a really interesting question and I am curious about what solutions other people come up with.
I read up a little on these two errors:
Variable "cls" is not valid as a type / Invalid base class "cls"
There seems to be an issue here with mypy that has been open for a long time now. There doesn't seem to be a workaround yet.
The problem, as I understand it, is that no matter how you annotate it, the function argument cls will always be a type variable and that is considered invalid as a base class. The reasoning is apparently that there is no way to make sure that the value of that variable isn't overwritten somewhere.
I honestly don't understand the intricacies well enough, but it is really strange to me that mypy seems to treat a class A defined via class A: ... different than a variable of Type[A] since the former should essentially just be syntactic sugar for this:
A = type('A', (object,), {})
There was also a related discussion in the mypy issue tracker. Again, hoping someone can shine some light onto this.
Adding a convenience property
In any case, from your example I gather that you are not dealing with foreign classes, but that you define them yourself. If that is the case, a Mix-in would be the simplest solution:
from typing import Any, Protocol
class AProtocol(Protocol):
a: dict[str, Any]
class MixinAccessB:
#property
def b(self: AProtocol) -> dict[str, bool]:
return self.a.get("b", {})
class SomeBase:
...
class OwnClass(MixinAccessB, SomeBase):
def __init__(self, a: dict[str, Any]):
self.a = a.copy()
demo1 = OwnClass({"b": {"allow_X": True}})
print(demo1.b["allow_X"])
Output: True
No mypy issues in --strict mode.
Mixin with a foreign class
If you are dealing with foreign classes, you could still use the Mix-in and then use functools.update_wrapper like this:
from functools import update_wrapper
from typing import Any, Protocol
class AProtocol(Protocol):
a: dict[str, Any]
class MixinAccessB:
"""My mixin"""
#property
def b(self: AProtocol) -> dict[str, bool]:
return self.a.get("b", {})
class Foreign:
"""Foreign class documentation"""
def __init__(self, a: dict[str, Any]):
self.a = a.copy()
class MixedForeign(MixinAccessB, Foreign):
"""foo"""
pass
update_wrapper(MixedForeign, Foreign, updated=())
demo2 = MixedForeign({"b": {"allow_X": True}})
print(demo2.b["allow_X"])
print(f'{MixedForeign.__name__=} {MixedForeign.__doc__=}')
Output:
True
MixedForeign.__name__='Foreign' MixedForeign.__doc__='Foreign class documentation'
Also no mypy issues in --strict mode.
Note that you still need the AProtocol to make it clear that whatever self will be in that property follows that protocol, i.e. has an attribute a with the type dict[str, Any].
I hope I understood your requirements correctly and this at least provides a solution for your particular situation, even though I could not enlighten you on the type variable issue.
I'm trying to define a class that takes another class as an attribute _model and will instantiate objects of that class.
from abc import ABC
from typing import Generic, TypeVar, Any, ClassVar, Type
Item = TypeVar("Item", bound=Any)
class SomeClass(Generic[Item], ABC):
_model: ClassVar[Type[Item]]
def _compose_item(self, **attrs: Any) -> Item:
return self._model(**attrs)
I think it should be obvious that self._model(**attrs) returns an instance of Item, since _model is explicitly declared as Type[Item] and attrs is declared as Dict[str, Any].
But what I'm getting from mypy 0.910 is:
test.py: note: In member "_compose_item" of class "SomeClass":
test.py:11: error: Returning Any from function declared to return "Item"
return self._model(**attrs)
^
What am I doing wrong?
MyPy can sometimes be a bit funny about the types of classes. You can solve this by specifying _model as Callable[..., Item] (which, after all, isn't a lie) instead of Type[Item]:
from abc import ABC
from typing import Generic, TypeVar, Any, ClassVar, Callable
Item = TypeVar("Item")
class SomeClass(Generic[Item], ABC):
_model: ClassVar[Callable[..., Item]]
def _compose_item(self, **attrs: Any) -> Item:
return self._model(**attrs)
Is there any way to type an abstract parent class method such that the child class method is known to return itself, instead of the abstract parent.
class Parent(ABC):
#abstractmethod
def method(self) -> [what to hint here]:
pass
class Child1(Parent)
def method(self):
pass
def other_method(self):
pass
class GrandChild1(Child1)
def other_method_2(self):
pass
This is more to improve autocompletes for IDEs like PyCharm or VScode's python plugin.
So, the general approach is described in the docs here
import typing
from abc import ABC, abstractmethod
T = typing.TypeVar('T', bound='Parent') # use string
class Parent(ABC):
#abstractmethod
def method(self: T) -> T:
...
class Child1(Parent):
def method(self: T) -> T:
return self
def other_method(self):
pass
class GrandChild1(Child1):
def other_method_2(self):
pass
reveal_type(Child1().method())
reveal_type(GrandChild1().method())
And mypy gives us:
test_typing.py:22: note: Revealed type is 'test_typing.Child1*'
test_typing.py:23: note: Revealed type is 'test_typing.GrandChild1*'
Note, I had to keep using type-variables to get this to work, so when I originally tried to use the actual child class in the child class annotation, it (erroneously?) inherited the type in the grandchild:
class Child1(Parent):
def method(self) -> Child1:
return self
I'd get with mypy:
test_typing.py:22: note: Revealed type is 'test_typing.Child1'
test_typing.py:23: note: Revealed type is 'test_typing.Child1'
Again, I am not sure if this is expected/correct behavior. The mypy documentation currently has a warning:
This feature is experimental. Checking code with type annotations for
self arguments is still not fully implemented. Mypy may disallow valid
code or allow unsafe code.
Python 3.11 introduced more elegant solution based on PEP-0673, predefined type Self (official docs). Example:
from typing import Self
class Parent(ABC):
#abstractmethod
def method(self) -> Self:
pass
class Child1(Parent)
def method(self) -> Self:
pass
def other_method(self) -> Self:
pass
class GrandChild1(Child1)
def other_method_2(self) -> Self:
pass
It covers classmethods too:
class Shape:
#classmethod
def from_config(cls, config: dict[str, float]) -> Self:
return cls(config["scale"])
NOTE: for pre-python-3.11 one can use:
1 - Quoted type, e.g.
class Parent:
def method(self) -> "Parent":
pass
2 - or postoponed type hint evaluation (PEP-0563, or other SO answer, python 3.7+):
from __future__ import annotations
class Parent:
def method(self) -> Parent:
pass
Given a class with a helper method for initialization:
class TrivialClass:
def __init__(self, str_arg: str):
self.string_attribute = str_arg
#classmethod
def from_int(cls, int_arg: int) -> ?:
str_arg = str(int_arg)
return cls(str_arg)
Is it possible to annotate the return type of the from_int method?
I'v tried both cls and TrivialClass but PyCharm flags them as unresolved references which sounds reasonable at that point in time.
Starting with Python 3.11 you can use the new typing.Self object. For older Python versions you can get the same object by using the typing-extensions project:
try:
from typing import Self
except ImportError:
from typing_extensions import Self
class TrivialClass:
# ...
#classmethod
def from_int(cls, int_arg: int) -> Self:
# ...
return cls(...)
Note that you don't need to annotate cls in this case.
Warning: mypy support for the Self type has not yet been released; you'll need to wait for the next version after 0.991. Pyright already supports it.
If you can't wait for Mypy support, then you can use a generic type to indicate that you'll be returning an instance of cls:
from typing import Type, TypeVar
T = TypeVar('T', bound='TrivialClass')
class TrivialClass:
# ...
#classmethod
def from_int(cls: Type[T], int_arg: int) -> T:
# ...
return cls(...)
Any subclass overriding the class method but then returning an instance of a parent class (TrivialClass or a subclass that is still an ancestor) would be detected as an error, because the factory method is defined as returning an instance of the type of cls.
The bound argument specifies that T has to be a (subclass of) TrivialClass; because the class doesn't yet exist when you define the generic, you need to use a forward reference (a string with the name).
See the Annotating instance and class methods section of PEP 484.
Note: The first revision of this answer advocated using a forward reference
naming the class itself as the return value, but issue 1212 made it possible to use generics instead, a better solution.
As of Python 3.7, you can avoid having to use forward references in annotations when you start your module with from __future__ import annotations, but creating a TypeVar() object at module level is not an annotation. This is still true even in Python 3.10, which defers all type hint resolution in annotations.
From Python 3.7 you can use __future__.annotations:
from __future__ import annotations
class TrivialClass:
# ...
#classmethod
def from_int(cls, int_arg: int) -> TrivialClass:
# ...
return cls(...)
Edit: you can't subclass TrivialClass without overriding the classmethod, but if you don't require this then I think it's neater than a forward reference.
A simple way to annotate the return type is to use a string as the annotation for the return value of the class method:
# test.py
class TrivialClass:
def __init__(self, str_arg: str) -> None:
self.string_attribute = str_arg
#classmethod
def from_int(cls, int_arg: int) -> 'TrivialClass':
str_arg = str(int_arg)
return cls(str_arg)
This passes mypy 0.560 and no errors from python:
$ mypy test.py --disallow-untyped-defs --disallow-untyped-calls
$ python test.py
In Python 3.11 there is a nicer way to do this using the new Self type:
from typing import Self
class TrivialClass:
def __init__(self, str_arg: str):
self.string_attribute = str_arg
#classmethod
def from_int(cls, int_arg: int) -> Self:
str_arg = str(int_arg)
return cls(str_arg)
This also works correctly with sub classes as well.
class TrivialSubClass(TrivialClasss):
...
TrivialSubclass.from_int(42)
The IDE shows return type TrivialSubClass and not TrivialClass.
This is described in PEP 673.
Introduction
With Python/MyPy type hints, one can use .pyi stubs to keep annotations in separate files to implementations. I am using this functionality to give basic hinting of SQLAlchemy's ORM (more specifically, the flask_sqlalchemy plugin).
Models are defined like:
class MyModel(db.Model):
id = db.Column()
...
...
where db.Model is included directly from SQLAlchemy.
They can be queried, for example, by:
MyModel.query.filter({options: options}).one_or_none()
where filter() returns another Query, and one_or_none() returns an instance of MyModel (or None, obviously).
The follwing .pyi file successfully hints the above construct, though it is incomplete - there is no way to hint the return type of one_or_none().
class _SQLAlchemy(sqlalchemy.orm.session.Session):
class Model:
query = ... # type: _Query
class _Query(sqlalchemy.orm.query.Query):
def filter(self, *args) -> query.Query: ...
def one_or_none(self) -> Any: ...
db = ... # type: _SQLAlchemy
The Question
How can one fully and generically hint the above, and hint the return type of one_or_none()?
My first attempt was to use generics, but it looks like I have no access to the the subtype in question (in the given example, MyModel). To illustrate a nonworking approach:
from typing import Generic, TypeVar
_T = TypeVar('_T')
class _SQLAlchemy(sqlalchemy.orm.session.Session):
class Model:
def __init__(self, *args, **kwargs):
self.query = ... # type: _Query[self.__class__]
class _Query(Generic[_T], sqlalchemy.orm.query.Query):
def filter(self, *args) -> _Query[_T]: ...
def one_or_none(self) -> _T: ...
db = ... # type: _SQLAlchemy
Is there any way to get this working?
Apologies for the specific and example, but I tried for a while to write this concisely with a generic example and it was never as clear as it is currently (which is possibly still not much!)
Edit
Another non-working approach (I'm aware this would have the limitation of having to call myModelInstance.query... instead of the static MyModel.query, but even this does not work):
from typing import Generic, TypeVar
_T = TypeVar('_T')
class _SQLAlchemy(sqlalchemy.orm.session.Session):
class Model:
#property
def query(self: _T) -> _Query[_T]: ...
class _Query(Generic[_T], sqlalchemy.orm.query.Query):
def filter(self, *args) -> _Query[_T]: ...
def one_or_none(self) -> _T: ...
db = ... # type: _SQLAlchemy
Type annotation stubs are fortunately now available for that specific issue https://github.com/dropbox/sqlalchemy-stubs
Exact implementation for Query type annotation is available here: https://github.com/dropbox/sqlalchemy-stubs/blob/master/sqlalchemy-stubs/orm/query.pyi (archive)