from typing_extensions import Protocol
class IFoo(Protocol):
value: int
class Foo(IFoo):
#property
def value(self) -> int:
return 2
_value: int
#value.setter
def value(self, value: int):
self._value = value
Pylance in strict mode(basic mode doesn't) is giving an error at the getter and the setter saying that:
"value" overrides symbol of the same name in class "IFoo"
"property" is incompatible with "int".
I could make this work by changing the Protocol to:
class IFoo(Protocol):
#property
def value(self) -> int:
raise NotImplemented
But this now makes this invalid:
class Foo(IFoo):
value: int
This doesn't makes sense, the Foo would still have the property value that is an int, why being a getter should makes it different (in typescript this doesn't make a difference)?
How can I fix this?
Reading the Defining a protocol section of the relevant pep (PEP 544), the example implementation (in their case, class Resource) does not directly inherit from the protocol - their class SupportsClose functions as a reference type for type hinting validators.
Your example is also reminiscent of the long established zope.interface package, which this PEP also referenced. Note that the example usage the PEP have cited the following example (irrelevant lines trimmed):
from zope.interface import Interface, implementer
class IEmployee(Interface):
...
#implementer(IEmployee)
class Employee:
...
The Employee class does not directly subclass from IEmployee (a common mistake for newbie Zope/Plone developers back in the days), it's simply decorated with the zope.interface.implementer(IEmployee) class decorator to denote that the class Employee implements from the interface IEmployee.
Likewise, reading further down under the section Protocol members, we have an example of the template and concrete classes (again, example trimmed):
from typing import Protocol
class Template(Protocol):
name: str # This is a protocol member
value: int = 0 # This one too (with default)
class Concrete:
def __init__(self, name: str, value: int) -> None:
self.name = name
self.value = value
var: Template = Concrete('value', 42) # OK
Again, note that the Concrete implementation does not inherit from the Template, yet, the variable var is denoted to have an expected type of Template. Note that the Concrete instance can be assigned to it as it matches the expected protocol as defined for Template.
So all that being said, given your example, you may wish to define class Foo: as is rather than having it inherit from IFoo as you had originally, and fill in the type information such that things that expect IFoo be type hinted as appropriately in the relevant context (e.g. some_foo: IFoo = Foo(...) or def some_func(foo: IFoo):).
As an addendum, you may wish to define Foo as such:
class Foo:
_value: int
#property
def value(self) -> int:
return 2
#value.setter
def value(self, value: int):
self._value = value
Having the _value definition in between the property and its setter seems to confuse mypy due to this issue.
Related
I have a set of unrelated classes (some imported) which all have a common attribute (or property) a of type dict[str, Any].
Within a there should be another dict under the key "b", which I would like to expose on any of these classes as an attribute b to simplify inst.a.get("b", {})[some_key] to inst.b[some_key].
I have made the following subclass factory to work as a class decorator for local classes and a function for imported classes.
But so far I'm failing to type hint its cls argument and return value correctly.
from functools import wraps
def access_b(cls):
#wraps(cls, updated=())
class Wrapper(cls):
#property
def b(self) -> dict[str, bool]:
return self.a.get("b", {})
return Wrapper
MRE of my latest typing attemp (with mypy 0.971 errors):
from functools import wraps
from typing import Any, Protocol, TypeVar
class AProtocol(Protocol):
a: dict[str, Any]
class BProtocol(AProtocol, Protocol):
b: dict[str, bool]
T_a = TypeVar("T_a", bound=AProtocol)
T_b = TypeVar("T_b", bound=BProtocol)
def access_b(cls: type[T_a]) -> type[T_b]:
#wraps(cls, updated=())
class Wrapper(cls): # Variable "cls" is not valid as a type & Invalid base class "cls"
#property
def b(self) -> dict[str, bool]:
return self.a.get("b", {})
return Wrapper
#access_b
class Demo1:
"""Local class."""
def __init__(self, a: dict[str, Any]):
self.a = a.copy()
demo1 = Demo1({"b": {"allow_X": True}})
demo1.b["allow_X"] # "Demo1" has no attribute "b"
class Demo2:
"""Consider me an imported class."""
def __init__(self, a: dict[str, Any]):
self.a = a.copy()
demo2 = access_b(Demo2)({"b": {"allow_X": True}}) # Cannot instantiate type "Type[<nothing>]"
demo2.b["allow_X"]
I do not understand why cls is not valid as a type, even after reading https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases.
I understand I should probably not return a Protocol (I suspect that is the source of Type[<nothing>]), but I don't see how I could specify "returns the original type with an extension".
PS1. I have also tried with a decorator which adds b dynamically, still failed to type it...
PS2. ...and with a decorator which uses a mixin as per #DaniilFajnberg's answer, still failing.
References:
functools.wraps(cls, update=()) from https://stackoverflow.com/a/65470430/17676984
(Type) Variables as base classes?
This is actually a really interesting question and I am curious about what solutions other people come up with.
I read up a little on these two errors:
Variable "cls" is not valid as a type / Invalid base class "cls"
There seems to be an issue here with mypy that has been open for a long time now. There doesn't seem to be a workaround yet.
The problem, as I understand it, is that no matter how you annotate it, the function argument cls will always be a type variable and that is considered invalid as a base class. The reasoning is apparently that there is no way to make sure that the value of that variable isn't overwritten somewhere.
I honestly don't understand the intricacies well enough, but it is really strange to me that mypy seems to treat a class A defined via class A: ... different than a variable of Type[A] since the former should essentially just be syntactic sugar for this:
A = type('A', (object,), {})
There was also a related discussion in the mypy issue tracker. Again, hoping someone can shine some light onto this.
Adding a convenience property
In any case, from your example I gather that you are not dealing with foreign classes, but that you define them yourself. If that is the case, a Mix-in would be the simplest solution:
from typing import Any, Protocol
class AProtocol(Protocol):
a: dict[str, Any]
class MixinAccessB:
#property
def b(self: AProtocol) -> dict[str, bool]:
return self.a.get("b", {})
class SomeBase:
...
class OwnClass(MixinAccessB, SomeBase):
def __init__(self, a: dict[str, Any]):
self.a = a.copy()
demo1 = OwnClass({"b": {"allow_X": True}})
print(demo1.b["allow_X"])
Output: True
No mypy issues in --strict mode.
Mixin with a foreign class
If you are dealing with foreign classes, you could still use the Mix-in and then use functools.update_wrapper like this:
from functools import update_wrapper
from typing import Any, Protocol
class AProtocol(Protocol):
a: dict[str, Any]
class MixinAccessB:
"""My mixin"""
#property
def b(self: AProtocol) -> dict[str, bool]:
return self.a.get("b", {})
class Foreign:
"""Foreign class documentation"""
def __init__(self, a: dict[str, Any]):
self.a = a.copy()
class MixedForeign(MixinAccessB, Foreign):
"""foo"""
pass
update_wrapper(MixedForeign, Foreign, updated=())
demo2 = MixedForeign({"b": {"allow_X": True}})
print(demo2.b["allow_X"])
print(f'{MixedForeign.__name__=} {MixedForeign.__doc__=}')
Output:
True
MixedForeign.__name__='Foreign' MixedForeign.__doc__='Foreign class documentation'
Also no mypy issues in --strict mode.
Note that you still need the AProtocol to make it clear that whatever self will be in that property follows that protocol, i.e. has an attribute a with the type dict[str, Any].
I hope I understood your requirements correctly and this at least provides a solution for your particular situation, even though I could not enlighten you on the type variable issue.
I would like to have a base class for all my descriptors which is of type descriptor:
Is it correct to use the GetSetDescriptorType?
class MyDescriptorBase(GetSetDescriptorType):
pass
Now when I declare
class MyDescriptor(DescriptorBase):
def __get__(self, __obj: MyObj, objtype: Type[MyObj] ):
pass
A pycharm inspection complains:
get() does not match signature.
I have looked up the signature which is:
# types.pyi
def __get__(self, __obj: Any, __type: type = ...) -> Any: ...
Is it an error in inspection or is my declaration wrong?
No, using GetSetDescriptor class as base is invalid for type checking since this class is final.
You don't really need this base, because descriptors have good support now.
The problem in your definition is missing default value of objtype: parent can be called with one argument and your child - no, so this violates LSP. So the following is compatible:
class MyDescriptor(MyDescriptorBase):
def __get__(self, obj: MyObj, objtype: type[MyObj] | None = None):
pass
I love the typechecker in Pylance (VS Code), but there seems to be a situation that I must choose between ignoring a Pylance warning and best practice for class variable declaration.
Many times, class variables are initialized using a None type in the class constructor, and the variable is set later. For example:
class Person:
def __init__(self) -> None:
self.name:str = None
def setName(self, name:str) -> None:
self.name = name
In this case, Pylance gives the following error on the assignment self.name:str = None:
Cannot assign member "name" for type "Person"
Expression of type "None" cannot be assigned to member "name" of class "Person"
Type "None" cannot be assigned to type "str"
Is there any way to declare self.name in the constructor in such a way that a value is not needed and Pylance is happy?
EDIT: Several have suggested the use of typing.Optional to suppress this Pylance warning. However, if another member function is created to return self.name and guarantee that it is returning an instance of str, another Pylance error is generated. See the following example:
class Person:
def __init__(self) -> None:
self._name:Optional[str] = None
#property
def name(self) -> str:
return self._name
#name.setter
def name(self, name:str) -> None:
self._name = name
In this case, Pylance generates the error:
(variable) _name: str | None
Expression of type "str | None" cannot be assigned to return type "str"
Type "str | None" cannot be assigned to type "str"
Type "None" cannot be assigned to type "str"
Ideally, there would be some way to initially "allocate" self._name in the constructor in such a way that its only allowed type is str but is not given any value. Is this possible?
You need to make it optional via typing.Optional:
#!/usr/bin/env python
from typing import Optional
class Person:
def __init__(self) -> None:
self.name: Optional[str] = None
def setName(self, name: str) -> None:
self.name = name
Python 3.10+ supports a simpler syntax via PEP 604's Union type expressions1, e.g.:
def __init__(self) -> None:
self.name: str | None = None
update: This is so obvious I assume it won't work in your case, but it is the "natural thing to do"
If you arte annotating instance attributes, and don't want them to be able to read "None" or other sentinel value, simply do not fill in a sentinel value, just declare the attribute and its annotation. That is, do not try to fill in a value in __init__, declare the annotation in the class body instead:
class Person:
name: str
# No need for this:
# def __init__(self) -> None:
# self.name:str = None
def setName(self, name:str) -> None:
self.name = name
Assuming you need to initialize it with an invalid value for whatever reason, the original answer applies:
Well, if this was certainly a case were the programmer knows better than the type-checker, I think it would be the case for using typing.cast: this is a no-operation in runtime, but which tells the static checkers that the value being cast is actually of that type, regardless of previous declarations:
(to be clear: don't do this:)
import typing as t
class Person:
def __init__(self) -> None:
self._name: t.Optional[str, None] = None
#property
def name(self) -> str:
return t.cast(str, self._name)
...
However, upon writing this, I came to the realisation that "self._name" might actually be "None" at this time. The typechecker can do some state-checking by following the program steps, but not in such a generic way - trying to read instance.name before it was set should cause in Python a runtime error instead.
So, if the method is called to express exactly that, it works as the tools can follow parts of the code guarded by isinstance (and if one needs a more complicated check than isinstance, take a look on the docs for typing.TypeGuard) - and we thank the tools for ensuring the program will run without a hard to debug error afterwards:
(this is the correct way)
import typing as t
class Person:
def __init__(self) -> None:
self._name: t.Optional[str, None] = None
#property
def name(self) -> str:
if isinstance(name:=self._name, str):
return name
raise AttributeError()
#name.setter
def name(self, name:str) -> None:
self._name = name
So I have a simple wrapper class for making read-only attributes of a class.
class ReadOnly:
"""
Simple wrapper class to make an object read only
"""
def __init__(self, value) -> None:
self._value = value
def __get__(self, instance, owner):
return self._value
def __set__(self, value) -> None:
raise AttributeError("This value is read-only")
I use this elsewhere in my code to create read-only parameters. For example,
class Something:
def __init__(self) -> None:
self.attr = ReadOnly("A read-only string")
a = Something()
# Not allowed
a.attr = "This won't work"
The problem with this is that when I access a.attr, its type is unknown, meaning my editor can't get autocompletions or type information, which could make it confusing for users of my code.
I understand that this is much easier to do using properties, but due to the nature of my program, doing that would result in hundreds of extra lines of repeated code.
As such, is there a way of getting the type of the value in the parameters of ReadOnly.__init__(), so that I can explicitly mark the return value of ReadOnly.__get__() as the same type?
I have tried using a TypeVar from the typing module, and marking the parameter type with it and then letting it infer from there, but the type just comes out as T#__init__.
The other alternative would be to annotate exact types in subclasses of the ReadOnly type, for example
class ReadOnlyStr(ReadOnly):
def __init__(self, value: str) -> None:
super().__init__(value)
def __get__(self, instance, owner) -> str:
return super().__get__(instance, owner)
but this feels sorta janky, and would also quickly lead to repeated code if I had to do so with many types.
As such, what is the best way for me to tell Python the type of a variable given its type in a different function.
If it helps, I'm using VS Code with Pylance as my language server.
Using Generics & TypeVar would be your best bet
from typing import TypeVar, Generic
PropType = TypeVar("PropType")
class ReadOnly(Generic[PropType]):
"""
Simple wrapper class to make an object read only
"""
def __init__(self, value: PropType):
self._value = value
def __get__(self, instance, owner) -> PropType:
return self._value
def __set__(self, value):
raise AttributeError("This value is read-only")
Given a class with a helper method for initialization:
class TrivialClass:
def __init__(self, str_arg: str):
self.string_attribute = str_arg
#classmethod
def from_int(cls, int_arg: int) -> ?:
str_arg = str(int_arg)
return cls(str_arg)
Is it possible to annotate the return type of the from_int method?
I'v tried both cls and TrivialClass but PyCharm flags them as unresolved references which sounds reasonable at that point in time.
Starting with Python 3.11 you can use the new typing.Self object. For older Python versions you can get the same object by using the typing-extensions project:
try:
from typing import Self
except ImportError:
from typing_extensions import Self
class TrivialClass:
# ...
#classmethod
def from_int(cls, int_arg: int) -> Self:
# ...
return cls(...)
Note that you don't need to annotate cls in this case.
Warning: mypy support for the Self type has not yet been released; you'll need to wait for the next version after 0.991. Pyright already supports it.
If you can't wait for Mypy support, then you can use a generic type to indicate that you'll be returning an instance of cls:
from typing import Type, TypeVar
T = TypeVar('T', bound='TrivialClass')
class TrivialClass:
# ...
#classmethod
def from_int(cls: Type[T], int_arg: int) -> T:
# ...
return cls(...)
Any subclass overriding the class method but then returning an instance of a parent class (TrivialClass or a subclass that is still an ancestor) would be detected as an error, because the factory method is defined as returning an instance of the type of cls.
The bound argument specifies that T has to be a (subclass of) TrivialClass; because the class doesn't yet exist when you define the generic, you need to use a forward reference (a string with the name).
See the Annotating instance and class methods section of PEP 484.
Note: The first revision of this answer advocated using a forward reference
naming the class itself as the return value, but issue 1212 made it possible to use generics instead, a better solution.
As of Python 3.7, you can avoid having to use forward references in annotations when you start your module with from __future__ import annotations, but creating a TypeVar() object at module level is not an annotation. This is still true even in Python 3.10, which defers all type hint resolution in annotations.
From Python 3.7 you can use __future__.annotations:
from __future__ import annotations
class TrivialClass:
# ...
#classmethod
def from_int(cls, int_arg: int) -> TrivialClass:
# ...
return cls(...)
Edit: you can't subclass TrivialClass without overriding the classmethod, but if you don't require this then I think it's neater than a forward reference.
A simple way to annotate the return type is to use a string as the annotation for the return value of the class method:
# test.py
class TrivialClass:
def __init__(self, str_arg: str) -> None:
self.string_attribute = str_arg
#classmethod
def from_int(cls, int_arg: int) -> 'TrivialClass':
str_arg = str(int_arg)
return cls(str_arg)
This passes mypy 0.560 and no errors from python:
$ mypy test.py --disallow-untyped-defs --disallow-untyped-calls
$ python test.py
In Python 3.11 there is a nicer way to do this using the new Self type:
from typing import Self
class TrivialClass:
def __init__(self, str_arg: str):
self.string_attribute = str_arg
#classmethod
def from_int(cls, int_arg: int) -> Self:
str_arg = str(int_arg)
return cls(str_arg)
This also works correctly with sub classes as well.
class TrivialSubClass(TrivialClasss):
...
TrivialSubclass.from_int(42)
The IDE shows return type TrivialSubClass and not TrivialClass.
This is described in PEP 673.