I have come across an issue during set-up python classes. I have such classes:
class ModifierType(ABC):
"""Represents a generic modifier type."""
...
class DefaultModifierType(ModifierType):
"""Represents a default modifier type."""
#staticmethod
def check_modifier(modifier_id: str, item_name: str, default_modifiers: tuple[str]) -> None:
...
#dataclass
class RequiredModifierType(ModifierType):
"""Represents a required modifier type."""
default_quantity: int = None
action_codes: list[str] = field(default_factory=list)
def check_modifier(self, modifier_id: str, item_name: str) -> None:
...
#dataclass
class Modifier:
"""Represents a generic modifier of group."""
modifier_id: str
modifier_type: ModifierType
And now I also have the outer function that runs kind of such code:
if isinstance(modifier.modifier_type, RequiredModifierType):
modifier.modifier_type.check_modifier(
modifier_id=modifier.modifier_id,
item_name=item_name
)
elif isinstance(modifier.modifier_type, DefaultModifierType):
modifier.modifier_type.check_modifier(
modifier_id=modifier.modifier_id,
item_name=item_name,
default_modifiers=default_modifiers
)
The issue: as I figured out I cannot create an abstract method for ModifierType class because it has different params in DefaultModifierType and RequiredModifierType respectively. So I'd like to know if there's any opportunity to create this abstract method? If not, the checks better to move into Modifier class and check the instance of modifier_type there?
You could add some further arguments, give them a default value, and ignore them where they are irrelevant or don't make sense.
class ModifierType():
#staticmethod
def check_modifier(modifier_id: str, item_name: str, default_modifiers: tuple[str] = None) -> None:
...
class DefaultModifierType(ModifierType):
"""Represents a default modifier type."""
#staticmethod
def check_modifier(modifier_id: str, item_name: str, default_modifiers: tuple[str]) -> None:
assert (default_modifiers is not None)
...
However, if they really do have quite different signatures, it is worth considering that the two methods are not conceptually the same, and should be different methods that the calling side needs to explicitly distinguish.
Related
I have classes like this:
class Transaction(ABC):
self.uuid: str
class CashTransaction(Transaction):
...
class SecurityTransaction(Transaction):
...
Instances of the classes above are stored in a class called RecordKeeper, which has a method for getting a list of the instances. The method accepts either of the three classes defined above and checks that all the instances are of this specified type.
The following implementation works with mypy, but not when type_ is Transaction (the ABC):
class RecordKeeper:
self._transactions: list[Transaction]
TransactionType = TypeVar("TransactionType", Transaction, CashTransaction, SecurityTransaction)
def get_transactions(
self,
uuids: Collection[str],
type_: type[TransactionType],
) -> list[TransactionType]:
"""Returns a list of Transactions matching uuuids, ensuring they are all of type 'type_'."""
# correctly inferred list[CashTransaction]
cash_transactions = self.get_transactions(uuids, CashTransaction)
# correctly inferred list[SecurityTransaction]
security_transactions = self.get_transactions(uuids, SecurityTransaction)
# VS Code type checker correctly infers any_transactions: list[Transaction], but mypy fails:
# Only concrete class can be given where "Type[Transaction]" is expected [type-abstract]
any_transactions = self.get_transactions(uuids, Transaction)
I tried using overloads like this:
#overload
def get_transactions(
self, uuids: Collection[str], type_: type[CashTransaction]) -> list[CashTransaction]:
...
#overload
def get_transactions(
self, uuids: Collection[str], type_: type[SecurityTransaction]) -> list[SecurityTransaction]:
...
#overload
def get_transactions(
self, uuids: Collection[str], type_: type[Transaction]) -> list[Transaction]:
...
def get_transactions(
self, uuids: Collection[str], type_: type[TransactionType]) -> list[TransactionType]:
# actual implementation here
but that leads to errors like this: Overloaded function signatures 1 and 3 overlap with incompatible return types [misc]
Is there any solution that would work when the ABC is supplied as the type_?
Thanks!
I'm trying to define a couple of dataclasses and an abstract class that manipulates those classes. Eventually, the my_class_handler types could be dealing with say: json, xml or sqlite files as concrete instance types.
Can someone please explain to me what this message means?
<bound method my_class_handler.class_name of <__main__.my_class_handler object at 0x000001A55FB96580>>
Here's the source code that generates the error for me.
from abc import ABC, abstractmethod
from dataclasses import dataclass
from typing import List
#dataclass
class column:
name: str
heading: str
#dataclass
class my_class:
class_name: str
class_description: str
columns: List[column]
class iclass_handler(ABC):
#abstractmethod
def class_name(self) -> str:
pass
#abstractmethod
def class_name(self, value: str):
pass
class my_class_handler(iclass_handler):
obj: my_class
def __init__(self):
self.obj = my_class("test-class", "", None)
def class_name(self) -> str:
return self.obj.class_names
def class_name(self, value: str):
if (value != self.obj.class_name):
self.obj.class_name = value
if __name__ == '__main__':
handler = my_class_handler()
print(handler.class_name)
If this is not the proper way of doing this, please point me in the direction where I might learn the proper way.
Thanks for your time,
Python does not allow overloading like Java, so remove methods that overlap.
#khelwood pointed out the answer to the original question. Thanks
As for the #property approach, I tried that and was having nothing but problems and couldn't find any useful examples of inherited properties so I just rewrote the function to take an additional parameter:
# I'm working from memory here but I believe this is the jist...
def class_name(self, new_value: str = None) -> str:
if (new_value is None)
return self.obj.class_name
if (isinstance(new_value, str)):
if (new_value != self.obj.class_name):
self.obj.class_name = new_value
return None
Anyhow, I have since refactored and have completely removed the whole class_name() method as a result of a redesign that dropped the whole concept of data-handlers.
Thanks again for the comments.
Coming from a C# background and knowing its generic type approaches I'm now trying to implement something similar in Python. I need to serialize and de-serialize classes in a special string format, so I created the following two base classes, the first for single entity serialization and the second one for list serialization of that entity type.
from typing import Any, TypeVar, List, cast, Type, Generic, NewType
import re
T = TypeVar('T')
class Serializable(Generic[T]):
def to_str(self) -> str:
raise NotImplementedError
#classmethod
def from_str(cls, str: str):
raise NotImplementedError
class SerializableList(List[Serializable[T]]):
def __init__(self):
self.separator: str = "\n"
#classmethod
def from_str(cls, str: str):
list = cls()
for match in re.finditer(list.separator, str):
list.append(T().from_str(match)) # <-- PROBLEM: HOW TO INIT A GENERIC ENTITY ???
# list.append(Serializable[T].from_str(match)) <-- Uses base class (NotImplemented) instead of derived class
return list
def to_str(self) -> str:
str = ""
for e in self:
str = str + f"{e.to_str()}{self.separator}"
return str
Then I can derive from those classes and have to implement to_str and from_str. Please see the marker <-- PROBLEM". I have no idea how I can init a new entity of the currently used type for the list. How do we do this in the Python way?
As #user2357112supportsMonica says in the comments, typing.Generic is pretty much only there for static analysis, and has essentially no effect at runtime under nearly all circumstances. From the look of your code, it looks like what you're doing might be better suited to Abstract Base Classes (documentation here, tutorial here), which can be easily combined with Generic.
A class that has ABCMeta as its metaclass is marked as an Abstract Base Class (ABC). A subclass of an ABC cannot be instantiated unless all methods in the ABC marked with the #abstractmethod decorator have been overridden. In my suggested code below, I've explicitly added the ABCMeta metaclass to your Serializable class, and implicitly added it to your SerializableList class by having it inherit from collections.UserList instead of typing.List. (collections.UserList already has ABCMeta as its metaclass.)
Using ABCs, you could define some interfaces like this (you won't be able to instantiate these because of the abstract methods):
### ABSTRACT INTERFACES ###
from abc import ABCMeta, abstractmethod
from typing import Any, TypeVar, Type, Generic
from collections import UserList
import re
T = TypeVar('T')
class AbstractSerializable(metaclass=ABCMeta):
#abstractmethod
def to_str(self) -> str: ...
#classmethod
#abstractmethod
def from_str(cls: Type[T], string: str) -> T: ...
S = TypeVar('S', bound=AbstractSerializable)
class AbstractSerializableList(UserList[S]):
separator = '\n'
#classmethod
#property
#abstractmethod
def element_cls(cls) -> Type[S]: ...
#classmethod
def from_str(cls, string: str):
new_list = cls()
for match in re.finditer(cls.separator, string):
new_list.append(cls.element_cls.from_str(match))
return new_list
def to_str(self) -> str:
return self.separator.join(e.to_str() for e in self)
You could then provide some concrete implementations like this:
class ConcreteSerializable(AbstractSerializable):
def to_str(self) -> str:
# put your actual implementation here
#classmethod
def from_str(cls: Type[T], string: str) -> T:
# put your actual implementation here
class ConcreteSerializableList(AbstractSerializableList[ConcreteSerializable]:
# this overrides the abstract classmethod-property in the base class
element_cls = ConcreteSerializable
(By the way — I changed several of your variable names — str, list, etc — as they were shadowing builtin types and/or functions. This can often lead to annoying bugs, and even if it doesn't, is quite confusing for other people reading your code! I also cleaned up your to_str method, which can be simplified to a one-liner, and moved your separator variable to be a class variable, since it appears to be the same for all class instances and does not appear to ever be altered.)
For now I found a dirty solution - this is to add a Type (constructor) parameter of the list entries like so:
class SerializableList(List[Serializable[T]]):
# This one
# |
# v
def __init__(self, separator: str = "\n", entity_class: Type = None):
self.separator = separator
self.entity_class = entity_class
#classmethod
def from_str(cls, str: str):
list = cls()
for match in re.finditer(list.separator, str):
list.append(list.entity_class.from_str(match))
return list
I wonder if there is a cleaner way to get the correct [T] type constructor from List[T] since it is already provided there?
I want to create an inheritable class that can be parameterized with a type. Here is a working example without type annotations:
class Simple:
pass
class Bla:
obj_class = Simple
def do(self):
return self.obj_class()
def check(self, x):
return isinstance(x, self.obj_class)
Users of this code would inherit from Bla, and can set a different obj_class, like so:
class Advanced(Simple):
pass
class Foo(Bla):
obj_class = Advanced
The problem starts when I want to correctly type annotate this. I thought of making Bla inherit from Generic[T], where T is defined as TypeVar('T', bound=Simple), but then the constructor T() and isinstance won't work, and also manually assigning a different class to obj_class also doesn't work.
Here is one non-working example, as T can't be used in non-typing contexts:
class Simple:
pass
T = TypeVar('T', bound=Simple)
class Bla(Generic[T]):
def do(self) -> T:
return T()
def check(self, x: Any) -> bool:
return isinstance(x, T)
Here is another non-working example, where I can't assign Simple to obj_class because of incompatible types.
class Simple:
pass
T = TypeVar('T', bound=Simple)
class Bla(Generic[T]):
obj_class: Type[T] = Simple
def do(self) -> T:
return self.obj_class()
def check(self, x: Any) -> bool:
return isinstance(x, self.obj_class)
class Advanced(Simple):
pass
class Foo(Bla):
obj_class = Advanced
Is there a way to solve this?
You don't need Type[T] = Simple.
mypy states:
Incompatible types in assignment (expression has type "Type[Simple]", variable has type "Type[T]").
You are trying to assign a concrete type to a generic type variable.
Instead, do something like:
class Simple:
pass
class Advanced(Simple):
pass
class Other:
pass
T = TypeVar('T', bound=Simple)
class Bla(Generic[T]):
obj_class: Type[T]
def do(self) -> T:
return self.obj_class()
def check(self, x: Any) -> bool:
return isinstance(x, self.obj_class)
class SimpleFoo(Bla[Simple]):
obj_class = Simple
class AdvancedFoo(Bla[Advanced]):
obj_class = Advanced
class OtherFoo(Bla[Other]):
obj_class = Other
Now, mypy correctly states:
error: Type argument "tmp.Other" of "Bla" must be a subtype of "tmp.Simple"
Note
OtherFoo has to subclass Bla with a specific type so mypy will correctly warn you.
The following produces no errors:
class OtherFoo(Bla):
obj_class = Other
Say I have a base class
from typing import List, Optional
class Node:
def __init__(self, name: str) -> None:
self.name = name
self.children: List['Node'] = []
...
and a subclass
class PropertiesNode(Node):
def __init__(
self, name: str, properties: List[str], inherit: Optional['PropertiesNode']
) -> None:
Node.__init__(self, name)
self.properties = set(properties)
if inherit:
self.properties.update(inherit.properties)
self.children = deepcopy(inherit.children)
for child in self.children:
child.properties.update(properties)
# ^ ERR: "Node" has no attribute "properties" [attr-defined]
As you can see, mypy (rightly) flags an error there, as Node.children was explicitly given a type of List[Node].
So I read up on generic types, and it seems to me the solution is to use TypeVars and Generic:
from typing import Generic, List, Optional, TypeVar
N = TypeVar('N', bound='Node')
P = TypeVar('P', bound='PropertiesNode')
class Node(Generic[N]):
def __init__(self: N, name: str) -> None:
self.name = name
self.children: List[N] = []
class PropertiesNode(Node[P]):
def __init__(
self: P, name: str, properties: List[str], inherit: Optional[P]
) -> None:
Node.__init__(self, name)
self.properties = set(properties)
if inherit:
self.properties.update(inherit.properties)
self.children = deepcopy(inherit.children)
for child in self.children:
child.properties.update(properties)
However, now when I instantiate the classes, I get
foo = Node("foo")
# ^ ERR Need type annotation for "foo" [var-annotated]
bar = PropertiesNode("bar", ["big", "green"], None)
# ^ ERR Need type annotation for "bar" [var-annotated]
Now, I could silence these by doing
foo: Node = Node("foo")
bar: PropertiesNode = PropertiesNode(...)
but why does that silence it - I'm not giving mypy any new info there? The more I think about, the less Generic seems like the right choice, because the thing is: all instances of Node or PropertiesNode will have self.children that are of exactly the same type as self.
But if I remove the Generic[N] from class Node(Generic[N]):, I end up with the original error again:
class PropertiesNode(Node):
...
child.properties.update(properties)
# ^ ERR "N" has no attribute "properties" [attr-defined]
There are two things going on here
1. Generics
Annotating a variable foo: Node, where Node is a generic class, is equivalent to annotating it as foo: Node[typing.Any]. It will silence MyPy on the default settings, but if you choose to use MyPy with some of the stricter flags set to True (which I recommend doing!), you'll find that MyPy still flags this kind of thing as an error.
If you run this in MyPy:
from typing import TypeVar, Generic, List
N = TypeVar('N', bound='Node')
class Node(Generic[N]):
def __init__(self: N, name: str) -> None:
self.name = name
self.children: List[N] = []
foo: Node = Node("foo")
reveal_type(foo)
You'll find that MyPy will come back to you with a message similar to:
Revealed type is "__main__.Node[Any]"
(N.B. reveal_type is a function that MyPy recognises, but that will fail if you try to use it at runtime.)
To get MyPy to flag unparameterised generics as errors, run MyPy with the command-line argument --disallow-any-generics. Doing so will mean MyPy will flag the following errors:
main.py:3: error: Missing type parameters for generic type "Node"
main.py:10: error: Missing type parameters for generic type "Node"
Forcing you to adjust your code to the following:
from typing import TypeVar, Generic, List, Any
N = TypeVar('N', bound='Node[Any]')
class Node(Generic[N]):
def __init__(self: N, name: str) -> None:
self.name = name
self.children: List[N] = []
foo: Node[Any] = Node("foo")
This makes MyPy happy once again, and says the same thing that you said in your original code, but more explicitly.
However...
2. I don't think it's necessary to use Generics in this situation
You don't have to inherit from generic in order to annotate the self argument in __init__ with a TypeVar. Moreover, as you say in your question, inheriting from Generic just doesn't really make sense here, either from the perspective of MyPy or from other humans reading your code. I'd modify your code like so:
from typing import List, Optional, TypeVar, Any
from copy import deepcopy
N = TypeVar('N', bound='Node')
P = TypeVar('P', bound='PropertiesNode')
class Node:
def __init__(self: N, name: str, *args: Any, **kwargs: Any) -> None:
self.name = name
self.children: List[N] = []
class PropertiesNode(Node):
def __init__(self: P, name: str, properties: List[str], inherit: Optional[P], *args: Any, **kwargs: Any) -> None:
super().__init__(name)
self.properties = set(properties)
if inherit is not None:
self.properties.update(inherit.properties)
self.children: List[P] = deepcopy(inherit.children)
for child in self.children:
child.properties.update(properties)
Now we have annotations that will make MyPy happy, even on the strictest settings, and they even make sense to humans as well!
N.B. I changed two other things in your code here:
I added *args, **kwargs parameters to your __init__ methods — as written, they violated the Liskov Substitution Principle. By adding in these parameters, you avoid that issue.
I changed your test from if inherit to if inherit is not None — lots of things can be False-y in python, so it's much safer to test by identity when testing if a value is None or not.