I'm trying to define a couple of dataclasses and an abstract class that manipulates those classes. Eventually, the my_class_handler types could be dealing with say: json, xml or sqlite files as concrete instance types.
Can someone please explain to me what this message means?
<bound method my_class_handler.class_name of <__main__.my_class_handler object at 0x000001A55FB96580>>
Here's the source code that generates the error for me.
from abc import ABC, abstractmethod
from dataclasses import dataclass
from typing import List
#dataclass
class column:
name: str
heading: str
#dataclass
class my_class:
class_name: str
class_description: str
columns: List[column]
class iclass_handler(ABC):
#abstractmethod
def class_name(self) -> str:
pass
#abstractmethod
def class_name(self, value: str):
pass
class my_class_handler(iclass_handler):
obj: my_class
def __init__(self):
self.obj = my_class("test-class", "", None)
def class_name(self) -> str:
return self.obj.class_names
def class_name(self, value: str):
if (value != self.obj.class_name):
self.obj.class_name = value
if __name__ == '__main__':
handler = my_class_handler()
print(handler.class_name)
If this is not the proper way of doing this, please point me in the direction where I might learn the proper way.
Thanks for your time,
Python does not allow overloading like Java, so remove methods that overlap.
#khelwood pointed out the answer to the original question. Thanks
As for the #property approach, I tried that and was having nothing but problems and couldn't find any useful examples of inherited properties so I just rewrote the function to take an additional parameter:
# I'm working from memory here but I believe this is the jist...
def class_name(self, new_value: str = None) -> str:
if (new_value is None)
return self.obj.class_name
if (isinstance(new_value, str)):
if (new_value != self.obj.class_name):
self.obj.class_name = new_value
return None
Anyhow, I have since refactored and have completely removed the whole class_name() method as a result of a redesign that dropped the whole concept of data-handlers.
Thanks again for the comments.
Related
First, a project that I am working using the plugin systme similar to PEP-0478 using a PriortyQueue for insuring plugins run in the right order. I am going to leave out the how this works:
It is well documented in PEP-487
Not directly required for understanding this issue.
First, I have basic classes as below:
# abstract.py
import abc
from dataclasses import data, field
#dataclass(order=True)
class PluginBase(abc.ABC):
value: int
def __eq__(self, __o: object) -> bool:
print("PluginBase __eq__ called")
return self.value == __o.value
# plugin_a.py
from abstract import PluginBase
class PluginA(PluginBase):
pass
# plugin_b.py
from abstract import PluginBase
class PluginB(PluginBase):
pass
First, after the plugins are discovered and loaded they are wrapped in a dataclass this is to ensure min amount of code for each plugin.
Problem: __eq__() is never passed to subclasses.
It does not matter how I setup the classes or even use funtools.total_ordering.
One thing I noticed, is that if you do assert PluginA(1) == PluginB(1) always returns false and assert PluginA(1) < PluginB(2) always returns TypeError: '<' not supported between instances with the current classes.
This this always intended?
To fix the first issue, where == doesn't work you need to add:
def __eq__(self, __o: object) -> bool:
print("Plugin __eq__ called")
return super().__eq__(__o)
To one or both subclasses which addes more boiler plate code to the plugin system which I do not want.
To fix the '<' issue, you need to change PluginBase to:
#functools.total_ordering
#dataclass
class PluginBase(abc.ABC):
value: int
def __eq__(self, __o: object) -> bool:
print("PluginBase __eq__ called")
return self.value == __o.value
def __lt__(self, __o: object) -> bool:
print("PluginBase __lt__ called")
return self.value < __o.value
This allows you to do PluginA(1) < PluginB(2) which will be true. However, == still does not work here.
I think this is due to the differences in class instances and Python is enforcing __eq__() to check the instance type before anything else. Is there anyway to get this to work?
One solution is to wrap each of the Plugins into a comparable object and use that into the Queue.
#chepner was on the right track here.
Change code to here works:
# abstract.py
import abc
from dataclasses import dataclass, field
#functools.total_ordering
#dataclass
class PluginBase(abc.ABC):
value: int
def __eq__(self, __o: object) -> bool:
print("PluginBase __eq__ called")
return self.value == __o.value
def __lt__(self, __o: object) -> bool:
print("PluginBase __lt__ called")
return self.value < __o.value
# plugin_a.py
from dataclasses import dataclass
from abstract import PluginBase
#dataclass(eq=False)
class PluginA(PluginBase):
pass
# plugin_b.py
from dataclasses import dataclass
from abstract import PluginBase
#dataclass(eq=False)
class PluginB(PluginBase):
pass
Adding the eq=False forces it not to genereate the interclass eq() functions which then defaults to the base class.
assert PluginA(1) < PluginB(2) # True
PluginBase __lt__ called
assert PluginA(1) == PluginB(1) # True
PluginBase __eq__ called
Some reason I just missed trying this combonation of options.
I have a class like the one below.
from enum import Enum, unique
from typing import Tuple
#unique
class Test(Enum):
v1: Tuple = (1, "value_1")
v2: Tuple = (2, "value_2")
def __init__(self, value_number: int, value_name: str) -> None:
self.value_number = value_number
self.value_name = value_name
#classmethod
def get_value_name_by_value_number(cls, number: int) -> str:
for member in cls.__members__.values():
if member.value_number == number:
return member.value_name
else:
raise NotImplementedError()
if __name__ == '__main__':
name = Test.get_value_name_by_value_number(number=2)
print(name)
When I run the code, the output is correct. But I don't understand why it's working with no problem. As far as I know the __init__ method is called on the creation of an instance. The method I'm calling is classmethod and thus I'm not creating an object and calling it directly from the class. So the __init__ is not invoked. But how does my class know what member.value_number and member.value_name are?
When creating the Test class, the Enum internals will create all enum members and manually call your __init__ method:
enum_member.__init__(*args)
You'd think it'd just instantiate your class normally somewhere, but with how much weird magic goes on in the Enum implementation, it turns out that wouldn't work.
Coming from a C# background and knowing its generic type approaches I'm now trying to implement something similar in Python. I need to serialize and de-serialize classes in a special string format, so I created the following two base classes, the first for single entity serialization and the second one for list serialization of that entity type.
from typing import Any, TypeVar, List, cast, Type, Generic, NewType
import re
T = TypeVar('T')
class Serializable(Generic[T]):
def to_str(self) -> str:
raise NotImplementedError
#classmethod
def from_str(cls, str: str):
raise NotImplementedError
class SerializableList(List[Serializable[T]]):
def __init__(self):
self.separator: str = "\n"
#classmethod
def from_str(cls, str: str):
list = cls()
for match in re.finditer(list.separator, str):
list.append(T().from_str(match)) # <-- PROBLEM: HOW TO INIT A GENERIC ENTITY ???
# list.append(Serializable[T].from_str(match)) <-- Uses base class (NotImplemented) instead of derived class
return list
def to_str(self) -> str:
str = ""
for e in self:
str = str + f"{e.to_str()}{self.separator}"
return str
Then I can derive from those classes and have to implement to_str and from_str. Please see the marker <-- PROBLEM". I have no idea how I can init a new entity of the currently used type for the list. How do we do this in the Python way?
As #user2357112supportsMonica says in the comments, typing.Generic is pretty much only there for static analysis, and has essentially no effect at runtime under nearly all circumstances. From the look of your code, it looks like what you're doing might be better suited to Abstract Base Classes (documentation here, tutorial here), which can be easily combined with Generic.
A class that has ABCMeta as its metaclass is marked as an Abstract Base Class (ABC). A subclass of an ABC cannot be instantiated unless all methods in the ABC marked with the #abstractmethod decorator have been overridden. In my suggested code below, I've explicitly added the ABCMeta metaclass to your Serializable class, and implicitly added it to your SerializableList class by having it inherit from collections.UserList instead of typing.List. (collections.UserList already has ABCMeta as its metaclass.)
Using ABCs, you could define some interfaces like this (you won't be able to instantiate these because of the abstract methods):
### ABSTRACT INTERFACES ###
from abc import ABCMeta, abstractmethod
from typing import Any, TypeVar, Type, Generic
from collections import UserList
import re
T = TypeVar('T')
class AbstractSerializable(metaclass=ABCMeta):
#abstractmethod
def to_str(self) -> str: ...
#classmethod
#abstractmethod
def from_str(cls: Type[T], string: str) -> T: ...
S = TypeVar('S', bound=AbstractSerializable)
class AbstractSerializableList(UserList[S]):
separator = '\n'
#classmethod
#property
#abstractmethod
def element_cls(cls) -> Type[S]: ...
#classmethod
def from_str(cls, string: str):
new_list = cls()
for match in re.finditer(cls.separator, string):
new_list.append(cls.element_cls.from_str(match))
return new_list
def to_str(self) -> str:
return self.separator.join(e.to_str() for e in self)
You could then provide some concrete implementations like this:
class ConcreteSerializable(AbstractSerializable):
def to_str(self) -> str:
# put your actual implementation here
#classmethod
def from_str(cls: Type[T], string: str) -> T:
# put your actual implementation here
class ConcreteSerializableList(AbstractSerializableList[ConcreteSerializable]:
# this overrides the abstract classmethod-property in the base class
element_cls = ConcreteSerializable
(By the way — I changed several of your variable names — str, list, etc — as they were shadowing builtin types and/or functions. This can often lead to annoying bugs, and even if it doesn't, is quite confusing for other people reading your code! I also cleaned up your to_str method, which can be simplified to a one-liner, and moved your separator variable to be a class variable, since it appears to be the same for all class instances and does not appear to ever be altered.)
For now I found a dirty solution - this is to add a Type (constructor) parameter of the list entries like so:
class SerializableList(List[Serializable[T]]):
# This one
# |
# v
def __init__(self, separator: str = "\n", entity_class: Type = None):
self.separator = separator
self.entity_class = entity_class
#classmethod
def from_str(cls, str: str):
list = cls()
for match in re.finditer(list.separator, str):
list.append(list.entity_class.from_str(match))
return list
I wonder if there is a cleaner way to get the correct [T] type constructor from List[T] since it is already provided there?
So I have a situation where one module writes some code that processes records, so lets say it does this:
from datetime import datetime
def is_record_today(rec: Record) -> bool:
return (rec.date == str(datetime.now().date()))
def is_record_valid(rec: Record) -> bool:
return record.is_valid()
so at this time I need to define:
import abc
class Record(abc.ABC):
#abc.abstractproperty
def date(self) -> str:
print('I am abstract so should never be called')
#abc.abstractmethod
def is_valid(self) -> bool:
print('I am abstract so should never be called')
now when I am processing a record in another module I want to inherit from this abstract class so I raise an error if I don't have date and is_valid. However, I also want to inherit from another class to get the default properties, lets say I want my record to be manipulatable like a dictionary. Then:
class ValidRecord(Record, dict):
#property
def date(self) -> str:
return self['date']
def is_valid(self) -> bool:
return self['valid']
class InvalidRecord(Record, dict):
#property
def date(self) -> str:
return self['date']
we would expect that ValidRecord should create without issue and InvalidRecord should throw TypeError, however both seem fine, and I can even call the missing abstractmethod from the Record class which as far as I understand abstract methods should not be possible:
data = {
'date': '2021-05-01',
'valid': False
}
InValidRecord(data).is_valid()
>>> "I am abstract so should never be called"
If I take away the dictionary inheritance I get the expected error so, what is going on here and how can I get the desired behavior of inheriting from one class but requiring additional methods be added?
Scenario
Lets say I have a generic Store class that implements various methods to retrieve StoreObjects. And to fill the store it defines an abstract method load_object.
Then I create a CarStore. I derive from the Store and overwrite the load_object method to return Car objects.
Now the question is how to add type hints for this. First the code:
from typing import Dict
import weakref
import abc
class StoreObject:
pass
class Car(StoreObject):
def __init__(self, color: str):
self.color = color # type: str
class Store(abc.ABC):
def __init__(self):
self._cache = weakref.WeakValueDictionary() # type: weakref.WeakValueDictionary[int, StoreObject]
def get(self, index: int) -> StoreObject:
try:
return self._cache[index]
except KeyError:
obj = self.load_object(index)
self._cache[index] = obj
return obj
#abc.abstractmethod
def load_object(self, index: int) -> StoreObject:
raise NotImplementedError
class CarStore(Store):
def load_object(self, index: int) -> Car:
if index < 100:
return Car("red")
else:
return Car("blue")
store = CarStore()
car = store.get(10)
print("Your car color is", car.color)
Type Checking Errors
The problem is in the following line:
print("Your car color is", car.color)
Here PyCharm gives the following warning:
Unresolved attribute reference 'color' for class 'StoreObject'
Mypy gives the following error:
development/storetyping.py:39: error: "StoreObject" has no attribute "color"
Also the PyCharm code completion does obviously not include the name method for store.get(10).?.
Question
How can I type the base class such that PyCharm and mypy can successfully check this code?
Is there a way to parameterize the types in Store such that when creating CarStore I can tell it to use Car instead of StoreObject in the annotations?
In more static languages you would create Store as a generic class and use Car as a type parameter when inheriting from Store.
We can actually do that using the typing module in python.
Here is a minimal example:
from typing import Generic, TypeVar
T = TypeVar('T') # this is the generic placeholder for a type
# Store is a generic class with type parameter T
class Store(Generic[T]):
def get(self) -> T: # this returns a T
return self.load_object()
def load_object(self) -> T: # this also returns a T
raise NotImplementedError
class Car:
def __init__(self, color):
self.color = color
# Now we inherit from the Store and use Car as the type parameter
class CarStore(Store[Car]):
def load_object(self):
return Car('red')
s = CarStore()
c = s.get()
print(c.color) # Code completion works and no warnings are shown
Edit:
To address ShadowRanger's note: If you want Car and all the products to have a common base class you can use the bound parameter of the TypeVar. Thank you juanpa.arrivillaga for the hint.
So we create a Product class and bind the TypeVar to it.
class Product:
def get_id(self):
raise NotImplementedError
T = TypeVar('T', bound=Product)
Mypy will now complain about this:
class CarStore(Store[Car]):
def load_object(self):
return Car('red')
because a Car is not a Product. So let' change that, too:
class Car(Product):
def get_id(self):
return ...
def __init__(self, color):
self.color = color
And now, mypy is happy.
Edit2:
Here is the full code with some more annotations, that make even mypy --strict happy.
from typing import Generic, TypeVar
class Product:
def get_id(self) -> int:
raise NotImplementedError
T = TypeVar('T', bound=Product)
class Store(Generic[T]):
def get(self) -> T:
return self.load_object()
def load_object(self) -> T:
raise NotImplementedError
class Car(Product):
def get_id(self) -> int:
return hash(self.color)
def __init__(self, color: str):
self.color = color
class CarStore(Store[Car]):
def load_object(self) -> Car:
return Car('red')
if __name__ == '__main__':
s = CarStore()
c = s.get()
print(c.color)
Your type checking is behaving correctly; get is not overridden in CarStore, so the annotation on it continues to specify that it returns StoreObject. If you want to change the annotation, you'd have to redefine get in CarStore, e.g. by adding:
def get(self, index: int) -> Car:
return typing.cast(Car, super().get(index))
Make sure to import typing to gain access to cast (or use unqualified cast and add it to your from typing import Dict import).
To avoid runtime performance overhead, you could only conditionally define get based on an if typing.TYPE_CHECKING: test (which returns True when static checkers are analyzing the code, and False when running it), so the get overload isn't actually defined at runtime.