My dataclass has a field that holds an array of data in a custom type (actually it is a PyROOT std vector). However, for the user it is supposed to be visible as a list. This is simple enough with dataclass getters and setters, that convert the vector to list and vice versa. However, this works only if the user initialises the field with a full list. If the user wants to append to the list, it, obviously, doesn't work, as there is no permanent list associated with the field.
I wonder if there is a way to inhibit the ".append()" call on the field and call instead the vector's push_back()? Or perhaps there is a good Pythonic way to deal with it in general?
The context is, that I need the dataclass fields in the PyROOT format, as later I am storing the data in ROOT TTrees. However, I am creating this interface, so that the user does not need to know ROOT to use the dataclass. I know that I could create both the vector and the list that would hold the same data, but that seems like a waste of memory, and I am not certain how to update the vector each time the list is modified, anyway.
According to the Python Docs, “Lists are mutable sequences, typically used to store collections of homogeneous items (where the precise degree of similarity will vary by application).” (emphasis added)
With that in mind, I would start off with something like this:
from collections.abc import MutableSequence
class ListLike(MutableSequence):
def __init__(self):
self.backing_data = object() # Replace with the type your using
ListLike()
When you run that code, you’ll get the error: TypeError: Can't instantiate abstract class ListLike with abstract methods __delitem__, __getitem__, __len__, __setitem__, insert. Once you implement those methods, you’ll have have a type that acts a lot like list, but isn’t.
To make ListLikes act even more like lists, use this code to compare the two:
example_list = list()
example_list_like = ListLike()
list_attributes = [attribute for attribute in dir(example_list)]
list_like_attributes = [attribute for attribute in dir(example_list_like)]
for attribute in list_attributes:
if attribute not in list_like_attributes:
print(f"ListLikes don't have {attribute}")
print("-----------")
for attribute in list_like_attributes:
if attribute not in list_attributes:
print(f"lists don't have {attribute}")
and change your implementation accordingly.
Related
I have a class family for which I need to be able to iterate through attributes of type: Metric.
The family consists of an abstract base class parent and child classes. The child classes will all have varying number of class attributes of type Metric, and they inherit an __iter__ method from the parent class that allows me to iterate through the attributes.
I am using iterable attributes rather than a dict because I want my objects to be typed, but I need to be able to call metrics in sequence, and by name.
So I need to be able to do:
Metrics.metric_1
and
for metric in Metrics:
My question is, how do I correctly hint in the base class that there are a variable number of attributes of the same type?
I'm currently using a couple of attribute hints with an ellipsis:
class MetricsBase(ABC):
metric_1: Metric
metric_2: Metric
...
#classmethod
def __iter__(cls):
for attr, value in cls.__dict__.items():
if not attr.startswith("__"):
yield value
class MetricChild(MetricsBase):
metric_1 = Metric(x)
metric_2 = Metric(y)
metric_3 = Metric(z)
But I'm not sure if this is pythonic or correct, and wondering if there is a neater way of doing this.
Many thanks for any input!
I am not answering how to "fix" static type checking on that.
That said, this is ok as Python code, hence "pythonic" . the problem is that you want to use static type checking on it - and you are using a dynamic meta programming technique there. Static type checking is not meant to check this (in a way of saying, it can only handle a small subset of what would be "pythonic"). Maybe there is a way to "solve" this - but if you can, just mark the static type checkers to skip that, and spare hours yourself hours of meaningless work (since it won't change how the code works)
More important than that, that __iter__ method won't work for the class itself, regardless of you marking it as a #classmethod. (It will work fot the instances, despite you doing so, though). If you want to iterate on the class, you will have to resort to a metaclass:
import abc
class MetricMeta(abc.ABCMeta):
def __iter__(cls):
# this will make _instances of this metaclass__ iterable
for attr, value in cls.__dict__.items():
if not attr.startswith("__"):
yield value
class MetricsBase(metaclass=MetricsMeta):
metric_1: Metric
metric_2: Metric
...
Type chekers actually, should, supposedly, not need one to expliclty annotate all variables, reducing Python to a subset of Pascal or the like. If you simply type in your class attributes in each subclass, attributing a Metric instance to then it should work, without the need to explictly annotate each one with a :Metric.
They will certainly complain when you try to iterate over a class with a statement like for metric in Metrics:, but that is easily resolvable by asserting to it explicitly that the class is iterable, using typing.cast. No tool (at least not yet) will be able to "see" that the metaclass you are using feature an __iter__ method that enables the class itself to be iterable.
from typing import cast
from collections.abc import Iterable
...
for metric in cast(Iterable, metrics):
...
I'm struggling to get type-hinting to work as I would expect regarding the content types for custom container subclasses in PyCharm. Let's start with a case that works as I would expect. You can create a subclass of list and specify that it will always have int content, and Pycharm then recognizes that each item in such a list will be an int.
class IntList(list[int]): pass
il = IntList(())
answer1 = il[0]
When I mouse-over answer1 Pycharm says that it expects this to have type int, presumably because the class declaration specified that an IntList wouldn't be any old list, but instead would be a list[int]. (Never mind the fact that this code would raise an error when run because il would be empty. This was just a minimal example to show that PyCharm can sometimes draw type-hinting information from the bracketed [int] in the class declaration. The same issues arise in other cases where getting an item from the container wouldn't raise an error.)
So this worked fine when subclassing from list. But what I want to do is to create my own generic container class -- call it Box -- that could contain a variety of different types of objects. Then I want to declare my own subclass IntBox that will contain only int items, and I want PyCharm to recognize this in its various mouse-over hints, auto-completion suggestions, and linting error detection, just like it could for IntList. So here's a very pared-down example of what I'd like.
class Box(list): pass
class IntBox(Box[int]): pass
ib = IntBox(())
answer2 = ib[0]
In this case, when I mouse-over answer2, PyCharm says that it could have type Any and does not recognize that the [int] implies that this is not just a generic Box/list, but instead one whose contents have been type-hinted to be int.
I've tried all the variations I can imagine using typing.TypeVar and typing.Generic to try to more explicitly indicate that each subclass of Box will have a single type of contents, that Box.__getitem__ will return that type, and that, for the subclass IntBox that type is int.
The only solution I've found is that, when I create ib I can explicitly declare that this instance has type IntBox[int] and then PyCharm will know to expect that ib[0] will be an int. But it seems like I shouldn't need to explicitly say that each time I create an IntBox instance, and instead there should be some way to get PyCharm to infer this from the [int] in the class declaration for IntBox just like it could for IntList.
Of course this was just a toy example. In the real case that's motivating this, I want my generic container class "Box" to define other methods (not just __getitem__) that are type-hinted to return whichever specific type of object the subclass of "Box" in question always contains, where this varies across subclasses. Using TypeVar and Generic I can get this to work, if I explicitly type-declare that each subclass instance will contain a particular [contenttype], but I can't find a way to get it to work without tedious explicit type-declarations on each instance.
Edit: since solutions that work in the simple case of telling what elements will be in a list sub-subclass apparently don't automatically scale up to this real case, here's an example closer to what I need, including that Box is a nested Sequence rather than a simple list, including a Box.get_first() method that should also receive type-hinting of int for IntBox, and including what I think is roughly the right use of TypeVar:
from typing import TypeVar, Sequence
T = TypeVar('T')
class Box(Sequence[Sequence[T]]):
def get_first(self:'Box[T]')->T:
return self[0][0]
class IntBox(Box[int]): pass
ib = IntBox() # works only if I declare this is type: IntBox[int]
answer = ib.get_first() # hovering over answer should show it will be int
Further edit: the problem with the preceding seems to be the nested Sequence[Sequence[T]]. Changing this to Generic[T] makes things work as expected.
Use typing.Generic to pass a type to the Box superclass from a subclass using a typing.TypeVar
from typing import Generic, TypeVar
T = TypeVar('T')
class Box(Generic[T], list[T]):
pass
class IntBox(Box[int]):
pass
class StrBox(Box[str]):
pass
ib = IntBox(())
answer1 = ib[0] # int
sb = StrBox(())
answer2 = sb[0] # str
I have a namedtuple that's a private property of a class instance. I want to create a new tuple based off an instance's namedtuple, but tuples are immutable, so I create a new one and just insert what I want. Is there a better way of doing this? It's extremely redundant, and I would like a specific function or so that would just take a namedtuple as a parameter and as the other parameter the attribute you want to change and it returns the new namedtuple. What would this look like?
There's a _replace method for that:
new_namedtuple = old_namedtuple._replace(field_name=new_value)
Plus, it'll avoid the bug in your current code, where if two fields share the same value, your code might end up replacing the wrong one.
I was checking out the very nice collections library and more specific the Abstract Base Classes (ABC). One I could not get my head around: the MappingView.
What is its use? What is its advantage over Sized? An example perhaps?
Documentation says its base class is Sized, but on the other hand there is a len mixin... So do we have to implement len, or not?
For the documentation, see collections
MappingView is essentially the base class for user defined views. It implements the Sized interface by providing the __len__ attribute which is the length of its _mapping member, so if this implementation is fine for you, you do not need to implement your own __len__.
It holds common code for KeysView, ItemsView and ValuesView. These last classes can be used wherever a view like my_dict.keys(), my_dict.items() or my_dict.values() would be expected. If you create a new user defined kind of data and want to create a view that can be neither compared to keys, values or items, then you could subclass MappingView directly and implement differently the __contains__ and __iter__ functions.
Might be a n00b question, but I currently have a class that implements an iterator so I can do something like
for i in class():
but I want to be able to access the class by index as well like
class()[1]
How can I do that?
Thanks!
The current accepted answer from #Ignacio Vazquez-Abrams is sufficient. However, others interested in this question may want to consider inheriting their class from an abstract base class (ABC) (such as those found in the standard module collections.abc). This does a number of things (there are probably others as well):
ensures that all of the methods you need to treat your object "like a ____" are there
it is self-documenting, in that someone reading your code is able to instantly know that you intend your object to "act like a ____".
allows isinstance(myobject,SomeABC) to work correctly.
often provides methods auto-magically so we don't have to define them ourselves
(Note that, in addition to the above, creating your own ABC can allow you to test for the presence of a specific method or set of methods in any object, and based on this to declare that object to be a subclass of the ABC, even if the object does not inherit from the ABCdirectly. See this answer for more information.)
Example: implement a read-only, list-like class using ABC
Now as an example, let's choose and implement an ABC for the class in the original question. There are two requirements:
the class is iterable
access the class by index
Obviously, this class is going to be some kind of collection. So what we will do is look at our menu of collection ABC's to find the appropriate ABC (note that there are also numeric ABCs). The appropriate ABC is dependent upon which abstract methods we wish to use in our class.
We see that an Iterable is what we are after if we want to use the method __iter__(), which is what we need in order to do things like for o in myobject:. However, an Iterable does not include the method __getitem__(), which is what we need in order to do things like myobject[i]. So we'll need to use a different ABC.
On down the collections.abc menu of abstract base classes, we see that a Sequence is the simplest ABC to offer the functionality we require. And - would you look at that - we get Iterable functionality as a mixin method - which means we don't have to define it ourselves - for free! We also get __contains__, __reversed__, index, and count. Which, if you think about it, are all things that should be included in any indexed object. If you had forgotten to include them, users of your code (including, potentially, yourself!) might get pretty annoyed (I know I would).
However, there is a second ABC that also offers this combination of functionality (iterable, and accessible by []): a Mapping. Which one do we want to use?
We recall that the requirement is to be able to access the object by index (like a list or a tuple), i.e. not by key (like a dict). Therefore, we select Sequence instead of Mapping.
Sidebar: It's important to note that a Sequence is read-only (as is a Mapping), so it will not allow us to do things like myobject[i] = value, or random.shuffle(myobject). If we want to be able do things like that, we need to continue down the menu of ABCs and use a MutableSequence (or a MutableMapping), which will require implementing several additional methods.
Example Code
Now we are able to make our class. We define it, and have it inherit from Sequence.
from collections.abc import Sequence
class MyClass(Sequence):
pass
If we try to use it, the interpreter will tell us which methods we need to implement before it can be used (note that the methods are also listed on the Python docs page):
>>> myobject = MyClass()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class MyClass with abstract methods __getitem__, __len__
This tells us that if we go ahead and implement __getitem__ and __len__, we'll be able to use our new class. We might do it like this in Python 3:
from collections.abc import Sequence
class MyClass(Sequence):
def __init__(self,L):
self.L = L
super().__init__()
def __getitem__(self, i):
return self.L[i]
def __len__(self):
return len(self.L)
# Let's test it:
myobject = MyClass([1,2,3])
try:
for idx,_ in enumerate(myobject):
print(myobject[idx])
except Exception:
print("Gah! No good!")
raise
# No Errors!
It works!
Implement both __iter__() and __getitem__() et alia methods.