I'm using Python 3 typing feature for better autocomplete.
Many times I have functions that return key/value (dictionary) with specific keys. super simple example:
def get_info(name):
name_first_letter = name[0]
return {'my_name': name, 'first_letter': name_first_letter}
I want to add type hinting to this function to tell others who use this function what to expect.
I can do something like:
class NameInfo(object):
def __init__(self, name, first_letter):
self.name = name
self.first_letter = first_letter
and then change the function signature to:
def get_info(name) -> NameInfo:
But it requires too much code for each dictionary.
What is the best practice in that case?
As pointed out by Blckknght, you and Stanislav Ivanov in the comments, you can use NamedTuple:
from typing import NamedTuple
class NameInfo(NamedTuple):
name: str
first_letter: str
def get_info(name: str) -> NameInfo:
return NameInfo(name=name, first_letter=name[0])
Starting from Python 3.8 you can use TypedDict which is more similar to what you want:
from typing import TypedDict
class NameInfo(TypedDict):
name: str
first_letter: str
def get_info(name: str) -> NameInfo:
return {'name': name, 'first_letter': name[0]}
Related
This is my first question here, very exciting.
I have a class with a bunch of values attached to it. I would like to get a specific value using a classmethod, or return a list of values. However I just don't seem to be able to get it right.
class StrEnum(str, Enum):
def __str__(self) -> str:
return str.__str__(self)
class BunchOfCreatures(StrEnum):
SPIDERS: str = "Spiders"
BEETLES: str = "Beetles"
HORSES: str = "Horses"
#classmethod
def get_that_string(cls, BunchOfCreatures: BunchOfCreatures) -> List[str]:
return [item.value for item in BunchOfCreatures]
What I want to be able to do it say BunchOfCreatures.get_that_string(HORSES), or even a list that includes multiple things, such as HORSES, BEETLES, and have a list containing all the associated string values.
The reason I don't go straight for BunchOfCreatures.HORSES is that I sometimes need to give a list of values to return or compare.
Very grateful for your help.
Welcome to StackOverflow!!!! Here's a way to return any sublist of the creatures. It also has a branch to return ALL if no creatures are specified:
from enum import Enum
class StrEnum(str, Enum):
def __str__(self) -> str:
return str.__str__(self)
class BunchOfCreatures(StrEnum):
SPIDERS: str = "Spiders"
BEETLES: str = "Beetles"
HORSES: str = "Horses"
#classmethod
def list_creatures(cls, creatures=None):
if creatures==None:
return list(map(lambda c: c.value, cls))
return [c.value for c in creatures]
print(BunchOfCreatures.list_creatures())
print(BunchOfCreatures.list_creatures([BunchOfCreatures.HORSES]))
Note that you do need to append the name of the class (BunchOfCreatures.HORSES) in order to pass something meaningful as an argument. Passing simply HORSES will result in BackTrace (HORSES hasn't been defined outside of the class method!)
I'd like to create a config dataclass in order to simplify whitelisting of and access to specific environment variables (typing os.environ['VAR_NAME'] is tedious relative to config.VAR_NAME). I therefore need to ignore unused environment variables in my dataclass's __init__ function, but I don't know how to extract the default __init__ in order to wrap it with, e.g., a function that also includes *_ as one of the arguments.
import os
from dataclasses import dataclass
#dataclass
class Config:
VAR_NAME_1: str
VAR_NAME_2: str
config = Config(**os.environ)
Running this gives me TypeError: __init__() got an unexpected keyword argument 'SOME_DEFAULT_ENV_VAR'.
Cleaning the argument list before passing it to the constructor is probably the best way to go about it. I'd advice against writing your own __init__ function though, since the dataclass' __init__ does a couple of other convenient things that you'll lose by overriding it.
Also, since the argument-cleaning logic is very tightly bound to the behavior of the class and returns an instance, it might make sense to put it into a classmethod:
from dataclasses import dataclass
import inspect
#dataclass
class Config:
var_1: str
var_2: str
#classmethod
def from_dict(cls, env):
return cls(**{
k: v for k, v in env.items()
if k in inspect.signature(cls).parameters
})
# usage:
params = {'var_1': 'a', 'var_2': 'b', 'var_3': 'c'}
c = Config.from_dict(params) # works without raising a TypeError
print(c)
# prints: Config(var_1='a', var_2='b')
I would just provide an explicit __init__ instead of using the autogenerated one. The body of the loop only sets recognized value, ignoring unexpected ones.
Note that this won't complain about missing values without defaults until later, though.
#dataclass(init=False)
class Config:
VAR_NAME_1: str
VAR_NAME_2: str
def __init__(self, **kwargs):
names = set([f.name for f in dataclasses.fields(self)])
for k, v in kwargs.items():
if k in names:
setattr(self, k, v)
Alternatively, you can pass a filtered environment to the default Config.__init__.
field_names = set(f.name for f in dataclasses.fields(Config))
c = Config(**{k:v for k,v in os.environ.items() if k in field_names})
I used a combination of both answers; setattr can be a performance killer. Naturally, if the dictionary won't have some records in the dataclass, you'll need to set field defaults for them.
from __future__ import annotations
from dataclasses import field, fields, dataclass
#dataclass()
class Record:
name: str
address: str
zip: str = field(default=None) # won't fail if dictionary doesn't have a zip key
#classmethod
def create_from_dict(cls, dict_) -> Record:
class_fields = {f.name for f in fields(cls)}
return Record(**{k: v for k, v in dict_.items() if k in class_fields})
Using the dacite python library to populate a dataclass using a dictionary of values ignores extra arguments / values present in the dictionary (along with all the other benefits the library provides).
from dataclasses import dataclass
from dacite import from_dict
#dataclass
class User:
name: str
age: int
is_active: bool
data = {
'name': 'John',
'age': 30,
'is_active': True,
"extra_1": 1000,
"extra_2": "some value"
}
user = from_dict(data_class=User, data=data)
print(user)
# prints the following: User(name='John', age=30, is_active=True)
I did this based on previous answers:
import functools
import inspect
#functools.cache
def get_dataclass_parameters(cls: type):
return inspect.signature(cls).parameters
def instantiate_dataclass_from_dict(cls: type, dic: dict):
parameters = get_dataclass_parameters(cls)
dic = {k: v for k, v in dic.items() if k in parameters}
return cls(**dic)
Since inspect.signature(cls).parameters takes much more time than the actual instantiation / initialization, I use functools.cache to cache the result for each class.
I'm trying to define a couple of dataclasses and an abstract class that manipulates those classes. Eventually, the my_class_handler types could be dealing with say: json, xml or sqlite files as concrete instance types.
Can someone please explain to me what this message means?
<bound method my_class_handler.class_name of <__main__.my_class_handler object at 0x000001A55FB96580>>
Here's the source code that generates the error for me.
from abc import ABC, abstractmethod
from dataclasses import dataclass
from typing import List
#dataclass
class column:
name: str
heading: str
#dataclass
class my_class:
class_name: str
class_description: str
columns: List[column]
class iclass_handler(ABC):
#abstractmethod
def class_name(self) -> str:
pass
#abstractmethod
def class_name(self, value: str):
pass
class my_class_handler(iclass_handler):
obj: my_class
def __init__(self):
self.obj = my_class("test-class", "", None)
def class_name(self) -> str:
return self.obj.class_names
def class_name(self, value: str):
if (value != self.obj.class_name):
self.obj.class_name = value
if __name__ == '__main__':
handler = my_class_handler()
print(handler.class_name)
If this is not the proper way of doing this, please point me in the direction where I might learn the proper way.
Thanks for your time,
Python does not allow overloading like Java, so remove methods that overlap.
#khelwood pointed out the answer to the original question. Thanks
As for the #property approach, I tried that and was having nothing but problems and couldn't find any useful examples of inherited properties so I just rewrote the function to take an additional parameter:
# I'm working from memory here but I believe this is the jist...
def class_name(self, new_value: str = None) -> str:
if (new_value is None)
return self.obj.class_name
if (isinstance(new_value, str)):
if (new_value != self.obj.class_name):
self.obj.class_name = new_value
return None
Anyhow, I have since refactored and have completely removed the whole class_name() method as a result of a redesign that dropped the whole concept of data-handlers.
Thanks again for the comments.
Coming from a C# background and knowing its generic type approaches I'm now trying to implement something similar in Python. I need to serialize and de-serialize classes in a special string format, so I created the following two base classes, the first for single entity serialization and the second one for list serialization of that entity type.
from typing import Any, TypeVar, List, cast, Type, Generic, NewType
import re
T = TypeVar('T')
class Serializable(Generic[T]):
def to_str(self) -> str:
raise NotImplementedError
#classmethod
def from_str(cls, str: str):
raise NotImplementedError
class SerializableList(List[Serializable[T]]):
def __init__(self):
self.separator: str = "\n"
#classmethod
def from_str(cls, str: str):
list = cls()
for match in re.finditer(list.separator, str):
list.append(T().from_str(match)) # <-- PROBLEM: HOW TO INIT A GENERIC ENTITY ???
# list.append(Serializable[T].from_str(match)) <-- Uses base class (NotImplemented) instead of derived class
return list
def to_str(self) -> str:
str = ""
for e in self:
str = str + f"{e.to_str()}{self.separator}"
return str
Then I can derive from those classes and have to implement to_str and from_str. Please see the marker <-- PROBLEM". I have no idea how I can init a new entity of the currently used type for the list. How do we do this in the Python way?
As #user2357112supportsMonica says in the comments, typing.Generic is pretty much only there for static analysis, and has essentially no effect at runtime under nearly all circumstances. From the look of your code, it looks like what you're doing might be better suited to Abstract Base Classes (documentation here, tutorial here), which can be easily combined with Generic.
A class that has ABCMeta as its metaclass is marked as an Abstract Base Class (ABC). A subclass of an ABC cannot be instantiated unless all methods in the ABC marked with the #abstractmethod decorator have been overridden. In my suggested code below, I've explicitly added the ABCMeta metaclass to your Serializable class, and implicitly added it to your SerializableList class by having it inherit from collections.UserList instead of typing.List. (collections.UserList already has ABCMeta as its metaclass.)
Using ABCs, you could define some interfaces like this (you won't be able to instantiate these because of the abstract methods):
### ABSTRACT INTERFACES ###
from abc import ABCMeta, abstractmethod
from typing import Any, TypeVar, Type, Generic
from collections import UserList
import re
T = TypeVar('T')
class AbstractSerializable(metaclass=ABCMeta):
#abstractmethod
def to_str(self) -> str: ...
#classmethod
#abstractmethod
def from_str(cls: Type[T], string: str) -> T: ...
S = TypeVar('S', bound=AbstractSerializable)
class AbstractSerializableList(UserList[S]):
separator = '\n'
#classmethod
#property
#abstractmethod
def element_cls(cls) -> Type[S]: ...
#classmethod
def from_str(cls, string: str):
new_list = cls()
for match in re.finditer(cls.separator, string):
new_list.append(cls.element_cls.from_str(match))
return new_list
def to_str(self) -> str:
return self.separator.join(e.to_str() for e in self)
You could then provide some concrete implementations like this:
class ConcreteSerializable(AbstractSerializable):
def to_str(self) -> str:
# put your actual implementation here
#classmethod
def from_str(cls: Type[T], string: str) -> T:
# put your actual implementation here
class ConcreteSerializableList(AbstractSerializableList[ConcreteSerializable]:
# this overrides the abstract classmethod-property in the base class
element_cls = ConcreteSerializable
(By the way — I changed several of your variable names — str, list, etc — as they were shadowing builtin types and/or functions. This can often lead to annoying bugs, and even if it doesn't, is quite confusing for other people reading your code! I also cleaned up your to_str method, which can be simplified to a one-liner, and moved your separator variable to be a class variable, since it appears to be the same for all class instances and does not appear to ever be altered.)
For now I found a dirty solution - this is to add a Type (constructor) parameter of the list entries like so:
class SerializableList(List[Serializable[T]]):
# This one
# |
# v
def __init__(self, separator: str = "\n", entity_class: Type = None):
self.separator = separator
self.entity_class = entity_class
#classmethod
def from_str(cls, str: str):
list = cls()
for match in re.finditer(list.separator, str):
list.append(list.entity_class.from_str(match))
return list
I wonder if there is a cleaner way to get the correct [T] type constructor from List[T] since it is already provided there?
I have a function that receives a class and returns an instance of that class. A simple example:
def instantiate(class_: Type[enum.Enum], param: str) -> enum.Enum:
return class_(param)
The return value is the same as the type of the parameter class_, so if class_ is MyEnum, the return value will be of type MyEnum.
Is there some way to declare that?
There sure is! This should work perfectly for your use case.
from enum import Enum
from typing import TypeVar, Type
E = TypeVar('E', bound=Enum)
def instantiate(class_: Type[E], param: str) -> E:
return class_(param)
And to just do a quick test of it to confirm that type hinting is working as intended:
class MyEnum(Enum):
ONE = '1'
TWO = '2'
# I hover over 'e' and PyCharm is able to infer it's of type MyEnum.
e = instantiate(MyEnum, '1')
print(e)
# MyEnum.ONE
Note: As mentioned in comments, in Python 3.9, some typing constructs like Type, List, and Dict are deprecated, as you can just use the builtin types. So the above annotation could be defined like type[E] if you have a newer Python version.
That is pretty standard. You would have found this in the documentation pretty top level:
from typing import Type
class MyClass:
def __init__(self, param: str) -> None:
self.param = param
def instantiate(cls: Type[MyClass], param: str) -> MyClass:
return cls(param)