How do we define a Pydantic Class Type for the following nested dictionary structure (with complicated strings as keys):
{ "inputs" : {
"simpleStringName": <Simple_DateTime_Value>
"simpleStringName2": {
"Complicated|String|Name|with|Pipe|Characters" : <Simple-Int-Value>,
"Complicated|String|Name2|with|Pipe|Characters" : <Simple-Int-Value2>
}
}
}
Pydantic needs to enforce the entire structure. That is simply this will not work:
class XYZ(BaseModel):
inputs : Dict()
I want to enforce requests input format using Pydantic with fastAPI.
Something like this is probably what you're after
import datetime
from typing import Union, Dict
from pydantic import BaseModel
class XYZ(BaseModel):
inputs: Dict[str, Union[datetime.datetime, Dict[str, int]]]
>>> XYZ(**{
... "inputs" : {
... "simpleStringName": datetime.datetime.utcnow(),
... "simpleStringName2": {
... "Complicated|String|Name|with|Pipe|Characters" : 3,
... "Complicated|String|Name2|with|Pipe|Characters" : 4
... }
... }
... })
XYZ(inputs={'simpleStringName': datetime.datetime(2021, 11, 11, 16, 34, 21, 906156), 'simpleStringName2': {'Complicated|String|Name|with|Pipe|Characters': 3, 'Complicated|String|Name2|with|Pipe|Characters': 4}})
If you want to explicitly define the strings, you can use Pydantic's Constrained Types (here constr with a regex, but many options are available)
https://pydantic-docs.helpmanual.io/usage/types/#constrained-types
class XYZ(pydantic.BaseModel):
inputs: Dict[
pydantic.constr(regex="^[a-zA-Z\d]+$"),
Union[
datetime.datetime,
Dict[
pydantic.constr(regex="^[a-zA-Z\d\|]+$"),
int
]
]
]
Related
From a similar question, the goal is to create a model like this Typescript interface:
interface ExpandedModel {
fixed: number;
[key: string]: OtherModel;
}
However the OtherModel needs to be validated, so simply using:
class ExpandedModel(BaseModel):
fixed: int
class Config:
extra = "allow"
Won't be enough. I tried root (pydantic docs):
class VariableKeysModel(BaseModel):
__root__: Dict[str, OtherModel]
But doing something like:
class ExpandedModel(VariableKeysModel):
fixed: int
Is not possible due to:
ValueError: root cannot be mixed with other fields
Would something like #root_validator (example from another answer) be helpful in this case?
Thankfully, Python is not TypeScript. As mentioned in the comments here as well, an object is generally not a dictionary and dynamic attributes are considered bad form in almost all cases.
You can of course still set attributes dynamically, but they will for example never be recognized by a static type checker like Mypy or your IDE. This means you will not get auto-suggestions for those dynamic fields. Only attributes that are statically defined within the namespace of the class are considered members of that class.
That being said, you can abuse the extra config option to allow arbitrary fields to by dynamically added to the model, while at the same time enforcing all corresponding values to be of a specific type via a root_validator.
from typing import Any
from pydantic import BaseModel, root_validator
class Foo(BaseModel):
a: int
class Bar(BaseModel):
b: str
#root_validator
def validate_foo(cls, values: dict[str, Any]) -> dict[str, Any]:
for name, value in values.items():
if name in cls.__fields__:
continue # ignore statically defined fields here
values[name] = Foo.parse_obj(value)
return values
class Config:
extra = "allow"
Demo:
if __name__ == "__main__":
from pydantic import ValidationError
bar = Bar.parse_obj({
"b": "xyz",
"foo1": {"a": 1},
"foo2": Foo(a=2),
})
print(bar.json(indent=4))
try:
Bar.parse_obj({
"b": "xyz",
"foo": {"a": "string"},
})
except ValidationError as err:
print(err.json(indent=4))
try:
Bar.parse_obj({
"b": "xyz",
"foo": {"not_a_foo_field": 1},
})
except ValidationError as err:
print(err.json(indent=4))
Output:
{
"b": "xyz",
"foo2": {
"a": 2
},
"foo1": {
"a": 1
}
}
[
{
"loc": [
"__root__",
"a"
],
"msg": "value is not a valid integer",
"type": "type_error.integer"
}
]
[
{
"loc": [
"__root__",
"a"
],
"msg": "field required",
"type": "value_error.missing"
}
]
A better approach IMO is to just put the dynamic name-object-pairs into a dictionary. For example, you could define a separate field foos: dict[str, Foo] on the Bar model and get automatic validation out of the box that way.
Or you ditch the outer base model altogether for that specific case and just handle the data as a native dictionary with Foo values and parse them all via the Foo model.
I have two models:
from pydantic import BaseModel
class Nested(BaseModel):
id: int
title: str
class Model(BaseModel):
id: int
nested_id: int
nested: Nested
Model references Nested.
I make a query with a JOIN to my database and get something like this response:
data = {'id': 5, 'nested_id': 1, 'id_1': 1, 'title': 'хлеб'}
I would like to parse this response and assign the right fields to the Nested model,
Are there any methods in BaseModel, with which I can parse data?
Of course I can work with dict which I already have. But I want to do it in a method of BaseModel .
I use parse_as_obj(List[Model], data).
Something like this?
from pydantic import BaseModel, root_validator
DataDict = dict[str, object]
class Nested(BaseModel):
id: int
title: str
class Model(BaseModel):
id: int
nested_id: int
nested: Nested
#root_validator(pre=True)
def parse_flat_nested_data(cls, values: DataDict) -> DataDict:
if "nested" not in values:
values["nested"] = {
"id": values.get("id_1"),
"title": values.get("title"),
}
return values
if __name__ == "__main__":
data = {"id": 5, "nested_id": 1, "id_1": 1, "title": "хлеб"}
instance = Model.parse_obj(data)
print(instance)
Output: id=5 nested_id=1 nested=Nested(id=1, title='хлеб')
See documentation on validators for more.
In python 3, how can I deserialize an object structure from json?
Example json:
{ 'name': 'foo',
'some_object': { 'field1': 'bar', 'field2' : '0' },
'some_list_of_objects': [
{ 'field1': 'bar1', 'field2' : '1' },
{ 'field1': 'bar2', 'field2' : '2' },
{ 'field1': 'bar3', 'field2' : '3' },
]
}
Here's my python code:
import json
class A:
name: str
some_object: B
some_list_of_objects: list(C)
def __init__(self, file_name):
with open(file_name, "r") as json_file:
self.__dict__ = json.load(json_file)
class B:
field1: int
field2: str
class C:
field1: int
field2: str
How to force some_object to be of type B and some_list_of_objects to be of type list of C?
As you're using Python 3, I would suggest using dataclasses to model your classes. This should improve your overall code quality and also eliminate the need to explicltly declare an __init__ constructor method for your class, for example.
If you're on board with using a third-party library, I'd suggest looking into an efficient JSON serialization library like the dataclass-wizard that performs implicit type conversion - for example, string to annotated int as below. Note that I'm using StringIO here, which is a file-like object containing a JSON string to de-serialize into a nested class model.
Note: the following approach should work in Python 3.7+.
from __future__ import annotations
from dataclasses import dataclass
from io import StringIO
from dataclass_wizard import JSONWizard
json_data = StringIO("""
{ "name": "foo",
"some_object": { "field1": "bar", "field2" : "0" },
"some_list_of_objects": [
{ "field1": "bar1", "field2" : "1" },
{ "field1": "bar2", "field2" : "2" },
{ "field1": "bar3", "field2" : "3" }
]
}
""")
#dataclass
class A(JSONWizard):
name: str
some_object: B
some_list_of_objects: list[C]
#dataclass
class B:
field1: str
field2: int
#dataclass
class C:
field1: str
field2: int
a = A.from_json(json_data.read())
print(f'{a!r}') # alternatively: print(repr(a))
Output
A(name='foo', some_object=B(field1='bar', field2=0), some_list_of_objects=[C(field1='bar1', field2=1), C(field1='bar2', field2=2), C(field1='bar3', field2=3)])
Loading from a JSON file
As per the suggestions in this post, I would discourage overriding the constructor method to pass the name of a JSON file to load the data from. Instead, I would suggest creating a helper class method as below, that can be invoked like A.from_json_file('file.json') if desired.
#classmethod
def from_json_file(cls, file_name: str):
"""Deserialize json file contents into an A object."""
with open(file_name, 'r') as json_file:
return cls.from_dict(json.load(json_file))
Suggestions
Note that variable annotations (or annotations in general) are subscripted using square brackets [] rather than parentheses as appears in the original version above.
some_list_of_objects: list(C)
In the above solution, I've instead changed that to:
some_list_of_objects: list[C]
This works because using subscripted values in standard collections was introduced in PEP 585. However, using the from __future__ import annotations statement introduced to Python 3.7+ effectively converts all annotations to forward-declared string values, so that new-style annotations that normally only would work in Python 3.10, can also be ported over to Python 3.7+ as well.
One other change I made, was in regards to swapping out the order of declared class annotations. For example, note the below:
class B:
field1: int
field2: str
However, note the corresponding field in the JSON data, that would be deserialized to a B object:
'some_object': { 'field1': 'bar', 'field2' : '0' },
In the above implementation, I've swapped out the field annotations in such cases, so class B for instance is declared as:
class B:
field1: str
field2: int
I would like pydantic to choose the model to use for parsing the input dependent on the input value. Is this possible?
MVCE
I have a pydantic model which looks similar to this one:
from typing import List, Literal
from pydantic import BaseModel
class Animal(BaseModel):
name: str
type: Literal["mamal", "bird"]
class Bird(Animal):
max_eggs: int
class Mamal(Animal):
max_offspring: int
class Config(BaseModel):
animals: List[Animal]
cfg = Config.parse_obj(
{
"animals": [
{"name": "eagle", "type": "bird", "max_eggs": 3},
{"name": "Human", "type": "mamal", "max_offspring": 3},
]
}
)
print(cfg.json(indent=4))
gives
{
"animals": [
{
"name": "eagle",
"type": "bird"
<-- missing max_offspring, as "Animal" was used instead of Bird
},
{
"name": "Human",
"type": "mamal"
<-- missing max_offspring, as "Animal" was used instead of Mamal
}
]
}
I know that I could set Config.extra="allow" in Animal, but that is not what I want. I would like pydantic to see that a dictionary with 'type': 'mamal' should use the Mamal model to parse.
Is this possible?
You could add concrete literals to every child class to differentiate and put them in Union from more to less specific order. Like so:
class Animal(BaseModel):
name: str
type: str
class Bird(Animal):
type: Literal["bird"]
max_eggs: int
class Mamal(Animal):
type: Literal["mamal"]
max_offspring: int
class Config(BaseModel):
animals: List[Union[Bird, Mamal, Animal]] # From more specific to less
I want to knok if is possible to parameterize the type in a custom User defined types.
My working example is:
from pydantic import BaseModel
from typing import TypeVar, Dict, Union, Optional
ListSchemaType = TypeVar("ListSchemaType", bound=BaseModel)
GenericPagination = Dict[str, Union[Optional[int], List[ListSchemaType]]]
What i need to do is call GenericPagination with a parameter.
Like this:
pagination: GenericPagination(schemas.Posts)
My data is structured in this form:
"count": 2,
"limit": 10,
"page": 1,
"results": [{
"name": "Foo",
"url_base": "https://www.bar.cl",
"development_stage": "FOO",
"system_status": "BAR",
"id": 1
}]
}
After some research i found then answer here.
My final working case looks:
ListSchema = TypeVar("ListSchema", bound=BaseModel)
Pagination = Dict[str, Union[Optional[int], List[ListSchema]]]
Then i can just use it like this:
pagination: Optional[Pagination[schemas.Post]]