Previously I used the marshmallow library with the Flask. Some time ago I have tried FastAPI with Pydantic. At first glance pydantic seems similar to masrhmallow but on closer inspection they differ. And for me the main difference between them is post_load methods which are from marshmallow. I can't find any analogs for it in pydantic.
post_load is decorator for post-processing methods. Using it I can handle return object on my own, can do whatever I want:
class ProductSchema(Schema):
alias = fields.Str()
category = fields.Str()
brand = fields.Str()
#post_load
def check_alias(self, params, **kwargs):
"""One of the fields must be filled"""
if not any([params.get('alias'), params.get('category'), params.get('brand')]):
raise ValidationError('No alias provided', field='alias')
return params
Besides it used not only for validation. Code example is just for visual understanding, do not analyze it, I have just invented it.
So my question is:
is there any analog for post_load in pydantic?
It is not obvious but pydantic's validator returns value of the field. So there are two ways to handle post_load conversions: validator and
root_validator.
validator gets the field value as argument and returns its value.
root_validator is the same but manipulates with the whole object.
from pydantic import validator, root_validator
class PaymentStatusSchema(BaseModel):
order_id: str = Param(..., title="Order id in the shop")
order_number: str = Param(..., title="Order number in chronological order")
status: int = Param(..., title="Payment status")
#validator("status")
def convert_status(cls, status):
return "active" if status == 1 else "inactive"
#root_validator
def check_order_id(cls, values):
"""Check order id"""
if not values.get('orderNumber') and not values.get('mdOrder'):
raise HTTPException(status_code=400, detail='No order data provided')
return values
By default pydantic runs validators as post-processing methods. For pre-processing you should use validators with pre argument:
#root_validator(pre=True)
def check_order_id(cls, values):
"""Check order id"""
# some code here
return values
Yes, you can use Pydantic's #validator decorator to do pre-load, post-load, model validating etc.
Here is a Post load example
from pydantic import validator
class Person(BaseModel):
first_name: str
second_name: str
#validator("first_name")
def make_it_formal(cls, first_name):
return f"Mr. {first_name.capitalize()}"
p = Person(first_name="egvo", second_name="Example")
p.first_name
Out: Mr. Egvo
Alternatively, you can also override __init__ and post-process the instance there:
from pydantic import BaseModel
class ProductSchema(BaseModel):
alias: str
category: str
brand: str
def __init__(self, *args, **kwargs):
# Do Pydantic validation
super().__init__(*args, **kwargs)
# Do things after Pydantic validation
if not any([self.alias, self.category, self.brand]):
raise ValueError("No alias provided")
Though this happens outside of Pydantic's validation.
Related
I find myself writing tests for methods for two classes, one of which has str fields that it retrieves from another BaseModel class:
class Person(BaseModel):
firstname: Optional[str] = settings.get_settings().firstname
lastname: Optional[str] = settings.get_settings().lastname
Now, when I try to create the fixture I get this problem
#test file
#pytest.fixture
def person_class():
person = Person(
firstname="Marco",
lastname="Marche"
)
return person()
Error:
pydantic.error_wrappers.ValidationError: 31 validation errors for Settings
Settings is a BaseSettings class.
class Settings(BaseSettings):
firstname: str,
lastname: str,
address: str,
telephone: str
#lru_cache
def get_settings() -> Settings:
return Settings()
How can I solve it by creating a "fake" Person class with fields entered by me?
I'm assuming that you have a typo in your Settings definition and there should be no commas in the firstname, lastname, and address lines.
The error you're getting is most likely because your Settings class requires a value for firstname, lastname, and the other fields but you didn't provide any.
I think the easiest solution would be to give it some fake values in the instantiation in your get_settings() function, e.g.:
#lru_cache
def get_settings() -> Settings:
return Settings(
firstname="fake firstname",
lastname="fake lastname",
address="fake address",
telephone="fake telephone",
)
Pydantic does provide the construct() method on the Models which allows to create models without validation, however, I don't think this would work in your situation because you are getting the default values for Person from Settings which, in a case like yours, is still being evaluated.
I'm a little new to tinkering with class inheritance in python, particularly when it comes down to using class attributes. In this case I am using a class attribute to change an argument in pydantic's Field() function. This wouldn't be too hard to do if my class contained it's own constructor, however, my class User1 is inheriting this from pydantic's BaseModel.
The idea is that I would like to be able to change the class attribute prior to creating the instance.
Please see some example code below:
from pydantic import Basemodel, Field
class User1(BaseModel):
_set_ge = None # create class attribute
item: float = Field(..., ge=_set_ge)
# avoid overriding BaseModel's __init__
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
User1._set_ge = 0 # setting the class attribute to a new value
instance = User1(item=-1)
print(instance) # item=-1.0
When creating the instance using instance = User1(item=-1) I would expect a validation error to be thrown, but it instead passes validation and simply returns the item value.
If I had my own constructor there would be little issue in changing the _set_ge, but as User1 inheriting this constructor from BaseModel, things are a little more complicated.
The eventual aim is to add this class to a fastapi endpoint as follows:
from fastapi import Fastapi
from schemas import User1
class NewUser1(User1):
pass
NewUser1._set_ge = 0
#app.post("/")
def endpoint(request: NewUser1):
return User1.item
To reduce code duplication, I aimed to use this method to easily change Field() arguments. If there is a better way, I'd be glad to consider that too.
This question is quite closely related to this unanswered one.
In the end, the #validator proposal by #hernán-alarcón is probably the best way to do this. For example:
from pydantic import Basemodel, Field, NumberNotGeError
from typing import ClassVar
class User(BaseModel):
_set_ge = ClassVar[float] # added the ClassVar typing to make clearer, but the underscore should be sufficient
item: float = Field(...)
#validator('item')
def limits(cls, v):
limit_number = cls._set_ge
if v >= limit_number:
return v
else:
raise NumberNotGeError(limit_value=limit_number)
class User1(User)
_set_ge = 0 # setting the class attribute to a new value
instance = User1(item=-1) # raises the error
Problem in a nutshell
I am having issues with the hypothesis build strategy and custom pydantic data types (no values are returned when invoking the build strategy on my custom data type.
Problem in more detail
Given the following pydantic custom type, which just validates if a value is a timezone.
import pytz
from pydantic import StrictStr
TIMEZONES = pytz.common_timezones_set
class CountryTimeZone(StrictStr):
"""Validate a country timezone."""
#classmethod
def __get_validators__(cls):
yield from super().__get_validators__()
yield cls.validate_timezone
#classmethod
def validate_timezone(cls, v):
breakpoint()
if v not in TIMEZONES:
raise ValueError(f"{v} is not a valid country timezone")
return v
#classmethod
def __modify_schema__(cls, field_schema):
field_schema.update(examples=TIMEZONES)
When I attempt to use this in some schema...
from pydantic import BaseModel
class Foo(BaseModel):
bar: CountryTimeZone
and subsequently try to build an example in a test, using the pydantic hypothesis plugin like.
from hypothesis import given
from hypothesis import strategies as st
#given(st.builds(Foo))
def test_something_interesting(schema) -> None:
# Some assertions
...
schema.bar is always "".
Questions
Is there something missing from this implementation, meaning that values like "Asia/Krasnoyarsk" aren't being generated? From the documentation, examples like PaymentCardNumber and EmailStr build as expected.
Even when using StrictStr by itself, the resulting value is also an empty string. I tried to inherit from str but still no luck.
Came across the same problem today. Seems like the wording in the hypothesis plugin docs give the wrong impression. Pydantic has written hypothesis integrations for their custom types, not that hypothesis supports custom pydantic types out of the box.
Here is a full example of creating a custom class, assigning it a test strategy and using it in a pydantic model.
import re
from hypothesis import given, strategies as st
from pydantic import BaseModel
CAPITAL_WORD = r"^[A-Z][a-z]+"
CAPITAL_WORD_REG = re.compile(CAPITAL_WORD)
class MustBeCapitalWord(str):
"""Custom class that validates the string is a single of only letters
starting with a capital case letter."""
#classmethod
def __get_validators__(cls):
yield cls.validate
#classmethod
def __modify_schema__(cls, field_schema):
# optional stuff, updates the schema if you choose to export the
# pydantic schema
field_schema.UPDATE(
pattern=CAPITAL_WORD,
examples=["Hello", "World"],
)
#classmethod
def validate(cls, v):
if not isinstance(v, str):
raise TypeError("string required")
if not v:
raise ValueError("No capital letter found")
elif CAPITAL_WORD_REG.match(v) is None:
raise ValueError("Input is not a valid word starting with capital letter")
return cls(v)
def __repr__(self):
return f"MustBeCapitalWord({super().__repr__()})"
# register a strategy for our custom type
st.register_type_strategy(
MustBeCapitalWord,
st.from_regex(CAPITAL_WORD, fullmatch=True),
)
# use our custom type in a pydantic model
class Model(BaseModel):
word: MustBeCapitalWord
# test it all
#given(st.builds(Model))
def test_model(instance):
assert instance.word[0].isupper()
This is continue to this question.
I have added a model to get query params to pydantic model
class QueryParams(BaseModel):
x: str = Field(description="query x")
y: str = Field(description="query y")
z: str = Field(description="query z")
#app.get("/test-query-url/{test_id}")
async def get_by_query(test_id: int, query_params: QueryParams = Depends()):
print(test_id)
print(query_params.dict(by_alias=True))
return True
it is working as expected but description(added in model) is not reflecting in swagger ui
But if same model is used for request body, then description is shown in swagger
Am I missing anything to get the description for QueryParams(model) in swagger ui?
This is not possible with Pydantic models
The workaround to get the desired result is to have a custom dependency class (or function) rather than the Pydantic model
from fastapi import Depends, FastAPI, Query
app = FastAPI()
class CustomQueryParams:
def __init__(
self,
foo: str = Query(..., description="Cool Description for foo"),
bar: str = Query(..., description="Cool Description for bar"),
):
self.foo = foo
self.bar = bar
#app.get("/test-query/")
async def get_by_query(params: CustomQueryParams = Depends()):
return params
Thus, you will have the doc as,
References
Validate GET parameters in FastAPI--(FastAPI GitHub) It seems like there is less interest in extending the Pydantic model to validate the GET parameters
Classes as Dependencies--(FastAPI Doc)
This worked for me
from fastapi import Depends, FastAPI, Query
#app.post("/route")
def some_api(
self,
query_param_1: float = Query(None, description="description goes here", ),
query_param_2: float = Query(None, description="Param 2 does xyz"),
):
return "hello world"
The accepted answer refers to the use of custom dependencies using FastAPI classes as dependencies to define the query parameters in bulk and while I think it works great, I feel the using dataclasses would be better in this case and reduces the code duplication as the __init__ will be generated automatically.
Normal class as dependency
class QueryParams:
def __init__(self,
x: Query(
None, description="Arg1", example=10),
y: Query(
None, description="Arg2", example=20)
):
self.x = x
self.y = y
While for lesser number of query params it would be just fine to do it this way but if the number of args is large as it was for me for one of my api endpoints, this quickly becomes cumbersome.
Using dataclass as a dependency
#dataclass
class QueryParams:
x: Query(None, description="Arg1", example=10)
y: Query(None, description="Arg2", example=20)
Plus you will also have added goodies of dataclasses if you need more complex functioanlities.
Long gone are the days of creating marshmallow schemas identical to my models. I found this excellent answer that explained how I could auto generate schemas from my SQA models using a simple decorator, so I implemented it and replaced the deprecated ModelSchema for the newer SQLAlchemyAutoSchema:
def add_schema(cls):
class Schema(SQLAlchemyAutoSchema):
class Meta:
model = cls
cls.Schema = Schema
return cls
This worked great... until I bumped into a model with a bloody Enum.
The error: Object of type MyEnum is not JSON serializable
I searched online and I found this useful answer.
But I'd like to implement it as part of the decorator so that it is generated automatically as well. In other words, I'd like to automatically overwrite all Enums in my model with EnumField(TheEnum, by_value=True) when generating the schema using the add_schema decorator; that way I won't have to overwrite all the fields manually.
What would be the best way to do this?
I have found that the support for enum types that was initially suggested only works if OneOf is the only validation class that exists in field_details. I added in some argument parsing (in a rudimentary way by looking for choices after stringifying the results _repr_args() from OneOf) to check the validation classes to hopefully make this implementation more universally usable:
def add_schema(cls):
class Schema(ma.SQLAlchemyAutoSchema):
class Meta:
model = cls
fields = Schema._declared_fields
# support for enum types
for field_name, field_details in fields.items():
if len(field_details.validate) > 0:
check = str(field_details.validate[0]._repr_args)
if check.__contains__("choices") :
enum_list = field_details.validate[0].choices
enum_dict = {enum_list[i]: enum_list[i] for i in range(0, len(enum_list))}
enum_clone = Enum(field_name.capitalize(), enum_dict)
fields[field_name] = EnumField(enum_clone, by_value=True, validate=validate.OneOf(enum_list))
cls.Schema = Schema
return cls
Thank you jgozal for the initial solution, as I really needed this lead for my current project.
This is my solution:
from marshmallow import validate
from marshmallow_sqlalchemy import SQLAlchemyAutoSchema
from marshmallow_enum import EnumField
from enum import Enum
def add_schema(cls):
class Schema(SQLAlchemyAutoSchema):
class Meta:
model = cls
fields = Schema._declared_fields
# support for enum types
for field_name, field_details in fields.items():
if len(field_details.validate) > 0:
enum_list = field_details.validate[0].choices
enum_dict = {enum_list[i]: enum_list[i] for i in range(0, len(enum_list))}
enum_clone = Enum(field_name.capitalize(), enum_dict)
fields[field_name] = EnumField(enum_clone, by_value=True, validate=validate.OneOf(enum_list))
cls.Schema = Schema
return cls
The idea is to iterate over the fields in the Schema and find those that have validation (usually enums). From there we can extract a list of choices which can then be used to build an enum from scratch. Finally we overwrite the schema field with a new EnumField.
By all means, feel free to improve the answer!