Consider the follwoing code illustrating use of the pydantic BaseModel with validation:
from pydantic import BaseModel, validator
class User(BaseModel, frozen=True):
id_key: int
user_id: int
#validator('user_id')
def id_check(cls, v, values):
if v > 2 * values['id_key'] + 1:
raise ValueError('id check failed.')
return v
user_dict = {'user_id': 10, 'id_key': 60}
u = User(**user_dict)
Now, in my application, I don't really want id_key to be a regular, accessible field in model instances like u--its sole purpose is for validating user_id. For my example, is there a way to have access to id_key for validation purposes but not have it be a standard field?
The values argument you have in your id_check function being the internal dict of attributes already validated for your instance, if you need to have id_key only at instantiation time for checking and not after that, you could simply remove it from values.
from pydantic import BaseModel, validator
class User(BaseModel, frozen=True):
id_key: int
user_id: int
#validator('user_id')
def id_check(cls, v, values):
if v > 2 * values['id_key'] + 1:
raise ValueError('id check failed.')
values.pop('id_key')
return v
user_dict = {'user_id': 10, 'id_key': 60}
u = User(**user_dict)
print(u)
# output:
# user_id=10
There is one additional improvement I'd like to suggest for your code: in its present state, as pydantic runs the validations of all the fields before returning the validation errors, if you pass something completely invalid for id_key like "abc" for example, or omit it, it won't be added to values, and the validation of user_id will crash with KeyError: 'id_key', swallowing all the rest of the validation process and returning no sensible message.
user_dict = {'user_id': 10, 'id_key': 'abc'}
u = User(**user_dict)
# output:
# KeyError: 'id_key'
This is not very explicit, and might cause issues with your application if you expect a pydantic ValidationError. You might want to check that id_key is indeed present in values and raise the error cleanly if not.
from pydantic import BaseModel, validator
class User(BaseModel, frozen=True):
id_key: int
user_id: int
#validator('user_id')
def id_check(cls, v, values):
if 'id_key' not in values or v > 2 * values['id_key'] + 1:
raise ValueError('id check failed.')
values.pop('id_key')
return v
user_dict = {'user_id': 10, 'id_key': 'abc'}
u = User(**user_dict)
# output:
# pydantic.error_wrappers.ValidationError: 2 validation errors for User
# id_key
# value is not a valid integer (type=type_error.integer)
# user_id
# id check failed.(type=value_error)
Related
I have the following model:
from pydantic import BaseModel
class User(BaseModel):
user_id: Optional[int] = None
name: str
The user_id may be initially unknown (None), but when it is set to a non-None value then it should be immutable afterwards.
Examples:
user1 = User(name="John")
user1.user_id = 1 # user_id is set to a non-None value for the first time
# from now on, user_id must be immutable
user1.user_id = 10 # should raise a ValidationError
user2 = User(user_id=None, name="John")
user2.user_id = 2 # user_id is set to a non-None value for the first time
# from now on, user_id must be immutable
user2.user_id = 20 # should raise a ValidationError
user2.user_id = None # should raise a ValidationError
user3 = User(user_id=None, name="John")
user3.user_id = None # that's ok, the value is still None
user3.user_id = 3 # user_id is now set to a non-None value
# from now on, user_id must be immutable
user3.user_id = 30 # should raise a ValidationError
user4 = User(user_id=4, name="John")
# from now on, user_id must be immutable
user4.user_id = 40 # should raise a ValidationError
This solution requires the user_id to be initialized and thus would make it impossible to assign user1.user_id = 1.
I agree with #chepner that you should probably re-think your algorithm. It seems strange that you even have the possibility of re-assignment, if you know that this is not supposed to happen. In addition, you could just simply check before assigning. But it is hard to tell, what may be a better course of action without additional context.
To the matter at hand.
Pydantic model validation is stateless by design, as far as I know. All validators are class methods and there is no built-in way to have assignment validation depend on the state of the model instance being validated.
That being said, you can always construct a workaround using standard Python "dunder" magic, without getting too much in the way of Pydantic-specifics. Attribute assignment is done via __setattr__, even in the case of Pydantic models. We can hook into that method minimally and do our check there.
To be as much in line with the "Pydantic way", we should not raise a ValidationError there immediately. Instead we can simply assign a unique object that is guaranteed to be picked up by a validator down the line, so that we get a "clean" validation error.
Here is how this can be implemented:
from typing import Any, ClassVar
from pydantic import BaseModel, validator
class User(BaseModel):
_USER_ID_SENTINEL: ClassVar[object] = object()
user_id: int | None = None
name: str
class Config:
validate_assignment = True
def __setattr__(self, key: str, value: Any) -> None:
if key == "user_id" and self.user_id is not None:
value = self.__class__._USER_ID_SENTINEL
super().__setattr__(key, value)
#validator("user_id", pre=True)
def ensure_no_reassignment(cls, v: Any) -> Any:
if v is cls._USER_ID_SENTINEL:
raise ValueError("Re-assignment of user_id not allowed")
return v
Here is a full working test case:
from unittest import TestCase, main
from typing import Any, ClassVar
from pydantic import BaseModel, ValidationError, validator
class User(BaseModel):
... # see above
class Test(TestCase):
def test_default_delayed_assignment(self) -> None:
user = User(name="John")
user.user_id = 1
with self.assertRaises(ValidationError):
user.user_id = 10
with self.assertRaises(ValidationError):
user.user_id = None
def test_explicit_none_delayed_assignment(self) -> None:
user = User(user_id=None, name="John")
user.user_id = 2
with self.assertRaises(ValidationError):
user.user_id = 20
with self.assertRaises(ValidationError):
user.user_id = None
def test_delayed_assignment_none_first(self) -> None:
user = User(user_id=None, name="John")
user.user_id = None
user.user_id = 3
with self.assertRaises(ValidationError):
user.user_id = 30
with self.assertRaises(ValidationError):
user.user_id = None
def test_init_assignment(self) -> None:
user = User(user_id=4, name="John")
with self.assertRaises(ValidationError):
user.user_id = 40
def test_normal_functionality(self) -> None:
with self.assertRaises(ValidationError):
User(**{"name": object()})
with self.assertRaises(ValidationError):
User.parse_obj({"name": "John", "user_id": object()})
user = User(name="John")
user.name = "Alice"
self.assertDictEqual({"user_id": None, "name": "Alice"}, user.dict())
if __name__ == '__main__':
main()
The pre=True in the #validator is necessary to allow us to give a useful error message. We could omit the custom validator entirely. Then the regular int validator will pick the object up and raise an error, but that error will be misleading, indicating a wrong type.
The validate_assignment = True setting in the Config is obviously necessary for a validator to even be called after initialization.
Also, if you wanted to have even more information in the error message, you could (for example) use a special class instead of object for the sentinel and substitute an instance of that class in __setattr__ containing the pre-existing value of user_id and the value trying to be assigned. Then you could pick that up in the validator method and include that information in the error message. I don't know if that is useful though.
Note that (as always) there is no real immutability because in Python there is always a way to mutate any attribute of any object. However, this solution is as "real" as the Pydantic concept of immutability.
I am using pydantic for schema validations and I would like to throw an error when any extra field is added to a schema that isn't defined.
from typing import Literal, Union
from pydantic import BaseModel, Field, ValidationError
class Cat(BaseModel):
pet_type: Literal['cat']
meows: int
class Dog(BaseModel):
pet_type: Literal['dog']
barks: float
class Lizard(BaseModel):
pet_type: Literal['reptile', 'lizard']
scales: bool
class Model(BaseModel):
pet: Union[Cat, Dog, Lizard] = Field(..., discriminator='pet_type')
n: int
print(Model(pet={'pet_type': 'dog', 'barks': 3.14, 'eats': 'biscuit'}, n=1))
""" try:
Model(pet={'pet_type': 'dog'}, n=1)
except ValidationError as e:
print(e) """
In the above code, I have added the eats field which is not defined. The pydantic validations are applied and the extra values that I defined are removed in response. I wanna throw an error saying eats is not allowed for Dog or something like that. Is there any way to achieve that?
And is there any chance that we can provide the input directly instead of the pet object?
print(Model({'pet_type': 'dog', 'barks': 3.14, 'eats': 'biscuit', n=1})). I tried without descriminator but those specific validations are missing related to pet_type. Can someone guide me how to achive either one of that?
You can use the extra field in the Config class to forbid extra attributes during model initialisation (by default, additional attributes will be ignored).
For example:
from pydantic import BaseModel, Extra
class Pet(BaseModel):
name: str
class Config:
extra = Extra.forbid
data = {
"name": "some name",
"some_extra_field": "some value",
}
my_pet = Pet.parse_obj(data) # <- effectively the same as Pet(**pet_data)
will raise a VaidationError:
ValidationError: 1 validation error for Pet
some_extra_field
extra fields not permitted (type=value_error.extra)
Works as well when the model is "nested", e.g.:
class PetModel(BaseModel):
my_pet: Pet
n: int
pet_data = {
"my_pet": {"name": "Some Name", "invalid_field": "some value"},
"n": 5,
}
pet_model = PetModel.parse_obj(pet_data)
# Effectively the same as
# pet_model = PetModel(my_pet={"name": "Some Name", "invalid_field": "some value"}, n=5)
will raise:
ValidationError: 1 validation error for PetModel
my_pet -> invalid_field
extra fields not permitted (type=value_error.extra)
Pydantic is made to validate your input with the schema. In your case, you want to remove one of its validation feature.
I think you should create a new class that inherit from BaseModel
class ModifiedBaseModel(BaseModel):
def __init__(__pydantic_self__, **data: Any) -> None:
registered, not_registered = __pydantic_self__.filter_data(data)
super().__init__(**registered)
for k, v in not_registered.items():
__pydantic_self__.__dict__[k] = v
#classmethod
def filter_data(cls, data):
registered_attr = {}
not_registered_attr = {}
annots = cls.__annotations__
for k, v in data.items():
if k in annots:
registered_attr[k] = v
else:
not_registered_attr[k] = v
return registered_attr, not_registered_attr
then create your validation classes
class Cat(ModifiedBaseModel):
pet_type: Literal['cat']
meows: int
now you can create a new Cat without worries about undefined attribute. Like this
my_cat = Cat(pet_type='cat', meows=3, name='blacky', age=3)
2nd question, to put the input directly from dict you can use double asterisk **
Dog(**my_dog_data_in_dict)
or
Dog(**{'pet_type': 'dog', 'barks': 3.14, 'eats': 'biscuit', n=1})
I'm trying to make sure one of my objects used is always in a correct state. For this I should validate not only on creation but also on assignment, and also on the sub field assignments. Here is a basic example:
from typing import Optional
from pydantic import BaseModel, root_validator
class SubModel(BaseModel):
class Config:
validate_assignment = True
min: Optional[int]
max: Optional[int]
class TestModel(BaseModel):
class Config:
validate_assignment = True
field_1: Optional[SubModel]
#root_validator
def validate(cls, values):
field = values.get("field_1")
if field and field.min and field.max:
if field.min > field.max:
raise ValueError("error")
return values
If I now call
model = TestModel(field_1=SubModel(min=2, max=1))
or
model = TestModel()
field_1 = SubModel(min=2, max=1)
the validation is triggered and the ValueError is raised, which is fine.
But if I do the following
model = TestModel()
field_1 = SubModel()
field_1.min = 2
field_1.max = 1
no validation is triggered.
I know that I could do the validation on SubModel level but in my case (which is a little bit more complex than the basic code shows) I don't want every object of type SubModel to have min <= max but only the one field used in TestModel. Therefor moving the validator to the SubModel is no option for me.
Does anyone have an idea on how to trigger the validator of TestModel when assigning min and max on field_1?
Thank you in advance!
i wanna validate data in request
i have dictionary (a and b are use cases, 1234 are sub use cases)
d ={'a':[1,2],'b':[3,4]}
and request
#router.post("/documents")
from typing import Literal, List, Optional, Dict
#router.post("/documents")
async def data(usecase: Literal[frozenset(d.keys())] = Form(...))
it works and allowed values only a and b
But i wanna extend validation
#router.post("/documents")
async def data(usecase: Literal[frozenset(d.keys())] = Form(...),
subusecase: THERE I WANNA VALIDATE 1234 VALUES = Form(...)
)
I will be very grateful for the help
I'm not sure i quite understand the structure of the data received by your route.
If you want to validate the route input from a complexe structure like nested dict etc, in my opinion it would be beter to use a pydantic model with a pydantic validation function.
you pass to your route a pydantic model as parameter:
#router.post("/documents")
async def data(use_cases: UseCases):
# do something with your uses_cases
pass
pydantic model example:
from typing import List
from pydantic import BaseModel, validator
class UseCases(BaseModel):
a: List[int]
b: List[int]
#validator('a')
def a_must_containt_something(cls, v):
# add your validations here
return v
#validator('b')
def b_must_containt_something(cls, v):
# add your validations here
return v
I have dict-like object, like:
data = {
# A lot of data here
'json_data_feed':
{'address':
{'name': 'home_sweet_home'}
}
# A lot of data here
}
And i want to create Pydantic model with few fields. Im trying to do this:
class OfferById(pydantic.BaseModel):
short_address: str = pydantic.Field(..., alias='name')
#pydantic.validator('short_address', pre=True)
def validate_short_address(cls, value):
return value['json_data_feed']['address']
And it fails with exception:
Some = OfferById(**data)
File "pydantic/main.py", line 406, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for OfferById
name
field required (type=value_error.missing)
Are there any solution here?
You can achieve this by means of root validator. For example:
class OfferById(BaseModel):
short_address: str = Field(..., alias='name')
#root_validator(pre=True)
def validate_short_address(cls, values):
values['name'] = values['json_data_feed']['address']['name']
return values
print(OfferById(**data))