Develop Reference

Pydantic - Turn attribute immutable when defined as non-None for the first time

I have the following model:
from pydantic import BaseModel
class User(BaseModel):
user_id: Optional[int] = None
name: str
The user_id may be initially unknown (None), but when it is set to a non-None value then it should be immutable afterwards.
Examples:
user1 = User(name="John")
user1.user_id = 1 # user_id is set to a non-None value for the first time
# from now on, user_id must be immutable
user1.user_id = 10 # should raise a ValidationError
user2 = User(user_id=None, name="John")
user2.user_id = 2 # user_id is set to a non-None value for the first time
# from now on, user_id must be immutable
user2.user_id = 20 # should raise a ValidationError
user2.user_id = None # should raise a ValidationError
user3 = User(user_id=None, name="John")
user3.user_id = None # that's ok, the value is still None
user3.user_id = 3 # user_id is now set to a non-None value
# from now on, user_id must be immutable
user3.user_id = 30 # should raise a ValidationError
user4 = User(user_id=4, name="John")
# from now on, user_id must be immutable
user4.user_id = 40 # should raise a ValidationError
This solution requires the user_id to be initialized and thus would make it impossible to assign user1.user_id = 1.

I agree with #chepner that you should probably re-think your algorithm. It seems strange that you even have the possibility of re-assignment, if you know that this is not supposed to happen. In addition, you could just simply check before assigning. But it is hard to tell, what may be a better course of action without additional context.
To the matter at hand.
Pydantic model validation is stateless by design, as far as I know. All validators are class methods and there is no built-in way to have assignment validation depend on the state of the model instance being validated.
That being said, you can always construct a workaround using standard Python "dunder" magic, without getting too much in the way of Pydantic-specifics. Attribute assignment is done via __setattr__, even in the case of Pydantic models. We can hook into that method minimally and do our check there.
To be as much in line with the "Pydantic way", we should not raise a ValidationError there immediately. Instead we can simply assign a unique object that is guaranteed to be picked up by a validator down the line, so that we get a "clean" validation error.
Here is how this can be implemented:
from typing import Any, ClassVar
from pydantic import BaseModel, validator
class User(BaseModel):
_USER_ID_SENTINEL: ClassVar[object] = object()
user_id: int | None = None
name: str
class Config:
validate_assignment = True
def __setattr__(self, key: str, value: Any) -> None:
if key == "user_id" and self.user_id is not None:
value = self.__class__._USER_ID_SENTINEL
super().__setattr__(key, value)
#validator("user_id", pre=True)
def ensure_no_reassignment(cls, v: Any) -> Any:
if v is cls._USER_ID_SENTINEL:
raise ValueError("Re-assignment of user_id not allowed")
return v
Here is a full working test case:
from unittest import TestCase, main
from typing import Any, ClassVar
from pydantic import BaseModel, ValidationError, validator
class User(BaseModel):
... # see above
class Test(TestCase):
def test_default_delayed_assignment(self) -> None:
user = User(name="John")
user.user_id = 1
with self.assertRaises(ValidationError):
user.user_id = 10
with self.assertRaises(ValidationError):
user.user_id = None
def test_explicit_none_delayed_assignment(self) -> None:
user = User(user_id=None, name="John")
user.user_id = 2
with self.assertRaises(ValidationError):
user.user_id = 20
with self.assertRaises(ValidationError):
user.user_id = None
def test_delayed_assignment_none_first(self) -> None:
user = User(user_id=None, name="John")
user.user_id = None
user.user_id = 3
with self.assertRaises(ValidationError):
user.user_id = 30
with self.assertRaises(ValidationError):
user.user_id = None
def test_init_assignment(self) -> None:
user = User(user_id=4, name="John")
with self.assertRaises(ValidationError):
user.user_id = 40
def test_normal_functionality(self) -> None:
with self.assertRaises(ValidationError):
User(**{"name": object()})
with self.assertRaises(ValidationError):
User.parse_obj({"name": "John", "user_id": object()})
user = User(name="John")
user.name = "Alice"
self.assertDictEqual({"user_id": None, "name": "Alice"}, user.dict())
if __name__ == '__main__':
main()
The pre=True in the #validator is necessary to allow us to give a useful error message. We could omit the custom validator entirely. Then the regular int validator will pick the object up and raise an error, but that error will be misleading, indicating a wrong type.
The validate_assignment = True setting in the Config is obviously necessary for a validator to even be called after initialization.
Also, if you wanted to have even more information in the error message, you could (for example) use a special class instead of object for the sentinel and substitute an instance of that class in __setattr__ containing the pre-existing value of user_id and the value trying to be assigned. Then you could pick that up in the validator method and include that information in the error message. I don't know if that is useful though.
Note that (as always) there is no real immutability because in Python there is always a way to mutate any attribute of any object. However, this solution is as "real" as the Pydantic concept of immutability.

Pydantic Recursive Models

I am refencing the answer on this other stackoverflow post on using the Typing library Literal to specify a unique array to strings to validate the data with Pydantic but I am running into a problem of calling another class recursively.
This is what my code looks like:
from pydantic import BaseModel, PydanticValueError, ValidationError, validator
from typing import Literal,Optional
ACTION_TYPE_MAPPING = Literal["read", "write", "release"]
OBJECT_TYPE_MAPPING = Literal["multiStateValue", "multiStateInput", "multiStateOutput",
"analogValue", "analogInput", "analogOutput",
"binaryValue", "binaryInput", "binaryOutput"]
BOOLEAN_ACTION_MAPPING = Literal["active", "inactive"]
# MAIN MODEL
class BacnetRequestModel(BaseModel):
action_type: ACTION_TYPE_MAPPING
object_type: OBJECT_TYPE_MAPPING
object_instance: int
value: Optional[ValueModel(object_type)] <---- MESSED UP HERE, how to call ValueModel?
class ValueModel(BaseModel):
multiStateValue: Optional[int]
multiStateInput: Optional[int]
multiStateOutput: Optional[int]
analogValue: Optional[int]
analogInput: Optional[int]
analogOutput: Optional[int]
binaryValue: Optional[BOOLEAN_ACTION_MAPPING]
binaryInput: Optional[BOOLEAN_ACTION_MAPPING]
binaryOutput: Optional[BOOLEAN_ACTION_MAPPING]
test = BacnetRequestModel(action_type="write",
object_type="binaryOutput",
object_instance="3",
value = "active"
)
How do I call the class ValueModel based on the objectType that was inputted to the function where in this case it was binaryOutput that should only accept a value of BOOLEAN_ACTION_MAPPING. Any tips help not a lot of wisdom here...
Traceback is:
value = Optional[ValueModel(object_type)]
NameError: name 'ValueModel' is not defined

How to validate maximum size of a Pydantic object with extra fields

I have a Pydantic object and while I want to allow extra fields not described in the schema, I want to enforce through validation the maximum size of the entire object. Suppose I have this definition:
from pydantic import Extra, BaseModel
class MyObject(BaseModel):
x:int = 0
class Config:
extra = Extra.allow
I want MyObject(x=1, extra="value") to succeed but MyObject(x=1, extra="a"*1000) to throw a ValidationError.

Validation on the entire object can be enforced with #root_validator.
Assuming the maximum object size is measured as the size of its JSON serialization, the solution may look like this:
from pydantic import Extra, BaseModel, root_validator
MAX_SIZE = 1000
class MyObject(BaseModel):
x:int = 0
#root_validator(pre=True)
def check_max_size(cls, values):
if len(json.dumps(values)) > MAX_SIZE:
raise ValueError('Object is too large')
return values
class Config:
extra = Extra.allow
MyObject(x=1, extra="a"*1000) then raises
pydantic.error_wrappers.ValidationError: 1 validation error for MyObject
__root__
Object is too large (type=value_error)

Graphene input error for Pydantic models with discriminator while generating Input object schema

I am using pydantic validations for my requirements and it uses discriminator. I am writing GraphQL APIs and want to convert those pydantic models into graphene input objects.
Below is my code.
from graphene_pydantic import PydanticInputObjectType, PydanticObjectType
import graphene
from typing import Literal, Union
from pydantic import BaseModel, Field
class Cat(BaseModel):
pet_type: Literal['cat']
meows: int
class Dog(BaseModel):
pet_type: Literal['dog']
barks: float
class Lizard(BaseModel):
pet_type: Literal['reptile', 'lizard']
scales: bool
class Model(BaseModel):
pet: Union[Cat, Dog, Lizard] = Field(..., discriminator='pet_type')
n: int
print(Model(pet={'pet_type': 'dog', 'barks': 3.14, 'eats': 'biscuit'}, n=1))
class Input(PydanticInputObjectType):
class Meta:
model = Model
# exclude specified fields
exclude_fields = ("id",)
class Output(PydanticObjectType):
class Meta:
model = Model
# exclude specified fields
exclude_fields = ("id",)
class CreateAnimal(graphene.Mutation):
class Arguments:
input = Input()
output = Output
#staticmethod
def mutate(parent, info, input):
print(input)
# save model here
return input
class Mutation(graphene.ObjectType):
createPerson = CreateAnimal.Field()
schema = graphene.Schema(mutation=Mutation)
print(schema)
I tried by commenting on the discriminator code and it's working fine but I need those validations for graphql also. If I run that code it's throwing the below error.
File "\AppData\Local\Programs\Python\Python310\lib\site-packages\graphql\type\definition.py", line 1338, in fields raise TypeError(f"{self.name} fields cannot be resolved. {error}")
TypeError: Input fields cannot be resolved. The input field type must be a GraphQL input type.
Can someone help me with this?
I am using graphene-pydantic for this.

Hey i havent tested your code but i think this what you need:
class CreateAnimal(graphene.Mutation):
class Arguments:
input = graphene.Argument(Input)
output = graphene.Field(Output)

Pydantic validations for extra fields that not defined in schema

I am using pydantic for schema validations and I would like to throw an error when any extra field is added to a schema that isn't defined.
from typing import Literal, Union
from pydantic import BaseModel, Field, ValidationError
class Cat(BaseModel):
pet_type: Literal['cat']
meows: int
class Dog(BaseModel):
pet_type: Literal['dog']
barks: float
class Lizard(BaseModel):
pet_type: Literal['reptile', 'lizard']
scales: bool
class Model(BaseModel):
pet: Union[Cat, Dog, Lizard] = Field(..., discriminator='pet_type')
n: int
print(Model(pet={'pet_type': 'dog', 'barks': 3.14, 'eats': 'biscuit'}, n=1))
""" try:
Model(pet={'pet_type': 'dog'}, n=1)
except ValidationError as e:
print(e) """
In the above code, I have added the eats field which is not defined. The pydantic validations are applied and the extra values that I defined are removed in response. I wanna throw an error saying eats is not allowed for Dog or something like that. Is there any way to achieve that?
And is there any chance that we can provide the input directly instead of the pet object?
print(Model({'pet_type': 'dog', 'barks': 3.14, 'eats': 'biscuit', n=1})). I tried without descriminator but those specific validations are missing related to pet_type. Can someone guide me how to achive either one of that?

You can use the extra field in the Config class to forbid extra attributes during model initialisation (by default, additional attributes will be ignored).
For example:
from pydantic import BaseModel, Extra
class Pet(BaseModel):
name: str
class Config:
extra = Extra.forbid
data = {
"name": "some name",
"some_extra_field": "some value",
}
my_pet = Pet.parse_obj(data) # <- effectively the same as Pet(**pet_data)
will raise a VaidationError:
ValidationError: 1 validation error for Pet
some_extra_field
extra fields not permitted (type=value_error.extra)
Works as well when the model is "nested", e.g.:
class PetModel(BaseModel):
my_pet: Pet
n: int
pet_data = {
"my_pet": {"name": "Some Name", "invalid_field": "some value"},
"n": 5,
}
pet_model = PetModel.parse_obj(pet_data)
# Effectively the same as
# pet_model = PetModel(my_pet={"name": "Some Name", "invalid_field": "some value"}, n=5)
will raise:
ValidationError: 1 validation error for PetModel
my_pet -> invalid_field
extra fields not permitted (type=value_error.extra)

Pydantic is made to validate your input with the schema. In your case, you want to remove one of its validation feature.
I think you should create a new class that inherit from BaseModel
class ModifiedBaseModel(BaseModel):
def __init__(__pydantic_self__, **data: Any) -> None:
registered, not_registered = __pydantic_self__.filter_data(data)
super().__init__(**registered)
for k, v in not_registered.items():
__pydantic_self__.__dict__[k] = v
#classmethod
def filter_data(cls, data):
registered_attr = {}
not_registered_attr = {}
annots = cls.__annotations__
for k, v in data.items():
if k in annots:
registered_attr[k] = v
else:
not_registered_attr[k] = v
return registered_attr, not_registered_attr
then create your validation classes
class Cat(ModifiedBaseModel):
pet_type: Literal['cat']
meows: int
now you can create a new Cat without worries about undefined attribute. Like this
my_cat = Cat(pet_type='cat', meows=3, name='blacky', age=3)
2nd question, to put the input directly from dict you can use double asterisk **
Dog(**my_dog_data_in_dict)
or
Dog(**{'pet_type': 'dog', 'barks': 3.14, 'eats': 'biscuit', n=1})