it seems impossible to set a regex constraint with a __root__ field like this one:
class Cars(BaseModel):
__root__: Dict[str, CarData]
so, i've resorted to doing it at the endpoint:
#app.post("/cars")
async def get_cars(cars: Cars = Body(...)):
x = cars.json()
y = json.loads(x)
keys = list(y.keys())
try:
if any([re.search(r'^\d+$', i) is None for i in keys]):
raise ValidationError
except ValidationError as ex:
return 'wrong type'
return 'works'
this works well in that i get wrong type returned if i dont use a digit in the request body.
but i'd like to return something similar to what pydantic returns but with a custom message:
{
"detail": [
{
"loc": [
"body",
"__root__",
],
"msg": "hey there, you can only use digits!",
"type": "type_error.???"
}
]
}
You can pass your own error string by using raise ValidationError("Wrong data type").
Hope it helps.
if it helps anyone, here is how i validated a dynamic field:
class Cars(BaseModel):
__root__: Dict[str, CarData]
#pydantic.root_validator(pre=True)
#classmethod
def car_id_is_digit(cls, fields):
car_ids = list(list(fields.values())[0].keys())
print(car_ids)
if any([bool(re.search(r'^\d+$', car_id)) == False for car_id in car_ids]):
raise ValueError("car_id must be a string that is a digit.")
else:
return fields
since a regular field validator requires a field name as an argument, i used the root_validator which validates all fields - and does not require that argument.
all this, because __root__ cannot be referenced in the regular field validator, it seems.
however, this means you can only have __root__ fields - and they will all be under the same validation rules...not sure how to added more fields with this.
Related
I need to point Pydantic to a different attribute when serializing an ORM model. alias= doesn't seem to work as expected. In the example below I have an ORM object with both id and uuid attributes. I want to serialize uuid as id.
The API response should be:
{
"id": "12345678-1234-5678-1234-567812345678",
"foo": "bar"
}
Full example:
from uuid import UUID
from fastapi import FastAPI
from pydantic import BaseModel, Field
from dataclasses import dataclass
class ApiSchema(BaseModel):
class Config:
orm_mode = True
uuid: UUID = Field(alias='id')
foo: str | None = None
#dataclass
class ORMModel:
id: int
uuid: UUID
foo: str = 'bar'
app = FastAPI()
#app.get("/")
def endpoint() -> ApiSchema:
t = ORMModel(id=1, uuid=UUID('12345678123456781234567812345678'), foo='bar')
return t
This raises
File fastapi/routing.py", line 141, in serialize_response
raise ValidationError(errors, field.type_)
pydantic.error_wrappers.ValidationError: 1 validation error for ApiSchema
response -> id
value is not a valid uuid (type=type_error.uuid)
The marshmallow equivalent of what I'm trying to achieve would be this:
import marshmallow as ma
class ApiSchema(ma.Schema):
id = ma.fields.UUID(attribute='uuid')
foo = ma.fields.Str()
You misunderstand how aliases work. An alias on a field takes priority (over the actual field name) when the fields are populated. That means, during initialization, the class will look for the alias of a field in the data it is supposed to parse.
The way you defined ApiSchema, the field uuid has the alias id. Therefore, when you are parsing an instance of ORMModel (happens in FastAPI behind the scenes via ApiSchema.from_orm), the ApiSchema class will look for an attribute named id on that ORMModel object to populate the uuid field.
Since your ORMModel actually has an attribute named id (with the value 1 in your example), its value is taken to be assigned to the uuid field of ApiSchema.
Obviously, the integer 1 is not a UUID object and can not be coerced into one, so you get that validation error telling you that the value it found for id is not a valid UUID.
Here is the problem boiled down to the essentials:
from uuid import UUID
from pydantic import BaseModel, Field, ValidationError
class ApiSchema(BaseModel):
uuid: UUID = Field(alias='id')
foo: str | None = None
try:
ApiSchema.parse_obj({"uuid": "this is ignored", "foo": "bar"})
except ValidationError as exc:
print(exc.json(indent=2))
try:
ApiSchema.parse_obj({"id": 1, "foo": "bar"})
except ValidationError as exc:
print(exc.json(indent=2))
The output of the first attempt:
[
{
"loc": [
"id"
],
"msg": "field required",
"type": "value_error.missing"
}
]
The second:
[
{
"loc": [
"id"
],
"msg": "value is not a valid uuid",
"type": "type_error.uuid"
}
]
I think you want it the other way around. I assume that your actual goal is to have a field named id on your ApiSchema model (and have that appear in your API endpoint) and alias it with uuid, so that it takes the value of the ORMModel.uuid attribute during initialization:
from uuid import UUID
from pydantic import BaseModel, Field
class ApiSchema(BaseModel):
id: UUID = Field(alias="uuid")
foo: str | None = None
obj = ApiSchema.parse_obj(
{
"id": "this is ignored",
"uuid": UUID("12345678123456781234567812345678"),
"foo": "bar",
}
)
print(obj.json(indent=2))
The output:
{
"id": "12345678-1234-5678-1234-567812345678",
"foo": "bar"
}
To fix your FastAPI example, you would therefore probably do this:
from dataclasses import dataclass
from uuid import UUID
from fastapi import FastAPI
from pydantic import BaseModel, Field
class ApiSchema(BaseModel):
id: UUID = Field(alias="uuid")
foo: str | None = None
class Config:
orm_mode = True
#dataclass
class ORMModel:
id: int
uuid: UUID
foo: str = "bar"
app = FastAPI()
#app.get("/", response_model=ApiSchema, response_model_by_alias=False)
def endpoint() -> ORMModel:
t = ORMModel(id=1, uuid=UUID("12345678123456781234567812345678"), foo="bar")
return t
Side note: Yes, the actual return type of endpoint is ORMModel. The wrapper returned by the decorator then takes that and turns it into an instance of ApiSchema via from_orm.
PS
Forgot the last part to actually get the response you want. You need to set response_model_by_alias=False in the route decorator (it is True by default) for the response to actually use the regular field name instead of the alias. I fixed the last code snipped accordingly. Now the response will be:
{"id":"12345678-1234-5678-1234-567812345678","foo":"bar"}
In the Pydantic BaseModel.json method the by_alias parameter has the value False by default. FastAPI does this differently.
I am using pydantic for schema validations and I would like to throw an error when any extra field is added to a schema that isn't defined.
from typing import Literal, Union
from pydantic import BaseModel, Field, ValidationError
class Cat(BaseModel):
pet_type: Literal['cat']
meows: int
class Dog(BaseModel):
pet_type: Literal['dog']
barks: float
class Lizard(BaseModel):
pet_type: Literal['reptile', 'lizard']
scales: bool
class Model(BaseModel):
pet: Union[Cat, Dog, Lizard] = Field(..., discriminator='pet_type')
n: int
print(Model(pet={'pet_type': 'dog', 'barks': 3.14, 'eats': 'biscuit'}, n=1))
""" try:
Model(pet={'pet_type': 'dog'}, n=1)
except ValidationError as e:
print(e) """
In the above code, I have added the eats field which is not defined. The pydantic validations are applied and the extra values that I defined are removed in response. I wanna throw an error saying eats is not allowed for Dog or something like that. Is there any way to achieve that?
And is there any chance that we can provide the input directly instead of the pet object?
print(Model({'pet_type': 'dog', 'barks': 3.14, 'eats': 'biscuit', n=1})). I tried without descriminator but those specific validations are missing related to pet_type. Can someone guide me how to achive either one of that?
You can use the extra field in the Config class to forbid extra attributes during model initialisation (by default, additional attributes will be ignored).
For example:
from pydantic import BaseModel, Extra
class Pet(BaseModel):
name: str
class Config:
extra = Extra.forbid
data = {
"name": "some name",
"some_extra_field": "some value",
}
my_pet = Pet.parse_obj(data) # <- effectively the same as Pet(**pet_data)
will raise a VaidationError:
ValidationError: 1 validation error for Pet
some_extra_field
extra fields not permitted (type=value_error.extra)
Works as well when the model is "nested", e.g.:
class PetModel(BaseModel):
my_pet: Pet
n: int
pet_data = {
"my_pet": {"name": "Some Name", "invalid_field": "some value"},
"n": 5,
}
pet_model = PetModel.parse_obj(pet_data)
# Effectively the same as
# pet_model = PetModel(my_pet={"name": "Some Name", "invalid_field": "some value"}, n=5)
will raise:
ValidationError: 1 validation error for PetModel
my_pet -> invalid_field
extra fields not permitted (type=value_error.extra)
Pydantic is made to validate your input with the schema. In your case, you want to remove one of its validation feature.
I think you should create a new class that inherit from BaseModel
class ModifiedBaseModel(BaseModel):
def __init__(__pydantic_self__, **data: Any) -> None:
registered, not_registered = __pydantic_self__.filter_data(data)
super().__init__(**registered)
for k, v in not_registered.items():
__pydantic_self__.__dict__[k] = v
#classmethod
def filter_data(cls, data):
registered_attr = {}
not_registered_attr = {}
annots = cls.__annotations__
for k, v in data.items():
if k in annots:
registered_attr[k] = v
else:
not_registered_attr[k] = v
return registered_attr, not_registered_attr
then create your validation classes
class Cat(ModifiedBaseModel):
pet_type: Literal['cat']
meows: int
now you can create a new Cat without worries about undefined attribute. Like this
my_cat = Cat(pet_type='cat', meows=3, name='blacky', age=3)
2nd question, to put the input directly from dict you can use double asterisk **
Dog(**my_dog_data_in_dict)
or
Dog(**{'pet_type': 'dog', 'barks': 3.14, 'eats': 'biscuit', n=1})
def generate_waybill(shipper, consignee, services, profile):
success = True
request = {
"Shipper": shipper,
"Consignee": consignee,
"Services": services
}
client = Client(url)
res = client.service.GenerateWayBill(Request=request, Profile=profile)
print("RESPONSE GENERATE WAYBILL: \n\n", res, res.json())
if not res:
success = False
return res, success
While calling the generate_waybill function i get an execption from zeep >>> 'Invalid enum value 'ProductType.Dutiables'
In the services object it has a key named as ProductType and its Data Type ProductType [Enumerator] and the allowed values for this fields are ProductType.Docs, ProductType.Dutiables.
The services object
services = {
"ProductCode": shipping_options['bluedart']['product_code'],
"ProductType": shipping_options['bluedart']['product_type'],
"PieceCount": return_request_line_items.all().count(),
"ActualWeight": weight,
"CreditReferenceNo": reference_id,
"PickupDate": timezone.now() + timezone.timedelta(days=2),
"PickupTime": '1052',
"RegisterPickup": True,
"IsReversePickup": True
}
In the "ProductType": field I have passed hard coded string value as 'ProductType.Dutiables'
I have also tried creating a calss with Enum type like
from enum import Enum
class ProductType(Enum):
Dutiables = "Dutiables"
and used this as ProductType.Dutiables in the 'ProductType' field.
But nothing seems to work for me. Please help!
zeep.exceptions.Fault: The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://tempuri.org/:Request. The InnerException message was 'Invalid enum value 'ProductType.Dutiables' cannot be deserialized into type 'SAPI.Entities.Enums.AWBGeneration.ProductType'. Ensure that the necessary enum values are present and are marked with EnumMemberAttribute attribute if the type has DataContractAttribute attribute.'. Please see InnerException for more details.
I'm creating a django rest framework application with this structure (assuming imports are correct, so I omit them from the code below.
models.py:
class Door(models.Model):
type = models.CharField(max_length=40)
color = models.CharField(max_length=40)
serializers.py:
class DoorSerializer(serializers.ModelSerializer):
class Meta:
model = Door
fields = ['type', 'color']
views.py:
class DoorViewSet(viewsets.ModelViewSet):
serializer_class = DoorSerializer
queryset = Door.objects.all()
def get_queryset(self, *args, **kwargs):
queryset = Door.objects.all()
parameter = self.request.query_params.get('type', '')
if parameter:
return queryset.filter(type=parameter)
else:
return queryset
So far this behaves as intended, when I make an api call to localhost/Doors it lists all the doors. And when I make an api call to localhost/Doors/?type=big it lists all the doors that have the value "big" in their "type" field.
The addition I would like to make is another parameter check which would return a list of all the unique door types that exist in the database. This can be achieved in the manage.py shell by using: Door.objects.all().values('type').distinct()
My attempt was the following modifications to views.py:
...
parameter = self.request.query.params.get('type', '')
unique = self.request.query.params.get('unique', '')
if parameter:
...
elif unique:
return Door.objects.all().values('type').distinct()
...
My assumption was that this would return the same as Door.objects.all().values('type').distinct() when I make a call to localhost/Doors/?unique=whatever
However I am getting the error: "Got KeyError when attempting to get a value for field color on serializer DoorSerializer.\nThe serializer field might be named incorrectly and not match any attribute or key on the dict instance.\nOriginal exception text was: 'color'."
I assume this means that the serializer expects an object or a list of objects that contains all the fields of the corresponding model.
Is there some way I could circumvent this by fixing the view or should I create a different serializer? In either case, since I've gotten pretty confused with DRF / django differences and it is possible I won't be able to follow abstract instructions, could you provide a code solution that addresses the issue? Also, in the very likely case that my assumption is completely off, could you also explain what is causing the problem? Thank you for your time!
Edit for clarifying the desired result:
Assuming my database has 4 doors which are:
{
"id": 1,
"type": "big",
"color": "blue"
},
{
"id": 2,
"type": "big",
"color": "yellow"
},
{
"id": 3,
"type": "small",
"color": "green"
},
{
"id": 4,
"type": "big",
"color": "red"
},
I would like to make a get request to some url, for instance localhost/Doors/?unique=Yes and have the api return to me the list {"big", "small}
WRITING YOUR OWN VIEW: Short view that returns the list of type. You need to set up a new path here. I'd personally go for this option as the response you expect is way different to what the rest of your view does.
from rest_framework.decorators import api_view
from rest_framework.response import Response
#api_view()
def Unique_door_types(request):
types = Door.objects.values_list('type', flat=True).distinct()
return Response({"types": list(types)})
WITHOUT AN ADDITIONAL VIEW:
No need for additional view or serializer. Override the list method. Note that this is closer to a trick than to a good way of programming.
from rest_framework.response import Response
class DoorViewSet(viewsets.ModelViewSet):
serializer_class = DoorSerializer
def get_queryset(self, *args, **kwargs):
queryset = Door.objects.all()
parameter = self.request.query_params.get('type', '')
if parameter:
return queryset.filter(type=parameter)
else:
return queryset
def list(self, request):
unique = self.request.query_params.get('unique', '')
if unique:
types = Door.objects.values_list('type', flat=True).distinct()
return Response({"types": list(types)})
return super().list()
My suggestion would be to create a separate route like /doors/types/. You do this by adding a method to your DoorViewSet class with a #action decorator. See https://www.django-rest-framework.org/api-guide/viewsets/#marking-extra-actions-for-routing for more details about how to do this.
As per the GraphQL spec https://graphql.github.io/graphql-spec/draft/#sec-Input-Objects
the input object literal or unordered map must not contain any entries with names not defined by a field of this input object type, otherwise an error must be thrown.
Let's say I have a project which has a user called buyer, the schema for which is as follows
type Buyer {
id: ID!
name: String!
address: String!
email: String!
}
Now I can write a graphene schema for it
class BuyerType(DjangoObjectType):
class Meta:
model = Buyer
and make a mutation for this
class BuyerInput(graphene.InputObjectType):
name = graphene.String(required=False, default_value='')
address = graphene.String(required=False, default_value='')
email = graphene.String(required=False, default_value='')
class BuyerMutation(graphene.Mutation):
"""
API to create applicant
"""
class Arguments:
buyer_data = BuyerInput(required=True)
buyer = graphene.Field(BuyerType)
ok = graphene.Boolean()
def mutate(self, info, buyer_data=None):
buyer = Buyer(name=buyer.name, address=buyer.address, email=buyer.email)
buyer.save()
return BuyerMutation(buyer=buyer, ok=True)
and write the resolvers functions and query and mutation class. Pretty basic stuff so far.
And now to create a new buyer, I just call the mutation
mutation {
createBuyer(
name: "New Buyer"
address: "Earth"
email: "abc#example.com"
) {
ok
}
}
But if I pass an additional field called phone
mutation {
createBuyer(
name: "New Buyer"
address: "Earth"
email: "abc#example.com"
phone: 8541345474
) {
ok
}
}
an Unknown field 'phone' error comes up, which is understood.
But I want to make process for frontend devs easier. Rather than mentioning each field in the argument of mutation, they can just pass a dict which contains these three fields and also other arguments that might originate from a form submission, and then read the dict in the backend and only extract the fields that I need, much like it is done with REST APIs.
If this is possible, then how can I implement this?
We could get the class attributes of BuyerInput using the inspect module, and then assemble the arguments dictionary so that it ignores the keys which are not such an attribute. Say, we have the dict containing the parameters of createBuyer stored in the variable kwargs, this could look like the following:
import inspect
members = inspect.getmembers(BuyerInput, lambda a: not(inspect.isroutine(a)))
attributes = [
tup[0]
for tup in members
if not tup[0].startswith("_")
]
kwargs = {
key: value
for key, value in kwargs.items()
if key in attributes
}