I am trying to make a function that takes a pydantic BaseModel as an input to run another function. I need to unpack the BaseModel into kwargs. I tried doing this:
def run_routing_from_job(job):
return run_routing(
job.inputs.input_file,
**job.inputs.config.dict()
)
where job is of the format
class Job(BaseModel):
client_info: ClientInfo # Another BaseModel
inputs: RoutingJobInputs # Another BaseModel
uid: UUID = Field(default_factory=uuid4)
status: str = "job_queued"
result: int = None
However, doing .dict() parses all of the items recursively into a dictionary format. I want to keep the client_info and inputs as a BaseModel class, not convert it into a dictionary.
I could make a way to do it, but I can't find a clean way to do it.
I worked it out, just replace .dict() with __dict__
def run_routing_from_job(job):
return run_routing(
job.inputs.input_file,
**job.inputs.config.__dict__
)
Related
I have a json key with whitespace in it:
My_dict = {"my key": 1}
I want to create a Model to model it:
from pydantic import BaseModel
class MyModel(BaseModel):
mykey: int
# my key isn't a legit variable name
# my_key is, but like mykey - it doesn't catch the correct key from the json
MyModel(**my_dict)
This doesn't work.
I tried playing with the BaseModel.Config, but didn't get anywhere. Didn't see anything on the docs as well.
Is this possible?
I can use a workaround: Go over the json, replace all key's whitespaces into underscores, and then use pydantic but I would love to not use this...
Yes, it's possible by using Field's aliases:
from pydantic import BaseModel, Field
class MyModel(BaseModel):
mykey: int = Field(alias='my key')
class Config:
allow_population_by_field_name = True
print(MyModel(**{"my key": 1}))
print(MyModel(**{"mykey": 1}))
I have setup a simple class in Python with a member posted that defaults to utcnow() time as a string. However when I create an instance of the class then create another instance a few minutes later they both have the exact same posted time.
If in the route I have created I pass in posted=str(datetime.utcnow()) it works fine so Python is picking up the computer datetime correctly. I have also tested this on a Heroku app I have running and I get the same problem.
The time it puts in is the time the app was first run as if it's being treated as a static value.
import uuid
from datetime import datetime
from typing import Dict
from dataclasses import dataclass, field
from models.model import Model
#dataclass
class Dummy(Model):
collection: str = field(init=False, default='dummy')
text: str
posted: str = field(default=str(datetime.utcnow()))
_id: str = field(default_factory=lambda: uuid.uuid4().hex)
def json(self) -> Dict:
return {
"text": self.text,
"posted": self.posted,
"_id": self._id
}
You could use default_factory instead in case you want a dynamically default value.
#dataclass
class Dummy(Model):
...
posted: str = field(default_factory=lambda: str(datetime.utcnow()))
Previously I used the marshmallow library with the Flask. Some time ago I have tried FastAPI with Pydantic. At first glance pydantic seems similar to masrhmallow but on closer inspection they differ. And for me the main difference between them is post_load methods which are from marshmallow. I can't find any analogs for it in pydantic.
post_load is decorator for post-processing methods. Using it I can handle return object on my own, can do whatever I want:
class ProductSchema(Schema):
alias = fields.Str()
category = fields.Str()
brand = fields.Str()
#post_load
def check_alias(self, params, **kwargs):
"""One of the fields must be filled"""
if not any([params.get('alias'), params.get('category'), params.get('brand')]):
raise ValidationError('No alias provided', field='alias')
return params
Besides it used not only for validation. Code example is just for visual understanding, do not analyze it, I have just invented it.
So my question is:
is there any analog for post_load in pydantic?
It is not obvious but pydantic's validator returns value of the field. So there are two ways to handle post_load conversions: validator and
root_validator.
validator gets the field value as argument and returns its value.
root_validator is the same but manipulates with the whole object.
from pydantic import validator, root_validator
class PaymentStatusSchema(BaseModel):
order_id: str = Param(..., title="Order id in the shop")
order_number: str = Param(..., title="Order number in chronological order")
status: int = Param(..., title="Payment status")
#validator("status")
def convert_status(cls, status):
return "active" if status == 1 else "inactive"
#root_validator
def check_order_id(cls, values):
"""Check order id"""
if not values.get('orderNumber') and not values.get('mdOrder'):
raise HTTPException(status_code=400, detail='No order data provided')
return values
By default pydantic runs validators as post-processing methods. For pre-processing you should use validators with pre argument:
#root_validator(pre=True)
def check_order_id(cls, values):
"""Check order id"""
# some code here
return values
Yes, you can use Pydantic's #validator decorator to do pre-load, post-load, model validating etc.
Here is a Post load example
from pydantic import validator
class Person(BaseModel):
first_name: str
second_name: str
#validator("first_name")
def make_it_formal(cls, first_name):
return f"Mr. {first_name.capitalize()}"
p = Person(first_name="egvo", second_name="Example")
p.first_name
Out: Mr. Egvo
Alternatively, you can also override __init__ and post-process the instance there:
from pydantic import BaseModel
class ProductSchema(BaseModel):
alias: str
category: str
brand: str
def __init__(self, *args, **kwargs):
# Do Pydantic validation
super().__init__(*args, **kwargs)
# Do things after Pydantic validation
if not any([self.alias, self.category, self.brand]):
raise ValueError("No alias provided")
Though this happens outside of Pydantic's validation.
I am starting to learn FastAPI and Pydantic and have a doubt. I have the following subclass of BaseModel
class Product(BaseModel):
image: str
name: str
After saving this model, I want image to store the value /static/ + image so as to create nice hyperlinked REST endpoint. This is possible using __post_init_post_parse__ hook of pydantic dataclass but since FastAPI currently doesn't support it, I was wondering what can be a workaround this.
You could use a custom validator:
>>> from pydantic import BaseModel, validator
>>> class Product(BaseModel):
image: str
name: str
#validator('image')
def static_mage(cls, image):
return '/static/{}'.format(image)
>>> p = Product(image='pic.png', name='product_1')
>>> p
Product(image='/static/pic.png', name='product_1')
I have a list of objects that I need to jsonify. I've looked at the flask jsonify docs, but I'm just not getting it.
My class has several inst-vars, each of which is a string: gene_id, gene_symbol, p_value. What do I need to do to make this serializable as JSON?
My naive code:
jsonify(eqtls = my_list_of_eqtls)
Results in:
TypeError: <__main__.EqtlByGene object at 0x1073ff790> is not JSON serializable
Presumably I have to tell jsonify how to serialize an EqtlByGene, but I can't find an example that shows how to serialize an instance of a class.
I've been trying to follow some of the suggestions show below to create my own JSONEncoder subclass. My code is now:
class EqtlByGene(Resource):
def __init__(self, gene_id, gene_symbol, p_value):
self.gene_id = gene_id
self.gene_symbol = gene_symbol
self.p_value = p_value
class EqtlJSONEncoder(JSONEncoder):
def default(self, obj):
if isinstance(obj, EqtlByGene):
return {
'gene_id' : obj.gene_id,
'gene_symbol' : obj.gene_symbol,
'p_value' : obj.p_value
}
return super(EqtlJSONEncoder, self).default(obj)
class EqtlByGeneList(Resource):
def get(self):
eqtl1 = EqtlByGene(1, 'EGFR', 0.1)
eqtl2 = EqtlByGene(2, 'PTEN', 0.2)
eqtls = [eqtl1, eqtl2]
return jsonify(eqtls_by_gene = eqtls)
api.add_resource(EqtlByGeneList, '/eqtl/eqtlsbygene')
app.json_encoder(EqtlJSONEncoder)
if __name__ == '__main__':
app.run(debug=True)
When I try to reach it via curl, I get:
TypeError(repr(o) + " is not JSON serializable")
Give your EqltByGene an extra method that returns a dictionary:
class EqltByGene(object):
#
def serialize(self):
return {
'gene_id': self.gene_id,
'gene_symbol': self.gene_symbol,
'p_value': self.p_value,
}
then use a list comprehension to turn your list of objects into a list of serializable values:
jsonify(eqtls=[e.serialize() for e in my_list_of_eqtls])
The alternative would be to write a hook function for the json.dumps() function, but since your structure is rather simple, the list comprehension and custom method approach is simpler.
You can also be really adventurous and subclass flask.json.JSONEncoder; give it a default() method that turns your EqltByGene() instances into a serializable value:
from flask.json import JSONEncoder
class MyJSONEncoder(JSONEncoder):
def default(self, obj):
if isinstance(obj, EqltByGene):
return {
'gene_id': obj.gene_id,
'gene_symbol': obj.gene_symbol,
'p_value': obj.p_value,
}
return super(MyJSONEncoder, self).default(obj)
and assign this to the app.json_encoder attribute:
app = Flask(__name__)
app.json_encoder = MyJSONEncoder
and just pass in your list directly to jsonify():
return jsonify(my_list_of_eqtls)
You could also look at the Marshmallow project for a more full-fledged and flexible project for serializing and de-serializing objects to Python primitives that easily fit JSON and other such formats; e.g.:
from marshmallow import Schema, fields
class EqltByGeneSchema(Schema):
gene_id = fields.Integer()
gene_symbol = fields.String()
p_value = fields.Float()
and then use
jsonify(eqlts=EqltByGeneSchema().dump(my_list_of_eqtls, many=True)
to produce JSON output. The same schema can be used to validate incoming JSON data and (with the appropriate extra methods), used to produce EqltByGene instances again.
If you look at the docs for the json module, it mentions that you can subclass JSONEncoder to override its default method and add support for types there. That would be the most generic way to handle it if you're going to be serializing multiple different structures that might contain your objects.
If you want to use jsonify, it's probably easier to convert your objects to simple types ahead of time (e.g. by defining your own method on the class, as Martijn suggests).