I am trying to use Marshmallow schema to serialize the python object. Below is the schema I have defined for my data.
from marshmallow import Schema, fields
class User:
def __init__(self, name = None, age = None, is_active = None, details = None):
self.name = name
self.age = age
self.is_active = is_active
self.details = details
class UserSchema(Schema):
name = fields.Str()
age = fields.Int()
is_active = fields.Bool()
details = fields.Dict()
The input will be in dictionary format and all the values will be in string.
user_data = {"name":"xyz", "age":"20", "is_active": 'true',"details":"{'key1':'val1', 'key2':'val2'}"}
When I try to run the below snippet, values of age and is_active got converted into respective datatype but details remains unchanged.
user_schema = UserSchema()
user_dump_data = user_schema.dump(user_data)
print(user_dump_data)
Output:
{'name': 'xyz', 'is_active': True, 'details': "{'key1':'val1', 'key2':'val2'}", 'age': 20}
I need to serialize the input data into respective datatype I defined in my schema. Is there anything I am doing wrongly? Can anyone guide me how to acheive this using Marshmallow?
I am using
python 3.6
marshmallow 3.5.1
Edit
The above input data is fetched from HBase. By default HBase stores all its values as bytes and return as bytes. Below is the format I get from HBase
{b'name': b'xyz', b'age': b'20', b'is_active': b'true', b'details': b"{'key1':'val1', 'key2':'val2'}"}
Then I decode this dictionary and pass it to my UserSchema to serialize it to be used in web API.
You're confusing serializing (dumping) and deserializing (loading).
Dumping is going from object form to json-serializable basic python types (using Schema.dump) or json string (using Schema.dumps). Loading is the reverse operation.
Typically, your API loads (and validates) data from the outside world and dumps (without validation) your objects to the outside world.
If your input data is this data and you want to load it into objects, you need to use load, not dump.
user_data = {"name":"xyz", "age":"20", "is_active": 'true',"details":"{'key1':'val1', 'key2':'val2'}"}
user_loaded_data = user_schema.load(user_data)
user = User(**user_loaded_data)
Except if you do so, you'll be caught by another issue. DictField expects data as a dict, not a str. You need to enter
user_data = {"name":"xyz", "age":"20", "is_active": 'true',"details": {'key1':'val1', 'key2':'val2'}}
As Jérôme mentioned you're confusing serializing(dumping) with deserializing(loading). As per your requirement, you should use Schema.load as suggested.
Since, all the input values are expected to be of string type. You can use, pre_load to register a method for pre-processing the data as follows:
from marshmallow import Schema, fields, pre_load
class UserSchema(Schema):
name = fields.Str()
age = fields.Int()
is_active = fields.Bool()
details = fields.Dict()
#pre_load
def pre_process_details(self, data, **kwarg):
data['details'] = eval(data['details'])
return data
user_data = {"name":"xyz", "age":"20", "is_active": 'true',"details":"{'key1':'val1', 'key2':'val2'}"}
user_schema = UserSchema()
user_loaded_data = user_schema.load(user_data)
print(user_loaded_data)
Here, pre_process_details will convert string type to dictionary for correct deserialization.
I try to use marshmallow 2.18.0 on python 3.7 for validating data. I waiting for json {'name': 'foo', 'emailAddress': 'x#x.org'} and load it with schema:
class FooLoad(Schema):
name = fields.Str()
email = fields.Email(data_key='emailAddress', required=True)
I except that data_key on load will return me somesing like {'name': 'foo', 'email': 'x#x.org'}, but i got error in errors field:
schema_load = FooLoad()
after_load = schema_load.load({'name': 'foo', 'emailAddress': 'x#x.org'})
after_load.errors # return {'email': ['Missing data for required field.']}
But according example from marshmallow docs with devDependencies or github issue after_load must contain data like {'name': 'foo', 'email': 'x#x.org'}.
I want to deserialize the incoming date with names differ than schema attribute names (specifying what is required on the date_key), but i got errors when try it. How i can deserialize input data with names, different from schema attribute and declarited in data_key field of this attributes?
data_key was introduced in marshmallow 3.
See changelog entry:
Backwards-incompatible: Add data_key parameter to fields for specifying the key in the input and output data dict. This parameter replaces both load_from and dump_to (#717).
and associated pull-request.
When using marshmallow 2, you must use load_from/dump_to:
class FooLoad(Schema):
name = fields.Str()
email = fields.Email(load_from='emailAddress', dump_to='emailAddress', required=True)
You're using marshmallow 2 but reading the docs for marshmallow 3.
Note that marshmallow 3 contains a bunch of improvements and is in RC state, so if you're starting a project, you could go for marshmallow 3 and save yourself some transition work in the future.
I was experiencing the same phenomenon, trying to parse an API response. It turned out though I needed to drill 1 level deeper into the response, earlier than I was doing.
The response was:
{
"meta": {
"status": 200,
"message": null
},
"response": {
"ownerId": "…",
"otherData": […]
}
}
Then I was calling:
MySchema().load(response.json())
…
class MySchema(Schema):
owner_id = fields.String(data_key='ownerId')
…
Meta:
unknown = INCLUDE
#post_load
def load_my_object(self, data, **kwargs):
inner = data.get('response', data)
return MyObject(**inner)
But really, it should have been:
inner = data.get('response', data)
return MySchema().load(inner)
…
class MySchema(Schema):
owner_id = fields.String(data_key='ownerId')
…
Meta:
unknown = INCLUDE
#post_load
def load_my_object(self, data, **kwargs):
return MyObject(**data)
this code cannot work and give json serializable error
class Bank(peewee.Model): // create Bank table
bank_id = peewee.PrimaryKeyField()
bank_name = peewee.CharField()
account_no = peewee.CharField()
ifc_code = peewee.CharField()
swift_code = peewee.CharField(null = True)
modify_date = peewee.DateTimeField(default=datetime.datetime.now(),formats=['%Y-%m-%d'])/*date in yyyy-mm-dd formate*/
status = peewee.IntegerField(default = 0)
class Meta:
database = db
This answer is very incorrect - please see my answer below (#coleifer).
The default date that you are providing is not a datetime object. Rather it's a string!
modify_date = peewee.DateTimeField(default=datetime.datetime.now().strftime('%Y-%m-%d'))
type(datetime.datetime.now().strftime('%Y-%m-%d')) --> str
You can pass default current datetime object like this:
date = datetime.datetime.now().strftime('%Y-%m-%d')
need_date = datetime.strptime(date, '%Y-%m-%d')
modify_date = peewee.DateTimeField(default=need_date)
or
peewee.DateTimeField(default=datetime.datetime.now)
It looks like non-timezone aware datetimes work fine, so if you're using UTC then you can store datetime.utcnow() as that returns the current UTC date and time with tzinfo None i.e. as a "naive" datetime object.
I found this solution to store and retrieve the timezone aware field as text, however it's not ideal as the datetime object isn't being stored.
from datetime import datetime
from peewee import *
class TimestampTzField(Field):
"""
A timestamp field that supports a timezone by serializing the value
with isoformat.
"""
field_type = "TEXT"
def db_value(self, value: datetime) -> str:
if value:
return value.isoformat()
def python_value(self, value: str) -> str:
if value:
return datetime.fromisoformat(value)
https://compileandrun.com/python-peewee-timezone-aware-datetime/
If you want to store a date, use the DateField. Also, the default needs to be a callable -- in other words, leave OFF the parentheses!
class Bank(peewee.Model): // create Bank table
bank_id = peewee.PrimaryKeyField()
bank_name = peewee.CharField()
account_no = peewee.CharField()
ifc_code = peewee.CharField()
swift_code = peewee.CharField(null = True)
modify_date = peewee.DateField(default=datetime.date.today)
status = peewee.IntegerField(default = 0)
class Meta:
database = db
When it comes time to serialize this as Json, just use a custom json formatter that can handle python datetime.date objects. This is the proper way. You should always store your data using the appropriate format and worry about presentation (and serialization) in another layer.
It is very simple to extend Python's json serializer to handle unsupported types:
def convert_date(o):
if isinstance(o, datetime.date):
return o.__str__()
json.dumps(my_obj, default=convert_date)
This question already has answers here:
How to serialize SqlAlchemy result to JSON?
(37 answers)
Closed 4 years ago.
I'm trying to jsonify a SQLAlchemy result set in Flask/Python.
The Flask mailing list suggested the following method http://librelist.com/browser//flask/2011/2/16/jsonify-sqlalchemy-pagination-collection-result/#04a0754b63387f87e59dda564bde426e :
return jsonify(json_list = qryresult)
However I'm getting the following error back:
TypeError: <flaskext.sqlalchemy.BaseQuery object at 0x102c2df90>
is not JSON serializable
What am I overlooking here?
I have found this question: How to serialize SqlAlchemy result to JSON? which seems very similar however I didn't know whether Flask had some magic to make it easier as the mailing list post suggested.
Edit: for clarification, this is what my model looks like
class Rating(db.Model):
__tablename__ = 'rating'
id = db.Column(db.Integer, primary_key=True)
fullurl = db.Column(db.String())
url = db.Column(db.String())
comments = db.Column(db.Text)
overall = db.Column(db.Integer)
shipping = db.Column(db.Integer)
cost = db.Column(db.Integer)
honesty = db.Column(db.Integer)
communication = db.Column(db.Integer)
name = db.Column(db.String())
ipaddr = db.Column(db.String())
date = db.Column(db.String())
def __init__(self, fullurl, url, comments, overall, shipping, cost, honesty, communication, name, ipaddr, date):
self.fullurl = fullurl
self.url = url
self.comments = comments
self.overall = overall
self.shipping = shipping
self.cost = cost
self.honesty = honesty
self.communication = communication
self.name = name
self.ipaddr = ipaddr
self.date = date
It seems that you actually haven't executed your query. Try following:
return jsonify(json_list = qryresult.all())
[Edit]: Problem with jsonify is, that usually the objects cannot be jsonified automatically. Even Python's datetime fails ;)
What I have done in the past, is adding an extra property (like serialize) to classes that need to be serialized.
def dump_datetime(value):
"""Deserialize datetime object into string form for JSON processing."""
if value is None:
return None
return [value.strftime("%Y-%m-%d"), value.strftime("%H:%M:%S")]
class Foo(db.Model):
# ... SQLAlchemy defs here..
def __init__(self, ...):
# self.foo = ...
pass
#property
def serialize(self):
"""Return object data in easily serializable format"""
return {
'id' : self.id,
'modified_at': dump_datetime(self.modified_at),
# This is an example how to deal with Many2Many relations
'many2many' : self.serialize_many2many
}
#property
def serialize_many2many(self):
"""
Return object's relations in easily serializable format.
NB! Calls many2many's serialize property.
"""
return [ item.serialize for item in self.many2many]
And now for views I can just do:
return jsonify(json_list=[i.serialize for i in qryresult.all()])
[Edit 2019]:
In case you have more complex objects or circular references, use a library like marshmallow).
Here's what's usually sufficient for me:
I create a serialization mixin which I use with my models. The serialization function basically fetches whatever attributes the SQLAlchemy inspector exposes and puts it in a dict.
from sqlalchemy.inspection import inspect
class Serializer(object):
def serialize(self):
return {c: getattr(self, c) for c in inspect(self).attrs.keys()}
#staticmethod
def serialize_list(l):
return [m.serialize() for m in l]
All that's needed now is to extend the SQLAlchemy model with the Serializer mixin class.
If there are fields you do not wish to expose, or that need special formatting, simply override the serialize() function in the model subclass.
class User(db.Model, Serializer):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String)
password = db.Column(db.String)
# ...
def serialize(self):
d = Serializer.serialize(self)
del d['password']
return d
In your controllers, all you have to do is to call the serialize() function (or serialize_list(l) if the query results in a list) on the results:
def get_user(id):
user = User.query.get(id)
return json.dumps(user.serialize())
def get_users():
users = User.query.all()
return json.dumps(User.serialize_list(users))
I had the same need, to serialize into json. Take a look at this question. It shows how to discover columns programmatically. So, from that I created the code below. It works for me, and I'll be using it in my web app. Happy coding!
def to_json(inst, cls):
"""
Jsonify the sql alchemy query result.
"""
convert = dict()
# add your coversions for things like datetime's
# and what-not that aren't serializable.
d = dict()
for c in cls.__table__.columns:
v = getattr(inst, c.name)
if c.type in convert.keys() and v is not None:
try:
d[c.name] = convert[c.type](v)
except:
d[c.name] = "Error: Failed to covert using ", str(convert[c.type])
elif v is None:
d[c.name] = str()
else:
d[c.name] = v
return json.dumps(d)
class Person(base):
__tablename__ = 'person'
id = Column(Integer, Sequence('person_id_seq'), primary_key=True)
first_name = Column(Text)
last_name = Column(Text)
email = Column(Text)
#property
def json(self):
return to_json(self, self.__class__)
Here's my approach:
https://github.com/n0nSmoker/SQLAlchemy-serializer
pip install SQLAlchemy-serializer
You can easily add mixin to your model and then just call
.to_dict() method on its instance.
You also can write your own mixin on base of SerializerMixin.
For a flat query (no joins) you can do this
#app.route('/results/')
def results():
data = Table.query.all()
result = [d.__dict__ for d in data]
return jsonify(result=result)
and if you only want to return certain columns from the database you can do this
#app.route('/results/')
def results():
cols = ['id', 'url', 'shipping']
data = Table.query.all()
result = [{col: getattr(d, col) for col in cols} for d in data]
return jsonify(result=result)
Ok, I've been working on this for a few hours, and I've developed what I believe to be the most pythonic solution yet. The following code snippets are python3 but shouldn't be too horribly painful to backport if you need.
The first thing we're gonna do is start with a mixin that makes your db models act kinda like dicts:
from sqlalchemy.inspection import inspect
class ModelMixin:
"""Provide dict-like interface to db.Model subclasses."""
def __getitem__(self, key):
"""Expose object attributes like dict values."""
return getattr(self, key)
def keys(self):
"""Identify what db columns we have."""
return inspect(self).attrs.keys()
Now we're going to define our model, inheriting the mixin:
class MyModel(db.Model, ModelMixin):
id = db.Column(db.Integer, primary_key=True)
foo = db.Column(...)
bar = db.Column(...)
# etc ...
That's all it takes to be able to pass an instance of MyModel() to dict() and get a real live dict instance out of it, which gets us quite a long way towards making jsonify() understand it. Next, we need to extend JSONEncoder to get us the rest of the way:
from flask.json import JSONEncoder
from contextlib import suppress
class MyJSONEncoder(JSONEncoder):
def default(self, obj):
# Optional: convert datetime objects to ISO format
with suppress(AttributeError):
return obj.isoformat()
return dict(obj)
app.json_encoder = MyJSONEncoder
Bonus points: if your model contains computed fields (that is, you want your JSON output to contain fields that aren't actually stored in the database), that's easy too. Just define your computed fields as #propertys, and extend the keys() method like so:
class MyModel(db.Model, ModelMixin):
id = db.Column(db.Integer, primary_key=True)
foo = db.Column(...)
bar = db.Column(...)
#property
def computed_field(self):
return 'this value did not come from the db'
def keys(self):
return super().keys() + ['computed_field']
Now it's trivial to jsonify:
#app.route('/whatever', methods=['GET'])
def whatever():
return jsonify(dict(results=MyModel.query.all()))
If you are using flask-restful you can use marshal:
from flask.ext.restful import Resource, fields, marshal
topic_fields = {
'title': fields.String,
'content': fields.String,
'uri': fields.Url('topic'),
'creator': fields.String,
'created': fields.DateTime(dt_format='rfc822')
}
class TopicListApi(Resource):
def get(self):
return {'topics': [marshal(topic, topic_fields) for topic in DbTopic.query.all()]}
You need to explicitly list what you are returning and what type it is, which I prefer anyway for an api. Serialization is easily taken care of (no need for jsonify), dates are also not a problem. Note that the content for the uri field is automatically generated based on the topic endpoint and the id.
Here's my answer if you're using the declarative base (with help from some of the answers already posted):
# in your models definition where you define and extend declarative_base()
from sqlalchemy.ext.declarative import declarative_base
...
Base = declarative_base()
Base.query = db_session.query_property()
...
# define a new class (call "Model" or whatever) with an as_dict() method defined
class Model():
def as_dict(self):
return { c.name: getattr(self, c.name) for c in self.__table__.columns }
# and extend both the Base and Model class in your model definition, e.g.
class Rating(Base, Model):
____tablename__ = 'rating'
id = db.Column(db.Integer, primary_key=True)
fullurl = db.Column(db.String())
url = db.Column(db.String())
comments = db.Column(db.Text)
...
# then after you query and have a resultset (rs) of ratings
rs = Rating.query.all()
# you can jsonify it with
s = json.dumps([r.as_dict() for r in rs], default=alchemyencoder)
print (s)
# or if you have a single row
r = Rating.query.first()
# you can jsonify it with
s = json.dumps(r.as_dict(), default=alchemyencoder)
# you will need this alchemyencoder where your are calling json.dumps to handle datetime and decimal format
# credit to Joonas # http://codeandlife.com/2014/12/07/sqlalchemy-results-to-json-the-easy-way/
def alchemyencoder(obj):
"""JSON encoder function for SQLAlchemy special classes."""
if isinstance(obj, datetime.date):
return obj.isoformat()
elif isinstance(obj, decimal.Decimal):
return float(obj)
Flask-Restful 0.3.6 the Request Parsing recommend marshmallow
marshmallow is an ORM/ODM/framework-agnostic library for converting
complex datatypes, such as objects, to and from native Python
datatypes.
A simple marshmallow example is showing below.
from marshmallow import Schema, fields
class UserSchema(Schema):
name = fields.Str()
email = fields.Email()
created_at = fields.DateTime()
from marshmallow import pprint
user = User(name="Monty", email="monty#python.org")
schema = UserSchema()
result = schema.dump(user)
pprint(result)
# {"name": "Monty",
# "email": "monty#python.org",
# "created_at": "2014-08-17T14:54:16.049594+00:00"}
The core features contain
Declaring Schemas
Serializing Objects (“Dumping”)
Deserializing Objects (“Loading”)
Handling Collections of Objects
Validation
Specifying Attribute Names
Specifying Serialization/Deserialization Keys
Refactoring: Implicit Field Creation
Ordering Output
“Read-only” and “Write-only” Fields
Specify Default Serialization/Deserialization Values
Nesting Schemas
Custom Fields
Here is a way to add an as_dict() method on every class, as well as any other method you want to have on every single class.
Not sure if this is the desired way or not, but it works...
class Base(object):
def as_dict(self):
return dict((c.name,
getattr(self, c.name))
for c in self.__table__.columns)
Base = declarative_base(cls=Base)
I've been looking at this problem for the better part of a day, and here's what I've come up with (credit to https://stackoverflow.com/a/5249214/196358 for pointing me in this direction).
(Note: I'm using flask-sqlalchemy, so my model declaration format is a bit different from straight sqlalchemy).
In my models.py file:
import json
class Serializer(object):
__public__ = None
"Must be implemented by implementors"
def to_serializable_dict(self):
dict = {}
for public_key in self.__public__:
value = getattr(self, public_key)
if value:
dict[public_key] = value
return dict
class SWEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, Serializer):
return obj.to_serializable_dict()
if isinstance(obj, (datetime)):
return obj.isoformat()
return json.JSONEncoder.default(self, obj)
def SWJsonify(*args, **kwargs):
return current_app.response_class(json.dumps(dict(*args, **kwargs), cls=SWEncoder, indent=None if request.is_xhr else 2), mimetype='application/json')
# stolen from https://github.com/mitsuhiko/flask/blob/master/flask/helpers.py
and all my model objects look like this:
class User(db.Model, Serializer):
__public__ = ['id','username']
... field definitions ...
In my views I call SWJsonify wherever I would have called Jsonify, like so:
#app.route('/posts')
def posts():
posts = Post.query.limit(PER_PAGE).all()
return SWJsonify({'posts':posts })
Seems to work pretty well. Even on relationships. I haven't gotten far with it, so YMMV, but so far it feels pretty "right" to me.
Suggestions welcome.
I was looking for something like the rails approach used in ActiveRecord to_json and implemented something similar using this Mixin after being unsatisfied with other suggestions. It handles nested models, and including or excluding attributes of the top level or nested models.
class Serializer(object):
def serialize(self, include={}, exclude=[], only=[]):
serialized = {}
for key in inspect(self).attrs.keys():
to_be_serialized = True
value = getattr(self, key)
if key in exclude or (only and key not in only):
to_be_serialized = False
elif isinstance(value, BaseQuery):
to_be_serialized = False
if key in include:
to_be_serialized = True
nested_params = include.get(key, {})
value = [i.serialize(**nested_params) for i in value]
if to_be_serialized:
serialized[key] = value
return serialized
Then, to get the BaseQuery serializable I extended BaseQuery
class SerializableBaseQuery(BaseQuery):
def serialize(self, include={}, exclude=[], only=[]):
return [m.serialize(include, exclude, only) for m in self]
For the following models
class ContactInfo(db.Model, Serializer):
id = db.Column(db.Integer, primary_key=True)
user_id = db.Column(db.Integer, db.ForeignKey('user.id'))
full_name = db.Column(db.String())
source = db.Column(db.String())
source_id = db.Column(db.String())
email_addresses = db.relationship('EmailAddress', backref='contact_info', lazy='dynamic')
phone_numbers = db.relationship('PhoneNumber', backref='contact_info', lazy='dynamic')
class EmailAddress(db.Model, Serializer):
id = db.Column(db.Integer, primary_key=True)
email_address = db.Column(db.String())
type = db.Column(db.String())
contact_info_id = db.Column(db.Integer, db.ForeignKey('contact_info.id'))
class PhoneNumber(db.Model, Serializer):
id = db.Column(db.Integer, primary_key=True)
phone_number = db.Column(db.String())
type = db.Column(db.String())
contact_info_id = db.Column(db.Integer, db.ForeignKey('contact_info.id'))
phone_numbers = db.relationship('Invite', backref='phone_number', lazy='dynamic')
You could do something like
#app.route("/contact/search", methods=['GET'])
def contact_search():
contact_name = request.args.get("name")
matching_contacts = ContactInfo.query.filter(ContactInfo.full_name.like("%{}%".format(contact_name)))
serialized_contact_info = matching_contacts.serialize(
include={
"phone_numbers" : {
"exclude" : ["contact_info", "contact_info_id"]
},
"email_addresses" : {
"exclude" : ["contact_info", "contact_info_id"]
}
}
)
return jsonify(serialized_contact_info)
I was working with a sql query defaultdict of lists of RowProxy objects named jobDict
It took me a while to figure out what Type the objects were.
This was a really simple quick way to resolve to some clean jsonEncoding just by typecasting the row to a list and by initially defining the dict with a value of list.
jobDict = defaultdict(list)
def set_default(obj):
# trickyness needed here via import to know type
if isinstance(obj, RowProxy):
return list(obj)
raise TypeError
jsonEncoded = json.dumps(jobDict, default=set_default)
I just want to add my method to do this.
just define a custome json encoder to serilize your db models.
class ParentEncoder(json.JSONEncoder):
def default(self, obj):
# convert object to a dict
d = {}
if isinstance(obj, Parent):
return {"id": obj.id, "name": obj.name, 'children': list(obj.child)}
if isinstance(obj, Child):
return {"id": obj.id, "name": obj.name}
d.update(obj.__dict__)
return d
then in your view function
parents = Parent.query.all()
dat = json.dumps({"data": parents}, cls=ParentEncoder)
resp = Response(response=dat, status=200, mimetype="application/json")
return (resp)
it works well though the parent have relationships
It's been a lot of times and there are lots of valid answers, but the following code block seems to work:
my_object = SqlAlchemyModel()
my_serializable_obj = my_object.__dict__
del my_serializable_obj["_sa_instance_state"]
print(jsonify(my_serializable_object))
I'm aware that this is not a perfect solution, nor as elegant as the others, however for those who want o quick fix, they might try this.
I'm creating an API using peewee as the ORM and I need the ability to convert a peewee model object into a JSON object to send to the user. Does anyone know of a good way to do this?
Peewee has a model_to_dict and dict_to_model helpers in the playhouse.shortcuts extension module.
http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#model_to_dict
http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#dict_to_model
You could use these as follows:
from playhouse.shortcuts import model_to_dict, dict_to_model
user_obj = User.select().where(User.username == 'charlie').get()
json_data = json.dumps(model_to_dict(user_obj))
Also note that model_to_dict() can recurse through related models, include back-referenced models, and exclude certain fields from being serialized.
when single fetch
user = User.select().where(User.id == 1).get()
model_to_dict(user) #to Dict
when Multiple fetch
users = list(User.select().where(User.name ** 'a%').dicts())
also, you can get model as a dict, and then convert to json with correct field types (bool, int, float, etc.):
import peewee
import json
from bson import json_util
from datetime import datetime
class User(peewee.Model):
email = CharField()
status = BooleanField(default=True)
firstname = CharField()
lastname = CharField()
age = IntegerField()
created = DateTimeField(default=datetime.now())
class Meta:
database = db
user = User.select().dicts().get()
print json.dumps(user, default=json_util.default)
For anybody having issues like TypeError: Object of type date is not JSON serializable, this works for me (tested on Python 3.8.2).
from playhouse.shortcuts import model_to_dict
import json
def print_model(model):
print(json.dumps(model_to_dict(model), indent=4, sort_keys=True, default=str))
def print_models(models):
print(json.dumps(list(models.dicts()), indent=4, sort_keys=True, default=str))
Usage 1 - Single model
for person in Person.select():
print_model(person)
Usage 2 - Many models
print_models(Person.select())
I had this very same problem and ended up defining my own parser extension for JSON types that could not be automatically serialized. I'm fine for now in using strings as data represented (although you could possibly use different datatypes, but beware of approximation using floating points!
In the following example, I put this in a file called json_serialize.py inside a utils folder:
from decimal import Decimal
import datetime
try:
import uuid
_use_uuid = True
except ImportError:
_use_uuid = False
datetime_format = "%Y/%m/%d %H:%M:%S"
date_format = "%Y/%m/%d"
time_format = "%H:%M:%S"
def set_datetime_format(fmt_string):
datetime_format = fmt_string
def set_date_format(fmt_string):
date_format = fmt_string
def set_time_format(fmt_string):
time_format = fmt_string
def more(obj):
if isinstance(obj, Decimal):
return str(obj)
if isinstance(obj, datetime.datetime):
return obj.strftime(datetime_format)
if isinstance(obj, datetime.date):
return obj.strftime(date_format)
if isinstance(obj, datetime.time):
return obj.strftime(time_format)
if _use_uuid and isinstance(obj, uuid.UUID):
return str(obj.db_value())
raise TypeError("%r is not JSON serializable" % obj)
Then, in my app:
import json
from utils import json_serialize
...
json.dumps(model_to_dict(User.get()), default=json_serialize.more)
edit just to add: this is very largely inspired by json_utils.default module found in mongodb but mainly relies on the json module and needs no import of mongodb own bson/json_utils module.
Usually I update it to support new types as soon as my app raises the TypeError for it found a type not able to serialize
I usually implement the model to dict and dict to model functions, for maximum security and understanding of the inner workings of the code. Peewee does a lot of magic and you want to be in control over it.
The most obvious argument for why you should not iterate on the fields but rather explicitly specify them is because of security considerations. Not all fields can be exposed to the user, and I assume you need this functionality to implement some sort of REST API.
So - you should do something like this:
class UserData(db.Model):
user = db.ForeignKeyField(User)
data = db.CharField()
def serialize():
# front end does not need user ID here
return {
'data': self.data
}
#classmethod
def from_json(cls, json_data):
UserData.create(
# we enforce user to be the current user
user=current_user,
data=json_data['data']
)
you can do something like that:
class MyModel(peewee.Model):
def __str__(self):
r = {}
for k in self._data.keys():
try:
r[k] = str(getattr(self, k))
except:
r[k] = json.dumps(getattr(self, k))
return str(r)
class User(MyModel):
email = CharField()
status = CharField(default="enabled")
firstname = CharField()
lastname = CharField()
class Meta:
database = db