I am trying to serialize an object with Marshmallow in such a way that "related" properties are grouped together into a single dictionary that does not exist on the original object. My code:
from marshmallow import Schema, fields, pprint
import json
class StatsSchema(Schema):
population = fields.Int()
rating = fields.Int()
class AnimalSchema(Schema):
name = fields.Str()
features = fields.List(fields.Str())
stats = fields.Nested(StatsSchema)
dog = {
'name':'dog',
'features': ['tongue', 'wet nose'],
'population': 200,
'rating': 10
}
animal_schema = AnimalSchema()
data, errors = animal_schema.dump(dog)
print(json.dumps(data, indent=2))
Actual result:
{
"features": [
"tongue",
"wet nose"
],
"name": "dog"
}
Desired result:
{
"features": [
"tongue",
"wet nose"
],
"name": "dog",
"stats": {"population": 500, "rating": 10}
}
I understand that the "stats" key is missing from the output because it is not on the original object, but I am not sure how to specify that Marshmallow should create the new "stats" key as a new dictionary using the object.
I found one possible way to create the inner dictionary. Not sure if it is the only/best method:
class AnimalSchema(Schema):
name = fields.Str()
features = fields.List(fields.Str())
stats = fields.Method('get_stats')
def get_stats(self, post):
data, err = StatsSchema().dump(post)
return data
This is discussed in https://github.com/marshmallow-code/marshmallow/issues/940.
You could do that
class AnimalSchema(Schema):
name = fields.Str()
features = fields.List(fields.Str())
stats = fields.Nested(StatsSchema, dump_only=True)
class Animal:
[...]
#property
def stats(self):
return {'population': self.population, 'rating': self.rating}
Related
For googlers also that question in github.
I had users of 2 types. I need to validate user data by one of 2 schemas.
I prepare awesome scratch with code of my idea, of course its not working.
class ExtraType0(Schema):
nickname = fields.String()
class ExtraType1(Schema):
id = fields.Integer()
class UserSchema(Schema):
type = fields.String()
extra = fields.Method(deserialize="get_extra_schema_by_user_type")
def get_extra_schema_by_user_type(self, obj):
if obj == 0:
# call ExtraType0 scheme and its (de)serialization
return fields.Nested(ExtraType0())
else:
# call ExtraType01scheme and its (de)serialization
return fields.Nested(ExtraType1())
# correct data
result = UserSchema().load(
{
"type": 0,
"extra": {
"id": 0
}
})
# also correct data
result1 = UserSchema().load(
{
"type": 1,
"extra": {
"nickname": "user123"
}
})
How I can proper choose schema depends on loaded in type field data?
I am using sqlalchemy orm to map a json to multiple tables.
I have two tables, each for which I have a class.
One of the tables, has a parameter, which is a list of dictionaries and this has to be mapped to different columns of the second table.
from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
Base = declarative_base()
class A(Base):
__tablename__ = 'a'
id = Column(primary_key, Interger(10))
param1 = Column(String(45))
param2 = Column(String(45))
atobmap = relationship('B', lazy='subquery', back_populates="a")
class B(Base):
__tablename__ = 'b'
bid = Column(primary_key=True, nullable=True, Integer(15))
bparam1 = Column(ForeignKey('a.id'), nullable=False)
bparam2 = Column(String(45))
a = relationship('A', back_populates='b')
sample json:
{
"id": 1,
"param1": "Value",
"param2": "Value",
"atobmap": [
{ "bid": 1,
"bparam1": 1,
"bparam2": "Value"
},
{ "bid": 2,
"bparam1": 1,
"bparam2": "Value"
}
]
}
When I manually insert data into the tables and do a GET, its working fine and is returning values as expected, although it adds an _sa_instance_state key with value {} inside my atobmap dictionaries.
{
...
"atobmap": [
{ "_sa_instance_state": {}
"bid": 1,
"bparam1": 1,
"bparam2": "Value"
},
{ "_sa_instance_state": {}
"bid": 2,
"bparam1": 1,
"bparam2": "Value"
}
]
}
But when I try to POST the values, it throws an error saying "Unhashable type: dict".
You can use a function similar to the one below.
It takes an SQLAlchemy model and a dictionary of field/value pairs and
returns a model instance. Ignores keys that are in the dictionary
that aren't fields on the model. Traverses and instantiates relationships.
Example:
Our models include a Company model with a name field that is a string
and a users field that is a one-to-many relation to a User model.
Our data looks like the following.
{
"company":
{
"name": "Owoga"
"users": [
{
"name": "Eric"
}
]
}
}
We will use SQLAlchemy's class_mapper function to get the "mapper"
for the model. We can then access mapper.attrs.keys() to get a list
of all the mapped fields. Any of those keys that are not relationships, simply
grab them out of the data. For keys that are relationships, get the mapper
of the relationship and then recursively call from_json with that related mapper
and the sub-data. Then, attach that new instance[s] to the parent instance.
def from_json(model, data):
mapper = class_mapper(model)
keys = mapper.attrs.keys()
relationships = inspect(mapper).relationships
args = {k: v for k, v in data.items() if k in keys and k not in relationships}
instance = model(**args)
for attr, relationship in relationships.items():
if relationship.direction in [MANYTOMANY, ONETOMANY]:
relationship_instances = [
from_json(relationship.mapper.class_, el) for el in data[attr]
]
setattr(instance, attr, relationship_instances)
elif relationship.direction == MANYTOONE:
try:
relationship_instance = from_json(
relationship.mapper.class_, data[attr]
)
except KeyError:
pass
else:
setattr(instance, attr, relationship_instance)
return instance
In Marshmallow in order to have a list field you can use:
include_in = fields.List(cls_or_instance=fields.Str(),
default=['sample1', 'sample2'])
This is OK, but I have a new requirement to have a list of dictionaries in a field. A sample payload:
[{
"name": "Ali",
"age": 20
},
{
"name": "Hasan",
"age": 32
}]
This payload is part of the bigger schema, so now the question is how should I add and validate such a field?
EDIT-1:
I went a step further and could find out that there is a Dict field type in Marshmallow so until now I have the below code sample:
fields.List(fields.Dict(
keys=fields.String(validate=OneOf(('name', 'age'))),
values=fields.String(required=True)
))
Now new problem arise and I cannot set different data types for fields in the dictionary (name and age). I'd be happy if someone could shed some light on this.
If the items in the list have the same shape, you can use a nested field within fields.List, like so:
class PersonSchema(Schema):
name = fields.Str()
age = fields.Int()
class RootSchema(Schema):
people = fields.List(fields.Nested(PersonSchema))
Another approach for validating list of dictionaries in a field using one schema class.
from marshmallow import Schema, ValidationError
class PeopleSchema(Schema):
name = fields.Str(required=True)
age = fields.Int(required=True)
people = [{
"name": "Ali",
"age": 20
},
{
"name": "Hasan",
"age": 32
},
{
"name": "Ali",
"age": "twenty" # Error: Not an int
}
]
def validate_people():
try:
validated_data = PeopleSchema(many=True).load(people)
except ValidationError as err:
print(err.messages)
validate_people()
Output:
{2: {'age': ['Not a valid integer.']}}
I want to group contact_type and contact_value into contact nested object in output for following model:
class Account(Base):
__tablename__ = 'accounts'
id = sa.Column(sa.Integer, primary_key=True)
contact_type = sa.Column(sa.String)
contact_value = sa.Column(sa.String)
This result is expected:
{
"id": 1,
"contact": {
"type": "phone",
"value": "1234567"
}
}
What is the best way to implement this?
ma = Marshmallow()
class AccountContactSchema(ma.Schema):
type = ma.Str()
value = ma.Str()
class AccountSchema(ma.Schema):
id = ma.Int()
contact = ma.Nested(AccountContactSchema)
account_schema = AccountSchema()
I don't know if this is the best way, but you can do this:
from marshmallow import fields
class AccountSchema(ma.Schema):
id = ma.Int()
contact = fields.Function(lambda x : {'type': x.contact_type, 'value': x.contact_value})
and use:
>>> acc = Account(id=1, contact_type="phone", contact_value="1234567")
>>> acc_dict, errors = AccountSchema().dump(acc)
>>> print acc_dict
{u'contact': {'type': 'phone', 'value': '1234567'}, u'id': 1}
Or you can do in other styles of dict:
class AccountSchema(ma.Schema):
id = ma.Int()
contact = fields.Function(lambda x : {x.contact_type : x.contact_value})
This will result:
{u'contact': {'phone': '1234567'}, u'id': 1}
Take a look at Custom Fields
I am calling an API that returns JSON data. I am using json.loads to decode the JSON to Python as a dictionary. The dictionary that returns is a somewhat complex nested dictonary with nested lists as well.
For example:
{ "educations": { "_total": 1, "values": [{ "degree": "Bachelor of Arts", "fieldOfStudy": "Psychology", "schoolName": "Antioch University Seattle" }] }
How do I store each of the values into an SQLite database with the model defined as:
class Educations(models.Model):
name = models.ForeignKey(Candidate)
degree = models.CharField(max_length=200)
fieldOfStudy = models.CharField(max_length=200)
schoolName = models.CharField(max_length=200)
Each education is associated with a Candidate which is defined in the Candidate class (not shown here).
You can set the fields in new object like
# Get after json.dump
json_data = { "educations": { "_total": 1, "values": [{ "degree": "Bachelor of Arts", "fieldOfStudy": "Psychology", "schoolName": "Antioch University Seattle" }] }
for each_education in json_data['educations']['values']
new_education = Education(**each_education)
# Set the name foreign key
# new_educaiton.name = name
new_education.save()
Note: How you will get the data for the name foreignkey is not mention in your question so that also you can set.