How to save Python Dict Values to SQLite Model? - python

I am calling an API that returns JSON data. I am using json.loads to decode the JSON to Python as a dictionary. The dictionary that returns is a somewhat complex nested dictonary with nested lists as well.
For example:
{ "educations": { "_total": 1, "values": [{ "degree": "Bachelor of Arts", "fieldOfStudy": "Psychology", "schoolName": "Antioch University Seattle" }] }
How do I store each of the values into an SQLite database with the model defined as:
class Educations(models.Model):
name = models.ForeignKey(Candidate)
degree = models.CharField(max_length=200)
fieldOfStudy = models.CharField(max_length=200)
schoolName = models.CharField(max_length=200)
Each education is associated with a Candidate which is defined in the Candidate class (not shown here).

You can set the fields in new object like
# Get after json.dump
json_data = { "educations": { "_total": 1, "values": [{ "degree": "Bachelor of Arts", "fieldOfStudy": "Psychology", "schoolName": "Antioch University Seattle" }] }
for each_education in json_data['educations']['values']
new_education = Education(**each_education)
# Set the name foreign key
# new_educaiton.name = name
new_education.save()
Note: How you will get the data for the name foreignkey is not mention in your question so that also you can set.

Related

How should I add a field containing a list of dictionaries in Marshmallow Python?

In Marshmallow in order to have a list field you can use:
include_in = fields.List(cls_or_instance=fields.Str(),
default=['sample1', 'sample2'])
This is OK, but I have a new requirement to have a list of dictionaries in a field. A sample payload:
[{
"name": "Ali",
"age": 20
},
{
"name": "Hasan",
"age": 32
}]
This payload is part of the bigger schema, so now the question is how should I add and validate such a field?
EDIT-1:
I went a step further and could find out that there is a Dict field type in Marshmallow so until now I have the below code sample:
fields.List(fields.Dict(
keys=fields.String(validate=OneOf(('name', 'age'))),
values=fields.String(required=True)
))
Now new problem arise and I cannot set different data types for fields in the dictionary (name and age). I'd be happy if someone could shed some light on this.
If the items in the list have the same shape, you can use a nested field within fields.List, like so:
class PersonSchema(Schema):
name = fields.Str()
age = fields.Int()
class RootSchema(Schema):
people = fields.List(fields.Nested(PersonSchema))
Another approach for validating list of dictionaries in a field using one schema class.
from marshmallow import Schema, ValidationError
class PeopleSchema(Schema):
name = fields.Str(required=True)
age = fields.Int(required=True)
people = [{
"name": "Ali",
"age": 20
},
{
"name": "Hasan",
"age": 32
},
{
"name": "Ali",
"age": "twenty" # Error: Not an int
}
]
def validate_people():
try:
validated_data = PeopleSchema(many=True).load(people)
except ValidationError as err:
print(err.messages)
validate_people()
Output:
{2: {'age': ['Not a valid integer.']}}

Dynamically create classes- Python

I'm trying to figure out what would be the best way to create classes in a dynamic manner based on the contents of a JSON file. So for example, here's a snippet from the JSON file:
{
"stuff": [{
"name": "burger",
"aka": ["cheeseburger", "hamburger"]
},
{
"name": "fries",
"aka": ["french fries", "potatoes"]
},
{
"name": "meal",
"items": [{
"name": "burger",
"value": "<burger>"
},
{
"name": "fries",
"value": "<fries>"
}
]
}
]
}
And now based on this JSON, I want classes that represent these objects. So for example, something like:
class Burger:
def __init__(self):
self.name = "burger"
self.aka = ["cheeseburger", "hamburger"]
class Meal:
def __init__(self):
self.name = "meal"
self.burger = Burger()
self.fries = Fries()
So basically, based on that JSON, I want to be able to create classes that represent the same attributes and relationships that we see in the JSON. Any ideas about the best way to approach this would be appreciated!
Assuming json variable contains your json data try this:
for d in json:
name = d.pop('name')
t = type(name, (object,), d)
What it does is to call type, which will create new type in python (exactly the same as if you did class name, which correct name set to content of name variable, with base class object and attributes in d. Variable t will contain class object you want.

How to access ForeignKey data without making extra queries in Wagtail(django)

I have the following two classes in my app.models and i'm using the wagtail APIs to get the data as json
class AuthorMeta(Page):
author=models.OneToOneField(User)
city = models.ForeignKey('Cities', related_name='related_author')
class Cities(Page):
name = models.CharField(max_length=30)
So, when I try /api/v1/pages/?type=dashboard.AuthorMeta&fields=title,city, it returns the following data:
{
"meta": {
"total_count": 1
},
"pages": [
{
"id": 11,
"meta": {
"type": "dashboard.AuthorMeta",
"detail_url": "http://localhost:8000/api/v1/pages/11/"
},
"title": "Suneet Choudhary",
"city": {
"id": 10,
"meta": {
"type": "dashboard.Cities",
"detail_url": "http://localhost:8000/api/v1/pages/10/"
}
}
}
]
}
In the city field, it returns the id and meta of the city. How can I get the name of the city in the response here, without making an extra query? :/
I couldn't find any solution in the Documentation. Am I missing something?
Use Django model property to return through the ForeignKey:
class AuthorMeta(Page):
author=models.OneToOneField(User)
city = models.ForeignKey('Cities', related_name='related_author')
city_name = property(get_city_name)
def get_city_name(self):
return self.city.name
Check Term Property to better understand the concept
In case you have the foreign key in a Streamfield, e.g. a PageChooserBlock, you can customize the api response by overwriting the get_api_representation of a block, as described in the example as provided here:
class CustomPageChooserBlock(blocks.PageChooserBlock):
""" Customize the api response. """
def get_api_representation(self, value, context=None):
""" Return the url path instead of the id. """
return value.url_path

Dereference relations in MongoEngine embedded documents

I have a schema that using MongoEngine that looks like this
class User(db.Document)
email = db.EmailField(unique=True)
class QueueElement(db.EmbeddedDocument):
accepts = db.ListField(db.ReferenceField('Resource'))
user = db.ReferenceField(User)
class Resource(db.Document):
name = db.StringField(max_length=255, required=True)
current_queue_element = db.EmbeddedDocumentField('QueueElement')
class Queue(db.EmbeddedDocument):
name = db.StringField(max_length=255, required=True)
resources = db.ListField(db.ReferenceField(Resource))
queue_elements = db.ListField(db.EmbeddedDocumentField('QueueElement'))
class Room(db.Document):
name = db.StringField(max_length=255, required=True)
queues = db.ListField(db.EmbeddedDocumentField('Queue'))
and I would like to return a JSON object of a Room object that would include the information about its queues (together with the referenced resources), and the nested queue_elements ( together with their referenced "accepts" references, and user references)
However, when I want to return a Room with its relationships dereferenced:
room = Room.objects(slug=slug).select_related()
if (room):
return ast.literal_eval(room.to_json())
abort(404)
I don't get any dereferencing. I get:
{
"_cls":"Room",
"_id":{
"$oid":"552ab000605cd92f22347d79"
},
"created_at":{
"$date":1428842482049
},
"name":"second",
"queues":[
{
"created_at":{
"$date":1428842781490
},
"name":"myQueue",
"queue_elements":[
{
"accepts":[
{
"$oid":"552aafb3605cd92f22347d78"
},
{
"$oid":"552aafb3605cd92f22347d78"
},
{
"$oid":"552ab1f8605cd92f22347d7a"
}
],
"created_at":{
"$date":1428849389503
},
"user":{
"$oid":"552ac8c7605cd92f22347d7b"
}
}
],
"resources":[
{
"$oid":"552aafb3605cd92f22347d78"
},
{
"$oid":"552aafb3605cd92f22347d78"
},
{
"$oid":"552ab1f8605cd92f22347d7a"
}
]
}
],
"slug":"secondslug"
}
even though I'm using the select_related() function. I believe this is because MongoEngine may not follow references on embedded documents. Note, I can actually dereference in the python if I do something like this:
room = Room.objects(slug=slug).first().queues[0].queue_elements[0].accepts[0]
return ast.literal_eval(room.to_json())
which yields
{
"_id":{
"$oid":"552aafb3605cd92f22347d78"
},
"created_at":{
"$date":1428842849393
},
"name":"myRes"
}
which is clearly the dereferenced Resource document.
Is there a way I can follow references on embedded documents? Or is this coming up because I'm following a bad pattern, and should be finding a different way to store this information in MongoDB (or indeed, switch to a Relational DB) ? Thanks!

auto field values in ndb.StructuredProperty

I want to store locations in google's datastore. Each entry shall have got 'sys'-fields, which shall contain information set by the datastore.
I've got the class model below and the WebService JSON request/response looks ok, but I have to set the values manually. It looks like auto_current_user_add, auto_now_add, auto_current_user and auto_now does not trigger.
from google.appengine.ext import ndb
from endpoints_proto_datastore.ndb import EndpointsModel
class Created(EndpointsModel):
by = ndb.UserProperty(auto_current_user_add=True)
on = ndb.DateTimeProperty(auto_now_add=True)
class Updated(EndpointsModel):
by = ndb.UserProperty(auto_current_user=True)
on = ndb.DateTimeProperty(auto_now=True)
class Sys(EndpointsModel):
created = ndb.StructuredProperty(Created)
updated = ndb.StructuredProperty(Updated)
class Location(EndpointsModel):
name = ndb.StringProperty(required=True)
description = ndb.TextProperty()
address = ndb.StringProperty()
sys = ndb.StructuredProperty(Sys)
When I submit a create request (location.put()) I get the following response:
{
"id": "4020001",
"name": "asdf"
}
When I set it manually using:
location.sys = Sys(created=Created(on=datetime.datetime.now(),
by=current_user),
updated=Updated(on=datetime.datetime.now(),
by=current_user))
location.put()
I get the expected result:
{
"id": "4020002",
"name": "asdf",
"sys": {
"created": {
"by": {
"auth_domain": "gmail.com",
"email": "decurgia#XYZ"
},
"on": "2015-01-27T16:05:41.465497"
},
"updated": {
"by": {
"auth_domain": "gmail.com",
"email": "decurgia#XYZ"
},
"on": "2015-01-27T16:05:41.465577"
}
}
}
How can I get those fields (sys.created.on, sys.created.by, sys.updated.on, sys.updated.by) automatically set?
In my limited work with StructuredProperty, I found it to be slower and more difficult to use than simply inserting the properties directly into the model. NDB seems to store those properties separately and perform a "join" when retrieving them. My recommendation is to use a "flat" model:
class Location(EndpointsModel):
name = ndb.StringProperty(required=True)
description = ndb.TextProperty()
address = ndb.StringProperty()
created_by = ndb.UserProperty(auto_current_user_add=True)
created_on = ndb.DateTimeProperty(auto_now_add=True)
updated_by = ndb.UserProperty(auto_current_user=True)
updated_on = ndb.DateTimeProperty(auto_now=True)
This should cause the auto_ properties to be triggered automatically.

Categories

Resources