I want to import data that is in regular JSON format into a (SQLite) db within Django. It looks like fixtures is the recommended way of doing that. I'm not very clear on the steps I need take. Specifically, do I need to create a mapping of which field should go into which model (assuming my model is already defined in models.py) etc? The Django example looks like this:
[
{
"model": "myapp.person",
"pk": 1,
"fields": {
"first_name": "John",
"last_name": "Lennon"
} }, {
"model": "myapp.person",
"pk": 2,
"fields": {
"first_name": "Paul",
"last_name": "McCartney"
}
} ]
However my data isn't in this format. Do I need to edit the JSON file to look like this? Should I just abort and import json instead?
json.loads is all that would be needed. Something like
as_object_list = json.loads(your_json)
This converts the json to a dict, from there you can can create the model by doing
YourModel.objects.create(**as_object_list)
It seems that performing a PATCH on an endpoint with a many-to-many relation updates the object but doesn't return back the updated data until the next response vs returning it back in the PATCH response.
Example with original object:
{
"id": 35,
"interests": [
1,
2
],
"personal_statement": "Hello World",
"photo": "",
"resume": "",
"user": 2
}
PATCH request setting interests=[1,2,3,4,5] ... Example response:
{
"id": 35,
"interests": [
1,
2
],
"personal_statement": "Hello World",
"photo": "",
"resume": "",
"user": 2
}
Example of next GET Response:
{
"id": 35,
"interests": [
1,
2,
3,
4,
5
],
"personal_statement": "Hello World",
"photo": "",
"resume": "",
"user": 2
}
This is using Django v1.7.4 and Django REST Framework v2.4.3
My first assumption is that since it's a many to many relation it is saving the parent object first and returning back that data before saving the many to many relation data, but I'm not entirely sure. Any help would be appreciated. Thanks.
EDIT
The issue is actually an open issue on Django REST Framework with some possible solutions. It was being caused by prefetch_related in my ViewSet queryset:
https://github.com/tomchristie/django-rest-framework/issues/1556
https://github.com/tomchristie/django-rest-framework/issues/2442
According to REST API patterns, PATCH request performs a partial update, so Django-Rest-Framework just returns updated data.
May be you want to override PATCH view to return complete serializer data or change your PATCH request to PUT one.
I encountered the same while adding prefetch_related to a many to many field I had. I solved it by simply adding a custom update to my serializer:
def update(self, instance, validated_data):
super(SimpleClientModelResource, self).update(instance, validated_data)
return self.Meta.model.objects.get(id=instance.id)
I've been caching list and retrieve methods for ViewSets in Django REST Framework using drf-extensions https://chibisov.github.io/drf-extensions/docs/#caching.
I wrote a modification to cache the key indefinitely for those list and retrieve methods which invalidates the keys for these methods when an object is modified.
The issue I am trying to solve is cache invalidation when an embedded object is modified. I am not familiar with the typical patterns for cache invalidation for embedded objects.
Ex:
{
"id": 1,
"position": "manager",
"company": "Google",
"user": {
"id": 1,
"email": "johnsmith#example.com",
"first_name": "John",
"last_name": "Doe",
"last_login": "2015-01-05T20:46:15.400Z",
"joined_at": "2014-12-10T19:54:45.588Z",
}
}
A user modifies the user object, ideally the cache key set for the list and retrieve methods for the user ViewSet along with the embedded object ViewSet should both be invalidated.
Any help is appreciated. Thanks.
I have the following DJango view
def company(request):
company_list = Company.objects.all()
output = serializers.serialize('json', company_list, fields=('name','phonenumber','email','companylogo'))
return HttpResponse(output, content_type="application/json")
it result as follows:
[{"pk": 1, "model": "test.company", "fields": {"companylogo": null, "phonenumber": "741.999.5554", "name": "Remax", "email": "home#remax.co.il"}}, {"pk": 4, "model": "test.company", "fields": {"companylogo": null, "phonenumber": "641-7778889", "name": "remixa", "email": "a#aa.com"}}, {"pk": 2, "model": "test.company", "fields": {"companylogo": null, "phonenumber": "658-2233444", "name": "remix", "email": "b#bbb.com"}}, {"pk": 7, "model": "test.company", "fields": {"companylogo": null, "phonenumber": "996-7778880", "name": "remix", "email": "a#aba.com"}}]
my questions:
1. can i control the order of the fields
2. can i change the name of the fields
3. I was expecting to see the result with indentation in the browser i.e. instead of one long line to see something like this:
[
{
"pk": 1,
"model": "test.company",
"fields":
{
"companylogo": null,
"phonenumber": "741.999.5554",
"name": "Remax",
"email": "home#remax.co.il"
}
},
{
"pk": 4,
"model": "test.company",
"fields":
{
"companylogo": null,
"phonenumber": "641-7778889",
"name": "remixa",
"email": "a#aa.com"
}
},
....
}
]
you can get pretty format in this way:
return JsonResponse(your_json_dict, json_dumps_params={'indent': 2})
django doc as the first comment say
Python (unrelated to Django and starting with 2.6) has a built in json library that can accomplish the indentation you require. If you are looking for something quick and dirty for debug purposes you could do something like this:
from django.http import HttpResponse
from django.core import serializers
import json
def company(request, pretty=False):
company_list = Company.objects.all()
output = serializers.serialize('json', company_list, fields=('name','phonenumber','email','companylogo'))
if pretty:
output = json.dumps(json.loads(output), indent=4))
return HttpResponse(output, content_type="application/json")
But this is a performance issue if the Company model is large. I recommend taking Dan R's advice and use a browser plugin to parse and render the json or come up with some other client side solution. I have a script that takes in a json file and does exactly the same thing as the code above, reads in the json and prints it out with indent=4.
As for sorting your output, you can just use the order_by method on your query set:
def company(request):
company_list = Company.objects.order_by("sorting_field")
...
And if you always want that model sorted that way, you can use the ordering meta-class option:
class Company(models.Model):
class Meta:
ordering = ["sorting_field"]
...
As a final note, If your intent is to expose your models with a web service, I highly recommend taking a look at tastypie. It may help you in the long run as it provides many other convenient features that help towards that end.
With Django 1.7 I can get nicely indented JSON by using the indent parameter of the serializer. For instance, in a command that dumps data from my database:
self.stdout.write(serializers.serialize("json",
records,
indent=2))
The indent parameter has been in Django since version 1.5. The output I get looks like this:
[
{
"fields": {
"type": "something",
"word": "something else",
},
"model": "whatever",
"pk": 887060
},
{
"fields": {
"type": "something more",
"word": "and another thing",
},
"model": "whatever",
"pk": 887061
},
...
To order your records then you'd have to do what Kevin suggested and use order_by, or whatever method you want to order the records you pass to the serializer. For instance, I use itertools.chain to order different querysets that return instances of different models.
The serializer does not support ordering fields according to your preferences, or renaming them. You have to write your own code to do this or use an external tool.
JSON doesn't have indentation, it's simply structured data. Browsers or other tools may format the JSON so that it looks nice but by default it's not there. It's also not part of the JSON as the formatting is just how it looks on the screen. JSON is often processed by other code or services so they don't care about indentation, as long as the data is structured correctly.
so I just try to use tastypie put method to edit objects.
let's say my object have this structure:
{
"id": 38,
"media": [],
"name": "tesdr",
"resource_uri": "/api/v2/group/38/",
"status": 7,
"user_name": null,
"users": []
}
witch media and users are related many to many field. when I edit group and use put without any change in m2m fields every thing works fine.
but when I try to put something like this:
{
"id": 38,
"media": [
"/api/v2/media/70/"
],
"name": "testgpat",
"resource_uri": "/api/v2/group/40/",
"status": 6,
"user_name": null,
"users": []
}
tastypie return an 401 http error. so what is the solution? any idea?
ok! I just solved the problem. have to define a many to many field in both resources which wants to set relation.
thanks all! :D