Develop Reference

Django Rest Framework - re-using models logic when deserializing responses in a python client

When developing a REST API using django and django rest framework, it seems to me that there's a step missing at the very end of this chain, especially if you'll design a python client of that API:
define the model of your resources in django
use DRF's serializers to send your resource over HTTP
missing: deserialize your resource back in a python model
Let me give a simple example. I declare the following django model with a bit of business logic:
class Numberplate(models.Model):
number = models.CharField(max_length=128)
country_code = models.CharField(max_length=3)
def __repr__(self):
return f"{self.number} ({self.country_code})"
DRF is super powerful when it comes to building the API around that model. However, if I call that API from a python client, it seems that the best I can get as a response from that API will be a simple json without all the logic I might have written in my model (in this case, the __repr__ method), and I'll have to navigate through it as such.
Continuing my example, here is the difference with and without a proper deserializer:
# the best we can do without rewriting code:
>> print(api.get(id=3))
{"numberplate": "123ABC", "country_code": "BE"}
# instead of re-using my __repr__, which would be ideal:
>> print(api.get(id=3))
123ABC (BE)
Is there a clean way to stay DRY and write the logic only once?
I've thought about a couple of options which are not satisfactory in my opinion:
write a django-model-agnostic Numberplate class and make your model class inherit from both models.Model and that class
import your django models in your client (like this)

One option for you would be to use SerializerMethodField
class NumberplateSerializer(serializers.ModelSerializer):
display = serializers.SerializerMethodField()
class Meta:
model = Nameplate
fields = [..., 'display']
def get_display(self, obj):
return str(obj)
# Or you can move your business logic here directly.
# return f"{obj.number} ({obj.country_code})"
Now you should have:
>> print(api.get(id=3))
{"numberplate": "123ABC", "country_code": "BE", "display": "123ABC (BE)"}

What is the best way to create REST for nosql with Django and Titan?

Hello all and have a nice day! I wonder, how can i create REST with django-rest-framework and my nosql objects orm. For example i have bulbflow, which connects to my graph database Titan. Bulbflow allows you to make usual queries in ORM style like this:
MyNosqlobject.objects.all()
What i want is to be able to access my object via REST interface from javascript, but MyNosqlobject is not usual django model. It is actually a proxy for graph vertice.

I had to do something similar, in my case creating an API for a dynamoDB table, but I think this solution would apply to any NoSQL object, assuming it is serializable. For example, if you have a python Dict object.
In short, a lot of the cool functionality in Django Rest Framework requires Django Models, but you can still do a lot without them.
The following is an example for creating a 'list' of all objects. I am hard coding an array to make the case, but hopefully this makes it clear
from rest_framework import generics
from rest_framework import serializers
class NoSQLSerializer(serializers.BaseSerializer):
def to_representation(self, obj):
# If you don't have a json serializable object
# you can do the transformations here
return obj
class NoSQLViewSet(generics.ListAPIView):
serializer_class = NoSQLSerializer
def get_queryset(self):
if self.request.user.is_anonymous():
return None
# This is just an example, In a real NoSQL database
# you can just return the python representation of the
# object
obj_list = [
{ 'name':'foo', 'type':1 },
{ 'name':'bar', 'type':2 },
{ 'name':'foobar', 'type':1 },
]
return obj_list
I have not been able to figure out how to use (or if I can use) a Router, so I had to add to the url list as:
from .api_views import NoSQLViewSet
urlpatterns = patterns('',
url(r'^api/v1/yourObjName/', NoSQLViewSet.as_view(), name='api-nosql'),
)
but after that, all works very well, including the HTML views.
I don't know anything about Titan, but it may be that all you need to do in your case is define the get_queryset in my example as
class NoSQLViewSet(generics.ListAPIView):
serializer_class = TitanSerializer
def get_queryset(self):
return MyNosqlobject.objects.all()
and then just focus on getting a TitanSerializer to work.
Hope this helps.

Django Admin interface with pickled set

I have a model that has a pickled set of strings. (It has to be pickled, because Django has no built in set field, right?)
class Foo(models.Model):
__bar = models.TextField(default=lambda: cPickle.dumps(set()), primary_key=True)
def get_bar(self):
return cPickle.loads(str(self.__bar))
def set_bar(self, values):
self.__bar = cPickle.dumps(values)
bar = property(get_bar, set_bar)
I would like the set to be editable in the admin interface. Obviously the user won't be working with the pickled string directly. Also, the interface would need a widget for adding/removing strings from a set.
What is the best way to go about doing this? I'm not super familiar with Django's admin system. Do I need to build a custom admin widget or something?
Update: If I do need a custom widget, this looks helpful: http://www.fictitiousnonsense.com/archives/22
Update 2: Now I'm looking through different relational models to see if that will work. One idea I'm toying with:
class FooMember(models.Model):
name = models.CharField(max_length=120)
foo = models.ForeignKey('Foo')
class Foo(models.Model):
def get_names(self):
return FooMember.objects.filter(foo__exact=self)
Disadvantages of this include:
It feels excessive to make an entire model for one data field (name).
I would like the admin interface for Foo to allow the user to enter a list of strings. I'm not sure how to do that with this setup; making a custom form widget seems like less work.

Uhm. Django usually stores it's data in an SQL database. Storing a set as a pickled string is definietly not the best way to use an SQL database. It's not immediately obvious which is the right solution in your case, that depends what is in that set, but this is the wrong solution in any case.
You might want a new table for that set, or at least save it as comma separated values or something.

Serializing Foreign Key objects in Django

I have been working on developing some RESTful Services in Django to be used with both Flash and Android apps.
Developing the services interface has been quite simple, but I have been running into an issue with serializing objects that have foreign key and many to many relationships.
I have a model like this:
class Artifact( models.Model ):
name = models.CharField( max_length = 255 )
year_of_origin = models.IntegerField( max_length = 4, blank = True, null = True )
object_type = models.ForeignKey( ObjectType, blank = True, null = True )
individual = models.ForeignKey( Individual, blank = True, null = True )
notes = models.TextField( blank = True, null = True )
Then I would perform a query on this model like this, using select_related(), to be sure that foreign key relationships are followed:
artifact = Artifact.objects.select_related().get(pk=pk)
Once I have the object, I serialize it, and pass that back to my view:
serializers.serialize( "json", [ artifact ] )
This is what I get back, note that the foreign keys (object_type and individual) are just the id's to their related objects.
[
{
pk: 1
model: "artifacts.artifact"
fields: {
year_of_origin: 2010
name: "Dummy Title"
notes: ""
object_type: 1
individual: 1
}
}
]
This is great, but what I was hoping for when using select_related() was that it would automatically populate the foreign key fields with the related object, not just the object's id.
I am recent convert to Django, but put in a fair amount of time developing with CakePHP.
What I really like about the Cake ORM was that it would follow the relationships and create nested objects by default, with the ability to unbind the relationships when you were calling your query.
This made it very easy to abstract the services in a way that did not require any intervention on a case by case basis.
I see that Django does not do this by default, but is there a way to automatically serialize an object and all of it's related objects? Any tips or reading would be much appreciated.

I had a similar requirement although not for RESTful purposes. I was able to achieve what I needed by using a "full" serializing module, in my case Django Full Serializers. This is part of wadofstuff and is distributed under the new BSD license.
Wadofstuff makes this quite easy. For e.g. in your case you'd need to do the following:
First, install wadofstuff.
Second, add the following setting to your settings.py file:
SERIALIZATION_MODULES = {
'json': 'wadofstuff.django.serializers.json'
}
Third, make a slight change to the code used for serialization:
artifact = Artifact.objects.select_related().get(pk=pk)
serializers.serialize( "json", [ artifact ], indent = 4,
relations = ('object_type', 'individual',))
The key change is the relations keyword parameter. The only (minor) gotcha is to use the name of the fields forming the relation not the names of the related models.
Caveat
From the documentation:
The Wad of Stuff serializers are 100% compatible with the Django serializers when serializing a model. When deserializing a data stream the the Deserializer class currently only works with serialized data returned by the standard Django serializers.
(Emphasis added)
Hope this helps.

UPDATE:
Actually Manoj's solution is a bit outdated, Wad of Stuff's serializer has been left un-updated for some time and when I tried that, it seems that it does not support Django 1.6 anymore.
However, take a look at Django's official doc here. It does provide some way around using the built-in natural key. It seems that django's built-in serializer has a a little problem supporting using ImageField as part of the natural key. But that can be easily fixed by your self.

I'm aware this topic is years old, however, I'm sharing my solution for the people still searching for an answer (during my search, I ended up here).
Please note, I was looking for a simple function which would give me nested (foreign key) objects/dictionaries (which could contain nested (foreign key) objects/dictionaries as well) within my model/queryset which I could then convert to JSON.
In my models.py, I have a custom function (not within a model class):
Models.py
def django_sub_dict(obj):
allowed_fields = obj.allowed_fields() # pick the list containing the requested fields
sub_dict = {}
for field in obj._meta.fields: # go through all the fields of the model (obj)
if field.name in allowed_fields: # be sure to only pick fields requested
if field.is_relation: # will result in true if it's a foreign key
sub_dict[field.name] = django_sub_dict(
getattr(obj, field.name)) # call this function, with a new object, the model which is being referred to by the foreign key.
else: # not a foreign key? Just include the value (e.g., float, integer, string)
sub_dict[field.name] = getattr(obj, field.name)
return sub_dict # returns the dict generated
This function loops through all the fields in a models.Model object, if the models.Model is provided. I call the function within a model as follows (for completeness sake, including one entire model):
the same Models.py
class sheet_categories(models.Model):
id = models.AutoField(primary_key=True, unique=True)
create_date = models.DateField(auto_now_add=True)
last_change = models.DateField(auto_now=True)
name = models.CharField(max_length=128)
sheet_type = models.ForeignKey(
sheet_types, models.SET_NULL, blank=False, null=True)
balance_sheet_sort = models.IntegerField(unique=True)
def allowed_fields(self):
return [
'name',
'sheet_type',
'balance_sheet_sort',
]
def natural_key(self):
return django_sub_dict(self) # call the custom function (which is included in this models.py)
Note:
The nested JSON objects will only contain fields which are included in the allowed_fields of a model. Thus not including sensitive information.
To ultimately generate a JSON, I have the following view in my views.py.
views.py
class BalanceSheetData(ListView): # I believe this doesn't have to **be** a ListView.
model = models.sheet_categories
def get_queryset(self):
return super().get_queryset().filter() # the filter is for future purposes. For now, not relevant
def get(self, request, *args, **kwargs):
context = {
'queryset': serializers.serialize("json",
self.get_queryset(),
use_natural_foreign_keys=True, # this or the one below makes django include the natural_key() within a model. Not sure.
use_natural_primary_keys=True, # this or the one above makes django include the natural_key() within a model. Not sure.
),
}
return JsonResponse(context)
This ultimately provided me with all the nested details I required in a JSON response. Although I do not share the JSON response, as this one is barely readable.
Feel free to comment.

You can find more information on this ticket:
Allow In-depth serialization by specifying depth to follow relationship
https://code.djangoproject.com/ticket/4656

Adding a newer answer to this older question: I created and recently published django-serializable-model as an easily extensible way to serialize models, managers, and querysets. When your models extend SerializableModel, they receive an overridable .serialize method that has built-in support for all relations.
Using your example, once all of the involved models extend SerializableModel:
joins = ['object_type', 'individual']
artifact = Artifact.objects.select_related(*joins).get(pk=pk)
artifact.serialize(*joins)
Calling .serialize with the relations as arguments will have the library recurse over the related objects, calling .serialize on them as well. This returns a dictionary that looks like:
{
'id': 1,
'year_of_origin': 2010,
'name': 'Dummy Title',
'notes': '',
'object_type_id': 1,
'individual_id': 1,
'object_type': { ... nested object here ... },
'individual': { ... nested object here ... }
}
You can then call json.dumps on this dictionary to transform it to JSON.
By default, extending SerializableModel will also set the model's manager to SerializableManager (you can extend it yourself if you're using a custom manager) which uses SerializableQuerySet. This means you can call .serialize on a manager or queryset as well:
artifacts = Artifact.objects.select_related(*joins).all()
artifacts.serialize(*joins)
This simply calls .serialize on each model object in the queryset, returning a list of dictionaries in the same format as above.
django-serializable-model also allows you to easily override the default behavior on a per model basis, giving you the ability to do things like: add allowlists or denylists applied to each model's .serialize, always serialize certain joins (so you don't have to add them as arguments all the time), and more!

Django ease of building a RESTful interface

I'm looking for an excuse to learn Django for a new project that has come up. Typically I like to build RESTful server-side interfaces where a URL maps to resources that spits out data in some platform independent context, such as XML or JSON. This is
rather straightforward to do without the use of frameworks, but some of them such as Ruby on Rails conveniently allow you to easily spit back XML to a client based on the type of URL you pass it, based on your existing model code.
My question is, does something like Django have support for this? I've googled and found some 'RESTful' 3rd party code that can go on top of Django. Not sure if I'm too keen on that.
If not Django, any other Python framework that's already built with this in mind so I do not have to reinvent the wheel as I already have in languages like PHP?

This is probably pretty easy to do.
URL mappings are easy to construct, for example:
urlpatterns = patterns('books.views',
(r'^books/$', 'index'),
(r'^books/(\d+)/$', 'get'))
Django supports model serialization, so it's easy to turn models into XML:
from django.core import serializers
from models import Book
data = serializers.serialize("xml", Book.objects.all())
Combine the two with decorators and you can build fast, quick handlers:
from django.http import HttpResponse
from django.shortcuts import get_object_or_404
def xml_view(func):
def wrapper(*args, **kwargs):
result = func(*args, **kwargs)
return HttpResponse(serializers.serialize("xml", result),
mimetype="text/xml")
return wrapper
#xml_view
def index(request):
return Books.objects.all()
#xml_view
def get(request, id):
return get_object_or_404(Book, pk=id)

(I had to edit out the most obvious links.)
+1 for piston - (link above). I had used apibuilder (Washington Times open source) in the past, but Piston works easier for me. The most difficult thing for me is in figuring out my URL structures for the API, and to help with the regular expressions. I've also used surlex which makes that chore much easier.
Example, using this model for Group (from a timetable system we're working on):
class Group(models.Model):
"""
Tree-like structure that holds groups that may have other groups as leaves.
For example ``st01gp01`` is part of ``stage1``.
This allows subgroups to work. The name is ``parents``, i.e.::
>>> stage1group01 = Group.objects.get(unique_name = 'St 1 Gp01')
>>> stage1group01
>>> <Group: St 1 Gp01>
# get the parents...
>>> stage1group01.parents.all()
>>> [<Group: Stage 1>]
``symmetrical`` on ``subgroup`` is needed to allow the 'parents' attribute to be 'visible'.
"""
subgroup = models.ManyToManyField("Group", related_name = "parents", symmetrical= False, blank=True)
unique_name = models.CharField(max_length=255)
name = models.CharField(max_length=255)
academic_year = models.CharField(max_length=255)
dept_id = models.CharField(max_length=255)
class Meta:
db_table = u'timetable_group'
def __unicode__(self):
return "%s" % self.name
And this urls.py fragment (note that surlex allows regular expression macros to be set up easily):
from surlex.dj import surl
from surlex import register_macro
from piston.resource import Resource
from api.handlers import GroupHandler
group_handler = Resource(GroupHandler)
# add another macro to our 'surl' function
# this picks up our module definitions
register_macro('t', r'[\w\W ,-]+')
urlpatterns = patterns('',
# group handler
# all groups
url(r'^groups/$', group_handler),
surl(r'^group/<id:#>/$', group_handler),
surl(r'^group/<name:t>/$', group_handler),)
Then this handler will look after JSON output (by default) and can also do XML and YAML.
class GroupHandler(BaseHandler):
"""
Entry point for Group model
"""
allowed_methods = ('GET', )
model = Group
fields = ('id', 'unique_name', 'name', 'dept_id', 'academic_year', 'subgroup')
def read(self, request, id=None, name=None):
base = Group.objects
if id:
print self.__class__, 'ID'
try:
return base.get(id=id)
except ObjectDoesNotExist:
return rc.NOT_FOUND
except MultipleObjectsReturned: # Should never happen, since we're using a primary key.
return rc.BAD_REQUEST
else:
if name:
print self.__class__, 'Name'
return base.filter(unique_name = name).all()
else:
print self.__class__, 'NO ID'
return base.all()
As you can see, most of the handler code is in figuring out what parameters are being passed in urlpatterns.
Some example URLs are api/groups/, api/group/3301/ and api/group/st1gp01/ - all of which will output JSON.

Take a look at Piston, it's a mini-framework for Django for creating RESTful APIs.
A recent blog post by Eric Holscher provides some more insight on the PROs of using Piston: Large Problems in Django, Mostly Solved: APIs

It can respond with any kind of data. JSON/XML/PDF/pictures/CSV...
Django itself comes with a set of serializers.
Edit
I just had a look at at Piston — looks promising. Best feature:
Stays out of your way.
:)

Regarding your comment about not liking 3rd party code - that's too bad because the pluggable apps are one of django's greatest features. Like others answered, piston will do most of the work for you.

A little over a year ago, I wrote a REST web service in Django for a large Seattle company that does streaming media on the Internet.
Django was excellent for the purpose. As "a paid nerd" observed, the Django URL config is wonderful: you can set up your URLs just the way you want them, and have it serve up the appropriate objects.
The one thing I didn't like: the Django ORM has absolutely no support for binary BLOBs. If you want to serve up photos or something, you will need to keep them in a file system, and not in a database. Because we were using multiple servers, I had to choose between writing my own BLOB support or finding some replication framework that would keep all the servers up to date with the latest binary data. (I chose to write my own BLOB support. It wasn't very hard, so I was actually annoyed that the Django guys didn't do that work. There should be one, and preferably only one, obvious way to do something.)
I really like the Django ORM. It makes the database part really easy; you don't need to know any SQL. (I don't like SQL and I do like Python, so it's a double win.) The "admin interface", which you get for free, gives you a great way to look through your data, and to poke data in during testing and development.
I recommend Django without reservation.