I'm looking for an excuse to learn Django for a new project that has come up. Typically I like to build RESTful server-side interfaces where a URL maps to resources that spits out data in some platform independent context, such as XML or JSON. This is
rather straightforward to do without the use of frameworks, but some of them such as Ruby on Rails conveniently allow you to easily spit back XML to a client based on the type of URL you pass it, based on your existing model code.
My question is, does something like Django have support for this? I've googled and found some 'RESTful' 3rd party code that can go on top of Django. Not sure if I'm too keen on that.
If not Django, any other Python framework that's already built with this in mind so I do not have to reinvent the wheel as I already have in languages like PHP?
This is probably pretty easy to do.
URL mappings are easy to construct, for example:
urlpatterns = patterns('books.views',
(r'^books/$', 'index'),
(r'^books/(\d+)/$', 'get'))
Django supports model serialization, so it's easy to turn models into XML:
from django.core import serializers
from models import Book
data = serializers.serialize("xml", Book.objects.all())
Combine the two with decorators and you can build fast, quick handlers:
from django.http import HttpResponse
from django.shortcuts import get_object_or_404
def xml_view(func):
def wrapper(*args, **kwargs):
result = func(*args, **kwargs)
return HttpResponse(serializers.serialize("xml", result),
mimetype="text/xml")
return wrapper
#xml_view
def index(request):
return Books.objects.all()
#xml_view
def get(request, id):
return get_object_or_404(Book, pk=id)
(I had to edit out the most obvious links.)
+1 for piston - (link above). I had used apibuilder (Washington Times open source) in the past, but Piston works easier for me. The most difficult thing for me is in figuring out my URL structures for the API, and to help with the regular expressions. I've also used surlex which makes that chore much easier.
Example, using this model for Group (from a timetable system we're working on):
class Group(models.Model):
"""
Tree-like structure that holds groups that may have other groups as leaves.
For example ``st01gp01`` is part of ``stage1``.
This allows subgroups to work. The name is ``parents``, i.e.::
>>> stage1group01 = Group.objects.get(unique_name = 'St 1 Gp01')
>>> stage1group01
>>> <Group: St 1 Gp01>
# get the parents...
>>> stage1group01.parents.all()
>>> [<Group: Stage 1>]
``symmetrical`` on ``subgroup`` is needed to allow the 'parents' attribute to be 'visible'.
"""
subgroup = models.ManyToManyField("Group", related_name = "parents", symmetrical= False, blank=True)
unique_name = models.CharField(max_length=255)
name = models.CharField(max_length=255)
academic_year = models.CharField(max_length=255)
dept_id = models.CharField(max_length=255)
class Meta:
db_table = u'timetable_group'
def __unicode__(self):
return "%s" % self.name
And this urls.py fragment (note that surlex allows regular expression macros to be set up easily):
from surlex.dj import surl
from surlex import register_macro
from piston.resource import Resource
from api.handlers import GroupHandler
group_handler = Resource(GroupHandler)
# add another macro to our 'surl' function
# this picks up our module definitions
register_macro('t', r'[\w\W ,-]+')
urlpatterns = patterns('',
# group handler
# all groups
url(r'^groups/$', group_handler),
surl(r'^group/<id:#>/$', group_handler),
surl(r'^group/<name:t>/$', group_handler),)
Then this handler will look after JSON output (by default) and can also do XML and YAML.
class GroupHandler(BaseHandler):
"""
Entry point for Group model
"""
allowed_methods = ('GET', )
model = Group
fields = ('id', 'unique_name', 'name', 'dept_id', 'academic_year', 'subgroup')
def read(self, request, id=None, name=None):
base = Group.objects
if id:
print self.__class__, 'ID'
try:
return base.get(id=id)
except ObjectDoesNotExist:
return rc.NOT_FOUND
except MultipleObjectsReturned: # Should never happen, since we're using a primary key.
return rc.BAD_REQUEST
else:
if name:
print self.__class__, 'Name'
return base.filter(unique_name = name).all()
else:
print self.__class__, 'NO ID'
return base.all()
As you can see, most of the handler code is in figuring out what parameters are being passed in urlpatterns.
Some example URLs are api/groups/, api/group/3301/ and api/group/st1gp01/ - all of which will output JSON.
Take a look at Piston, it's a mini-framework for Django for creating RESTful APIs.
A recent blog post by Eric Holscher provides some more insight on the PROs of using Piston: Large Problems in Django, Mostly Solved: APIs
It can respond with any kind of data. JSON/XML/PDF/pictures/CSV...
Django itself comes with a set of serializers.
Edit
I just had a look at at Piston — looks promising. Best feature:
Stays out of your way.
:)
Regarding your comment about not liking 3rd party code - that's too bad because the pluggable apps are one of django's greatest features. Like others answered, piston will do most of the work for you.
A little over a year ago, I wrote a REST web service in Django for a large Seattle company that does streaming media on the Internet.
Django was excellent for the purpose. As "a paid nerd" observed, the Django URL config is wonderful: you can set up your URLs just the way you want them, and have it serve up the appropriate objects.
The one thing I didn't like: the Django ORM has absolutely no support for binary BLOBs. If you want to serve up photos or something, you will need to keep them in a file system, and not in a database. Because we were using multiple servers, I had to choose between writing my own BLOB support or finding some replication framework that would keep all the servers up to date with the latest binary data. (I chose to write my own BLOB support. It wasn't very hard, so I was actually annoyed that the Django guys didn't do that work. There should be one, and preferably only one, obvious way to do something.)
I really like the Django ORM. It makes the database part really easy; you don't need to know any SQL. (I don't like SQL and I do like Python, so it's a double win.) The "admin interface", which you get for free, gives you a great way to look through your data, and to poke data in during testing and development.
I recommend Django without reservation.
Related
When developing a REST API using django and django rest framework, it seems to me that there's a step missing at the very end of this chain, especially if you'll design a python client of that API:
define the model of your resources in django
use DRF's serializers to send your resource over HTTP
missing: deserialize your resource back in a python model
Let me give a simple example. I declare the following django model with a bit of business logic:
class Numberplate(models.Model):
number = models.CharField(max_length=128)
country_code = models.CharField(max_length=3)
def __repr__(self):
return f"{self.number} ({self.country_code})"
DRF is super powerful when it comes to building the API around that model. However, if I call that API from a python client, it seems that the best I can get as a response from that API will be a simple json without all the logic I might have written in my model (in this case, the __repr__ method), and I'll have to navigate through it as such.
Continuing my example, here is the difference with and without a proper deserializer:
# the best we can do without rewriting code:
>> print(api.get(id=3))
{"numberplate": "123ABC", "country_code": "BE"}
# instead of re-using my __repr__, which would be ideal:
>> print(api.get(id=3))
123ABC (BE)
Is there a clean way to stay DRY and write the logic only once?
I've thought about a couple of options which are not satisfactory in my opinion:
write a django-model-agnostic Numberplate class and make your model class inherit from both models.Model and that class
import your django models in your client (like this)
One option for you would be to use SerializerMethodField
class NumberplateSerializer(serializers.ModelSerializer):
display = serializers.SerializerMethodField()
class Meta:
model = Nameplate
fields = [..., 'display']
def get_display(self, obj):
return str(obj)
# Or you can move your business logic here directly.
# return f"{obj.number} ({obj.country_code})"
Now you should have:
>> print(api.get(id=3))
{"numberplate": "123ABC", "country_code": "BE", "display": "123ABC (BE)"}
I have an existing Django app that doesn't do any Database caching. I am working to implement memcached to get a performance boost and reduce the amount of expensive database hits.
So far I have installed and am running memcache on my server, installed pymclib, django-memcache-admin and modified my settings.py file as I described here. I have not yet modified any of my actual application code. But I can still see that caching is in effect when I look at the django-memcache-admin dashboard. The caching is also evident because when I load some of the views, the data displayed is out of date. IE: updated data is not getting into the cache. I need advice on how to fix this. More detailed explanation is given below.
Here is are my models:
class myObjectA(models.Model):
field1 = models.CharField(max_length=255)
def modify(self):
newB = myObjectB(fk_myObjectA=self, field2="Blah Blah")
newB.save()
def getBChildren(self):
return myObjectB.objects.filter(fk_myObjectA=self)
class myObjectB(models.Model):
fk_myObjectA = models.ForeignKey(myObjectA, related_name="Blah_Blah")
field2 = models.CharField(max_length=255)
Here is the url path:
url(
r'^api/myObjectA_Modify/(?P<myObjectA_ID>\d+)/?$',
myObjectA_Modify.as_view()
),
Here is the API view that modifies an instance of myObjectA by adding a new myObjectB child record:
class myObjectA_Modify(mixins.UpdateModelMixin, generics.GenericAPIView):
queryset = myObjectA.objects.all()
serializer_class = myObjectA_Serializer
def put(self, request, *args, **kwargs):
retrieved_myObjectA = get_object_or_404(
myObjectA,
pk=request.POST["myObjectA_ID"],
)
retrieved_myObjectA.modify()
return Response(
myObjectA_Serializer(retrieved_myObjectA.getBChildren()).data,
status=status.HTTP_200_OK,
)
The call to myObjectA_Modify can be with any arbitrary ID. I don't know in advance which ID will be used. myObjectA can have an indeterminate number of myObjectB children. Furthermore there are other separate APIs that returns the whole list of all myObjectAs and myObjectBs.
How can I modify this application code to work with memcache? What should my insertion keys be? I need to make sure that if any model has a newly-inserted or updated child record, the parent record in the cache is updates. Currently once something gets into the cache it doesn't get updated so the webpages display out-of-date information. If you can show me the actual code changes to the above snippet, it would be most helpful.
A relatively simple way is to attach a function to the post_save signal of the model, and invalidate the cache if the model instance is updated.
I'm rewriting the back end of an app to use Django, and I'd like to keep the front end as untouched as possible. I need to be consistent with the JSON that is sent between projects.
In models.py I have:
class Resource(models.Model):
# Name chosen for consistency with old app
_id = models.AutoField(primary_key=True)
name = models.CharField(max_length=255)
#property
def bookingPercentage(self):
from bookings.models import Booking
return Booking.objects.filter(resource=self)
.aggregate(models.Sum("percent"))["percent__sum"]
And in views.py that gets all resource data as JSON:
def get_resources(request):
resources = []
for resource in Resource.objects.all():
resources.append({
"_id": resource._id,
"name": resource.first,
"bookingPercentage": resource.bookingPercentage
})
return HttpResponse(json.dumps(resources))
This works exactly as I need it to, but it seems somewhat antithetical to Django and/or Python. Using .all().values will not work because bookinPercentage is a derived property.
Another issue is that there are other similar models that will need JSON representations in pretty much the same way. I would be rewriting similar code and just using different names for the values of the models. In general is there a better way to do this that is more pythonic/djangothonic/does not require manual creation of the JSON?
Here's what I do in this situation:
def get_resources(request):
resources = list(Resource.objects.all())
for resource in resources:
resource.booking = resource.bookingPercentage()
That is, I create a new attribute for each entity using the derived property. It's only a local attribute (not stored in the database), but it's available for your json.dumps() call.
It sounds like you just want a serialisation of your models, in JSON. You can use the serialisers in core:
from django.core import serializers
data = serializers.serialize('json', Resource.objects.all(), fields=('name','_id', 'bookingPercentage'))
So just pass in your Model class, and the fields you want to serialize into your view:
get_resources(request, model_cls, fields):
documentation here https://docs.djangoproject.com/en/dev/topics/serialization/#id2
I'm having some trouble understanding how entities and keys work in Google App Engine NDB.
I have a post entity and a user entity. How do I set the user_key on post to user?
In the interactive console, I have this so far:
from google.appengine.ext import ndb
from app.lib.posts import Post
from app.lib.users import User
from random import shuffle
users = User.query()
posts = Post.query().fetch()
for post in posts:
post.user_key = shuffle(users)[0]
post.put()
I'm just trying to set up some seed data for development. I know this probably isn't the ideal way to set things, but my first question is:
How do I get a key from an entity (the reverse is described in the docs)
How do I set associations in ndb?
try:
post.user_key = shuffle(users)[0].key
Maybe this helps to understand the NDB. I had the same questions with you.
class Person(ndb.Expando):
pass
class Favourite(ndb.Expando):
pass
class Picture(ndb.Expando):
pass
person = Person()
person.put()
picture = Picture()
picture.put()
fav = Favourite(parent=person.key,
person=person.key,
picture=picture.key
)
fav.put()
Verify that shuffle works in this case, as User.query() returns an iter, not a list. (You can convert it to a list using shuffle( [ x for x in users ] ). Beware, this list could be looong.
NDB has some really wired behavior sometimes, so id recommend you dont store an NDB-Key, but its serialized string, which is also compatible to ext.db: post.user_key = shuffle( [ x for x in users ] ).key.urlsafe()
You could use KeyProperty for associations. If you need a more fine-graned control over your relations, you must implement them yourself.
See https://developers.google.com/appengine/docs/python/ndb/properties#structured
I've been reading on the ways to implement authorization (and authentication) to my newly created Pyramid application. I keep bumping into the concept called "Resource". I am using python-couchdb in my application and not using RDBMS at all, hence no SQLAlchemy. If I create a Product object like so:
class Product(mapping.Document):
item = mapping.TextField()
name = mapping.TextField()
sizes = mapping.ListField()
Can someone please tell me if this is also called the resource? I've been reading the entire documentation of Pyramids, but no where does it explain the term resource in plain simple english (maybe I'm just stupid). If this is the resource, does this mean I just stick my ACL stuff in here like so:
class Product(mapping.Document):
__acl__ = [(Allow, AUTHENTICATED, 'view')]
item = mapping.TextField()
name = mapping.TextField()
sizes = mapping.ListField()
def __getitem__(self, key):
return <something>
If I were to also use Traversal, does this mean I add the getitem function in my python-couchdb Product class/resource?
Sorry, it's just really confusing with all the new terms (I came from Pylons 0.9.7).
Thanks in advance.
I think the piece you are missing is the traversal part. Is Product
the resource? Well it depends on what your traversal produces, it
could produce products.....
Perhaps it might be best to walk this through from the view back to
how it gets configured when the application is created...
Here's a typical view.
#view_config(context=Product, permission="view")
def view_product(context, request):
pass # would do stuff
So this view gets called when context is an instance of Product. AND
if the acl attribute of that instance has the "view"
permission. So how would an instance of Product become context?
This is where the magic of traversal comes in. The very logic of
traversal is simply a dictionary of dictionaries. So one way that this
could work for you is if you had a url like
/product/1
Somehow, some resource needs to be traversed by the segments of the
url to determine a context so that a view can be determined. What if
we had something like...
class ProductContainer(object):
"""
container = ProductContainer()
container[1]
>>> <Product(1)>
"""
def __init__(self, request, name="product", parent=None):
self.__name__ = name
self.__parent__ = parent
self._request = request
def __getitem__(self, key):
p = db.get_product(id=key)
if not p:
raise KeyError(key)
else:
p.__acl__ = [(Allow, Everyone,"view")]
p.__name__ = key
p.__parent__ = self
return p
Now this is covered in the documentation and I'm attempting to boil it
down to the basics you need to know. The ProductContainer is an object
that behaves like a dictionary. The "name" and "parent"
attributes are required by pyramid in order for the url generation
methods to work right.
So now we have a resource that can be traversed. How do we tell
pyramid to traverse ProductContainer? We do that through the
Configurator object.
config = Configurator()
config.add_route(name="product",
path="/product/*traverse",
factory=ProductContainer)
config.scan()
application = config.make_wsgi_app()
The factory parameter expects a callable and it hands it the current
request. It just so happens that ProductContainer.init will do
that just fine.
This might seem a little much for such a simple example, but hopefully
you can imagine the possibilities. This pattern allows for very
granular permission models.
If you don't want/need a very granular permission model such as row
level acl's you probably don't need traversal, instead you can use
routes with a single root factory.
class RootFactory(object):
def __init__(self, request):
self._request = request
self.__acl__ = [(Allow, Everyone, "view")] # todo: add more acls
#view_config(permission="view", route_name="orders")
def view_product(context, request):
order_id, product_id = request.matchdict["order_id"], request.matchdict["product_id"]
pass # do what you need to with the input, the security check already happened
config = Configurator(root_factory=RootFactory)
config.add_route(name="orders",
path="/order/{order_id}/products/{product_id}")
config.scan()
application = config.make_wsgi_app()
note: I did the code example from memory, obviously you need all the necessary imports etc. in other words this isn't going to work as a copy/paste
Have you worked through http://michael.merickel.org/projects/pyramid_auth_demo/ ? If not, I suspect it may help. The last section http://michael.merickel.org/projects/pyramid_auth_demo/object_security.html implements the pattern you're after (note the example "model" classes inherit from nothing more complex than object).