Develop Reference

How to use memcache with Django when updating and retrieving model instances with children?

I have an existing Django app that doesn't do any Database caching. I am working to implement memcached to get a performance boost and reduce the amount of expensive database hits.
So far I have installed and am running memcache on my server, installed pymclib, django-memcache-admin and modified my settings.py file as I described here. I have not yet modified any of my actual application code. But I can still see that caching is in effect when I look at the django-memcache-admin dashboard. The caching is also evident because when I load some of the views, the data displayed is out of date. IE: updated data is not getting into the cache. I need advice on how to fix this. More detailed explanation is given below.
Here is are my models:
class myObjectA(models.Model):
field1 = models.CharField(max_length=255)
def modify(self):
newB = myObjectB(fk_myObjectA=self, field2="Blah Blah")
newB.save()
def getBChildren(self):
return myObjectB.objects.filter(fk_myObjectA=self)
class myObjectB(models.Model):
fk_myObjectA = models.ForeignKey(myObjectA, related_name="Blah_Blah")
field2 = models.CharField(max_length=255)
Here is the url path:
url(
r'^api/myObjectA_Modify/(?P<myObjectA_ID>\d+)/?$',
myObjectA_Modify.as_view()
),
Here is the API view that modifies an instance of myObjectA by adding a new myObjectB child record:
class myObjectA_Modify(mixins.UpdateModelMixin, generics.GenericAPIView):
queryset = myObjectA.objects.all()
serializer_class = myObjectA_Serializer
def put(self, request, *args, **kwargs):
retrieved_myObjectA = get_object_or_404(
myObjectA,
pk=request.POST["myObjectA_ID"],
)
retrieved_myObjectA.modify()
return Response(
myObjectA_Serializer(retrieved_myObjectA.getBChildren()).data,
status=status.HTTP_200_OK,
)
The call to myObjectA_Modify can be with any arbitrary ID. I don't know in advance which ID will be used. myObjectA can have an indeterminate number of myObjectB children. Furthermore there are other separate APIs that returns the whole list of all myObjectAs and myObjectBs.
How can I modify this application code to work with memcache? What should my insertion keys be? I need to make sure that if any model has a newly-inserted or updated child record, the parent record in the cache is updates. Currently once something gets into the cache it doesn't get updated so the webpages display out-of-date information. If you can show me the actual code changes to the above snippet, it would be most helpful.

A relatively simple way is to attach a function to the post_save signal of the model, and invalidate the cache if the model instance is updated.

Django using self on __init__ when not saved in database

Is it possible to use self as a reference in the __init__ method when the object is not instantiated yet?
What I'm trying to do is :
class MyClass(models.Model)
__init__(self):
some_attributes = AnotherClass.objects.filter(foreignkey=self)
The thing is that as the instance of MyClass is not registered in db yet, I have an exception like "MyClass has not attribute id"
I tried to add
if self.pk:
but it doesn't work. Is there a method like
if self.is_saved_in_db():
#some code
or do I have to created this one ?
EDIT
To be more specific, I'll give an example. I have a generic class which I try to hydrate with attributes from another Model.
class MyClass(models.Model)
_init__(self):
self.hydrate()
def hydrate(self):
# Retrieving the related objects
attributes = Information.objects.filter(...)
for attr in attributes:
attribute_id = attr.name.lower().replace(" ","_")
setattr(self,attribute_id,attr)
By doing so, I can access to attributes with MyClass.my_attribute.
For a small example, if we replace MyClass by Recipe and Information with Ingredients I can do :
pasta_recipe.pasta
pasta_recipie.tomato
pasta_recipie.onions
It's a simple parsing from a foreign_key to an attribute
By writing it, I realise that it's a bit useless because I can directly use ForeignKey relationships. I think I'll do that but for my own culture, is it possible do the filter with self as attribute before database saving ?
Thanks!

This is a very strange thing to do. I strongly recommend you do not try to do it.
(That said, the self.pk check is the correct one: you need to provide more details than "it doesn't work".)

Cleaner / reusable way to emit specific JSON for django models

I'm rewriting the back end of an app to use Django, and I'd like to keep the front end as untouched as possible. I need to be consistent with the JSON that is sent between projects.
In models.py I have:
class Resource(models.Model):
# Name chosen for consistency with old app
_id = models.AutoField(primary_key=True)
name = models.CharField(max_length=255)
#property
def bookingPercentage(self):
from bookings.models import Booking
return Booking.objects.filter(resource=self)
.aggregate(models.Sum("percent"))["percent__sum"]
And in views.py that gets all resource data as JSON:
def get_resources(request):
resources = []
for resource in Resource.objects.all():
resources.append({
"_id": resource._id,
"name": resource.first,
"bookingPercentage": resource.bookingPercentage
})
return HttpResponse(json.dumps(resources))
This works exactly as I need it to, but it seems somewhat antithetical to Django and/or Python. Using .all().values will not work because bookinPercentage is a derived property.
Another issue is that there are other similar models that will need JSON representations in pretty much the same way. I would be rewriting similar code and just using different names for the values of the models. In general is there a better way to do this that is more pythonic/djangothonic/does not require manual creation of the JSON?

Here's what I do in this situation:
def get_resources(request):
resources = list(Resource.objects.all())
for resource in resources:
resource.booking = resource.bookingPercentage()
That is, I create a new attribute for each entity using the derived property. It's only a local attribute (not stored in the database), but it's available for your json.dumps() call.

It sounds like you just want a serialisation of your models, in JSON. You can use the serialisers in core:
from django.core import serializers
data = serializers.serialize('json', Resource.objects.all(), fields=('name','_id', 'bookingPercentage'))
So just pass in your Model class, and the fields you want to serialize into your view:
get_resources(request, model_cls, fields):
documentation here https://docs.djangoproject.com/en/dev/topics/serialization/#id2

Google App Engine base and subclass gets

I want to have a base class called MBUser that has some predefined properties, ones that I don't want to be changed. If the client wants to add properties to MBUser, it is advised that MBUser be subclassed, and any additional properties be put in there.
The API code won't know if the client actually subclasses MBUser or not, but it shouldn't matter. The thinking went that we could just get MBUser by id. So I expected this to work:
def test_CreateNSUser_FetchMBUser(self):
from nsuser import NSUser
id = create_unique_id()
user = NSUser(id = id)
user.put()
# changing MBUser.get.. to NSUser.get makes this test succeed
get_user = MBUser.get_by_id(id)
self.assertIsNotNone(get_user)
Here NSUser is a subclass of MBUser. The test fails.
Why can't I do this?
What's a work around?

Models are defined by their "kind", and a subclass is a different kind, even if it seems the same.
The point of subclassing is not to share values, but to share the "schema" you've created for a given "kind".
A kind map is created on base class ndb.Model (it seems like you're using ndb since you mentioned get_by_id) and each kind is looked up when you do queries like this.
For subclasses, the kind is just defined as the class name:
#classmethod
def _get_kind(cls):
return cls.__name__

I just discovered GAE has a solution for this. It's called the PolyModel:
https://developers.google.com/appengine/docs/python/ndb/polymodelclass

Pyramid resource: In plain English

I've been reading on the ways to implement authorization (and authentication) to my newly created Pyramid application. I keep bumping into the concept called "Resource". I am using python-couchdb in my application and not using RDBMS at all, hence no SQLAlchemy. If I create a Product object like so:
class Product(mapping.Document):
item = mapping.TextField()
name = mapping.TextField()
sizes = mapping.ListField()
Can someone please tell me if this is also called the resource? I've been reading the entire documentation of Pyramids, but no where does it explain the term resource in plain simple english (maybe I'm just stupid). If this is the resource, does this mean I just stick my ACL stuff in here like so:
class Product(mapping.Document):
__acl__ = [(Allow, AUTHENTICATED, 'view')]
item = mapping.TextField()
name = mapping.TextField()
sizes = mapping.ListField()
def __getitem__(self, key):
return <something>
If I were to also use Traversal, does this mean I add the getitem function in my python-couchdb Product class/resource?
Sorry, it's just really confusing with all the new terms (I came from Pylons 0.9.7).
Thanks in advance.

I think the piece you are missing is the traversal part. Is Product
the resource? Well it depends on what your traversal produces, it
could produce products.....
Perhaps it might be best to walk this through from the view back to
how it gets configured when the application is created...
Here's a typical view.
#view_config(context=Product, permission="view")
def view_product(context, request):
pass # would do stuff
So this view gets called when context is an instance of Product. AND
if the acl attribute of that instance has the "view"
permission. So how would an instance of Product become context?
This is where the magic of traversal comes in. The very logic of
traversal is simply a dictionary of dictionaries. So one way that this
could work for you is if you had a url like
/product/1
Somehow, some resource needs to be traversed by the segments of the
url to determine a context so that a view can be determined. What if
we had something like...
class ProductContainer(object):
"""
container = ProductContainer()
container[1]
>>> <Product(1)>
"""
def __init__(self, request, name="product", parent=None):
self.__name__ = name
self.__parent__ = parent
self._request = request
def __getitem__(self, key):
p = db.get_product(id=key)
if not p:
raise KeyError(key)
else:
p.__acl__ = [(Allow, Everyone,"view")]
p.__name__ = key
p.__parent__ = self
return p
Now this is covered in the documentation and I'm attempting to boil it
down to the basics you need to know. The ProductContainer is an object
that behaves like a dictionary. The "name" and "parent"
attributes are required by pyramid in order for the url generation
methods to work right.
So now we have a resource that can be traversed. How do we tell
pyramid to traverse ProductContainer? We do that through the
Configurator object.
config = Configurator()
config.add_route(name="product",
path="/product/*traverse",
factory=ProductContainer)
config.scan()
application = config.make_wsgi_app()
The factory parameter expects a callable and it hands it the current
request. It just so happens that ProductContainer.init will do
that just fine.
This might seem a little much for such a simple example, but hopefully
you can imagine the possibilities. This pattern allows for very
granular permission models.
If you don't want/need a very granular permission model such as row
level acl's you probably don't need traversal, instead you can use
routes with a single root factory.
class RootFactory(object):
def __init__(self, request):
self._request = request
self.__acl__ = [(Allow, Everyone, "view")] # todo: add more acls
#view_config(permission="view", route_name="orders")
def view_product(context, request):
order_id, product_id = request.matchdict["order_id"], request.matchdict["product_id"]
pass # do what you need to with the input, the security check already happened
config = Configurator(root_factory=RootFactory)
config.add_route(name="orders",
path="/order/{order_id}/products/{product_id}")
config.scan()
application = config.make_wsgi_app()
note: I did the code example from memory, obviously you need all the necessary imports etc. in other words this isn't going to work as a copy/paste

Have you worked through http://michael.merickel.org/projects/pyramid_auth_demo/ ? If not, I suspect it may help. The last section http://michael.merickel.org/projects/pyramid_auth_demo/object_security.html implements the pattern you're after (note the example "model" classes inherit from nothing more complex than object).