I'd like to create a directed graph in Django, but each node could be a separate model, with separate fields, etc.
Here's what I've got so far:
from bannergraph.apps.banners.models import *
class Node(models.Model):
uuid = UUIDField(db_index=True, auto=True)
class Meta:
abstract = True
class FirstNode(Node):
field_name = models.CharField(max_length=100)
next_node = UUIDField()
class SecondNode(Node):
is_something = models.BooleanField(default=False)
first_choice = UUIDField()
second_choice = UUIDField()
(obviously FirstNode and SecondNode are placeholders for the more domain-specific models, but hopefully you get the point.)
So what I'd like to do is query all the subclasses at once, returning all of the ones that match. I'm not quite sure how to do this efficiently.
Things I've tried:
Iterating over the subclasses with queries - I don't like this, as it could get quite heavy with the number of queries.
Making Node concrete. Apparently I have to still check for each subclass, which goes back to #1.
Things I've considered:
Making Node the class, and sticking a JSON blob in it. I don't like this.
Storing pointers in an external table or system. This would mean 2 queries per UUID, where I'd ideally want to have 1, but it would probably do OK in a pinch.
So, am I approaching this wrong, or forgetting about some neat feature of Django? I'd rather not use a schemaless DB if I don't have to (the Django admin is almost essential for this project). Any ideas?
The InheritanceManager from django-model-utils is what you are looking for.
You can iterate over all your Nodes with:
nodes = Node.objects.filter(foo="bar").select_subclasses()
for node in nodes:
#logic
Related
Currently I have the following Django model
class TestUser(models.Model):
name = models.CharField(max_length=250)
email = models.CharField(max_length=250)
...
class Meta:
db_table = 'temp_user'
and I've the following method.
def print_name(self):
return self.name
I want to add this method as a property to TempUser model. I know this can be done by putting the method inside TempUser class and user #property. But I want to do it dynamically.
I tried this from python shell.
In [10]: TempUser.print_name = property(print_name)
In [11]: TempUser.print_name
Out[11]: <property at 0x7efc374e7c58>
In [12]: user = TempUser.objects.get(pk=1)
In [13]: user.print_name
Out[13]: u'Test User'
But once I exit the shell, I loose the property. Is there any way to add the property permanently.
But once I exit the shell, I loose the property. Is there any way to add the property permanently.
No, not through attributes you set at run-time. Data associated with Django model instances is persisted through the database. Your only option for persisting data is to create a field in the database with which to store the information. If you want to persist methods available on a class, edit the source code.
You've expressed a desire to do this dynamically at runtime, too. However, the benefit of doing so is dubious, at best, and likely hazardous to your code base. It's hard to imagine a demonstrable use-case where this solves a real problem that does not already have a better solution. There is almost certainly a better way to get at your goal. If your goal is to keep your code DRY, consider other patterns like inheriting from abstract models.
You can implement regular properties (using the #property method decorator) on your model class that can take existing fields and look at related model fields to compose information on-the-fly, but it is still not persisted in the database...
For example, if you have a model that has a start_time and end_time you could add a total_time property without necessarily needing to create a field for it.
class MyModel(models.Model):
start_time = models.DateTimeField()
end_time = models.DateTimeField()
#property
def total_time(self):
return self.end_time - self.start_time
However, doing this would not allow you to query against this property. MyModel.objects.filter(total_time__lt=delta) for example would not be possible with a property alone.
Some other options you have include annotation and aggregation which can be done dynamically at run-time and gives you the benefit of being able to query against your database.
Using the same example as above, rather than using #property one can annotate a queryset in a similar way, which also lets you query the database on this 'virtual field' and even pass that annotated queryset around. You can even do arithmetic and aggregations like sum, average, and more...
from django.db.models import F, ExpressionWrapper, fields
duration = ExpressionWrapper(F('end_time') - F('start_time'), output_field=fields.DurationField())
qs = MyModel.objects.annotate(duration=duration)
# query for objects with a delta of more than five minutes
results = qs.filter(duration__gt=five_minutes)
See also Query expressions
In short: no, and if you could, it's almost definitely a bad idea. Stick with the established methods of doing things. Django is a very opinionated framework, it is designed for you to do things 'the django way'. You will likely only hurt yourself by going against the grain.
I have been trying to figure out the best way to automate creating multiple SQL tables based on separate but identical models, all based on the same base class. I'm basically creating pseudo message boards or walls with different Groups, and I wanted each Group to have its own db_table of Posts, each Post containing the user id, timestamp, etc.
My first thought was to have one base class of Posts and just include a field for Group name, but I thought this would be bad practice. My rationale was that one table containing every Post for all Groups would get really big (in theory anyway) and slow down filtering, and also that the extra field for group name would in the long run be a waste of memory when I could have separate tables per group and skip this field.
I've also considered using a ForeignKey with a Many-to-One relationship, but as far as I can tell this has the same drawbacks. Am I wrong to think that? Or are these size concerns not really an issue?
So my next idea was to make Posts an abstract class, and then create subclasses based on each Group. This is ultimately what I did. However, I found myself having to copy and paste the code over and over and change the class name each time. This felt very unPythonic to me. It was something like:
class Posts(models.Model):
timestamp = models.DateTimeField(auto_now_add=True, unique=False)
user_id = ...
#etc.
#
class Meta:
abstract = True
class GroupA(Posts):
class Meta(Posts.Meta):
db_table = 'groupa_board'
class GroupB(Posts):
class Meta(Posts.Meta):
db_table = 'groupb_board'
class GroupC...etc.
What I really was looking for was a factory function to do this for me. I tried this sort of thing:
def makeBoard(group):
class Board(Posts):
class Meta(Posts.Meta):
db_table = group
return board #note I tried with and without this line
And then I ran a simple for loop using a list of groups.
for group in groups:
makeBoard(group)
I found myself hitting a RuntimeError: conflicting models in application, and I probably deserved it. So then I figured what I need is something like:
def makeBoard(group):
class group(Posts): #***group here being a variable, not the class name
class Meta(Posts.Meta):
db_table = '%s' % group #maybe issues here too, but the table
return group #name is not that important if the class
#name works
But I couldn't figure out how to make this work! Is there a way to pass a variable from a list to a class name?
Anyway if you're still with me I appreciate it. I've been on stackoverflow all day and while I've found guides for creating abstract base classes and subclasses to solve similar issues, I didn't see a way to create a function to do this for me. I ultimately punted here and just make a subclass for each group by hand. If there is a way to automate this process, I'd love to hear it.
Also, if I'm being stupid for not just going with one db table containing every post, I'd like to know that too, and why! Or if there's a better way to implement this kind of system altogether. I apologize if this has been answered before, I really couldn't find it.
Thank you!
Using a single table would not be bad practice. The extra memory is minimal, on modern systems that shouldn't be a problem. You shouldn't worry about performance either, premature optimization (not including the actual system design) is considered bad practice, but if you run into performance problems you can always specify an index on the group column:
group = models.CharField(max_length=100, db_index=True)
That's not to say that it is the best option, or that your method isn't good. Also, it is entirely possible to dynamically create models, using the type() built-in function. The only difference with dynamically creating models and creating other classes is that you must specifically pass the __module__ attribute. You can create subclasses for Posts in the following way:
def fabric(names, baseclass=Posts):
for name in names:
class Meta:
db_table = '%s_table' % name.lower()
attrs = {'__module__': baseclass.__module__, 'Meta': Meta}
# specify any other class attributes here. E.g. you can specify extra fields:
attrs.update({'my_field': models.CharField(max_length=100)})
newclass = type(str(name), (baseclass,), attrs)
globals()[name] = newclass
fabric(['GroupA', 'GroupB', 'GroupC', etc...])
Put that code in your models.py after your Posts class, and all classes will be created for you. They can be used in any way normal classes can be used: Django doesn't even know you dynamically created this class. Though your Meta class doesn't inherit from Posts.Meta, your meta settings should still be preserved.
Tested with Django 1.4.
Try smth like this
import app.models as group_models
from django.db.models.base import ModelBase
def fabric(group):
for item in dir(group_models):
c = getattr(group_models, item)
if type(c) is ModelBase:
if c._meta.db_table == '%s_table' % group:
return c
return None
After building a few application on the gae platform I usually use some relationship between different models in the datastore in basically every application. And often I find my self the need to see what record is of the same parent (like matching all entry with same parent)
From the beginning I used the db.ReferenceProperty to get my relations going, like:
class Foo(db.Model):
name = db.StringProperty()
class Bar(db.Model):
name = db.StringProperty()
parentFoo = db.ReferanceProperty(Foo)
fooKey = someFooKeyFromSomePlace
bars = Bar.all()
for bar in bar:
if bar.parentFoo.key() == fooKey:
// do stuff
But lately I've abandoned this approch since the bar.parentFoo.key() makes a sub query to fetch Foo each time. The approach I now use is to store each Foo key as a string on Bar.parentFoo and this way I can string compare this with someFooKeyFromSomePlace and get rid of all the subquery overhead.
Now I've started to look at Entity groups and wondering if this is even a better way to go? I can't really figure out how to use them.
And as for the two approaches above I'm wondering is there any downsides to using them? Could using stored key string comeback and bit me in the * * *. And last but not least is there a faster way to do this?
Tip:
replace...
bar.parentFoo.key() == fooKey
with...
Bar.parentFoo.get_value_for_datastore(bar) == fooKey
To avoid the extra lookup and just fetch the key from the ReferenceProperty
See Property Class
I think you should consider this as well. This will help you fetch all the child entities of a single parent.
bmw = Car(brand="BMW")
bmw.put()
lf = Wheel(parent=bmw,position="left_front")
lf.put()
lb = Wheel(parent=bmw,position="left_back")
lb.put()
bmwWheels = Wheel.all().ancestor(bmw)
For more reference in modeling. you can refer this Appengine Data modeling
I'm not sure what you're trying to do with that example block of code, but I get the feeling it could be accomplished with:
bars = Bar.all().filter("parentFoo " = SomeFoo)
As for entity groups, they are mainly used if you want to alter multiple things in transactions, since appengine restricts that to entities within the same group only; in addition, appengine allows ancestor filters ( http://code.google.com/appengine/docs/python/datastore/queryclass.html#Query_ancestor ), which could be useful depending on what it is you need to do. With the code above, you could very easily also use an ancestor query if you set the parent of Bar to be a Foo.
If your purposes still require a lot of "subquerying" as you put it, there is a neat prefetch pattern that Nick Johnson outlines here: http://blog.notdot.net/2010/01/ReferenceProperty-prefetching-in-App-Engine which basically fetches all the properties you need in your entity set as one giant get instead of a bunch of tiny ones, which gets rid of a lot of the overhead. However do note his warnings, especially regarding altering the properties of entities while using this prefetch method.
Not very specific, but that's all the info I can give you until you be more specific about exactly what you're trying to do here.
When you design your modules you also need to consider whether you want to be able to save this within a transaction. However only do this if you need to use transactions.
An alternative approach is to assign the parent like so:
from google.appengine.ext import db
class Foo(db.Model):
name = db.StringProperty()
class Bar(db.Model):
name = db.StringProperty()
def _save_entities( foo_name, bar_name ):
"""Save the model data"""
foo_item = Foo( name = foo_name )
foo_item.put()
bar_item = Bar( parent = foo_item, name = bar_name )
bar_item.put()
def main():
# Run the save in a transaction, if any fail this should all roll back
db.run_in_transaction( _save_transaction, "foo name", "bar name" )
# to query the model data using the ancestor relationship
for item in bar_item.gql("WHERE ANCESTOR IS :ancestor", ancestor = foo_item.key()).fetch(1000):
# do stuff
I have a model that has a pickled set of strings. (It has to be pickled, because Django has no built in set field, right?)
class Foo(models.Model):
__bar = models.TextField(default=lambda: cPickle.dumps(set()), primary_key=True)
def get_bar(self):
return cPickle.loads(str(self.__bar))
def set_bar(self, values):
self.__bar = cPickle.dumps(values)
bar = property(get_bar, set_bar)
I would like the set to be editable in the admin interface. Obviously the user won't be working with the pickled string directly. Also, the interface would need a widget for adding/removing strings from a set.
What is the best way to go about doing this? I'm not super familiar with Django's admin system. Do I need to build a custom admin widget or something?
Update: If I do need a custom widget, this looks helpful: http://www.fictitiousnonsense.com/archives/22
Update 2: Now I'm looking through different relational models to see if that will work. One idea I'm toying with:
class FooMember(models.Model):
name = models.CharField(max_length=120)
foo = models.ForeignKey('Foo')
class Foo(models.Model):
def get_names(self):
return FooMember.objects.filter(foo__exact=self)
Disadvantages of this include:
It feels excessive to make an entire model for one data field (name).
I would like the admin interface for Foo to allow the user to enter a list of strings. I'm not sure how to do that with this setup; making a custom form widget seems like less work.
Uhm. Django usually stores it's data in an SQL database. Storing a set as a pickled string is definietly not the best way to use an SQL database. It's not immediately obvious which is the right solution in your case, that depends what is in that set, but this is the wrong solution in any case.
You might want a new table for that set, or at least save it as comma separated values or something.
I've got two models. One represents a piece of equipment, the other represents a possible attribute the equipment has. Semantically, this might look like:
Equipment: tractor, Attributes: wheels, towing
Equipment: lawnmower, Attributes: wheels, blades
Equipment: hedgetrimmer, Attributes: blades
I want to make queries like,
wheels = Attributes.objects.get(name='wheels')
blades = Attributes.objects.get(name='blades')
Equipment.objects.filter(has_attribute=wheels) \
.exclude(has_attribute=blades)
How can I create Django models to do this?
This seems simple, but I'm just too dense to see the right solution.
One solution that popped into my head is to encode the list of Attribute IDs in an integer list like |109|14|3 and test for attributes using Equipment.objects.filter(attributes_contains='|%d|' % id) -- but this seems really wrong.
Your second example is pretty close, but you need to understand how the QuerySet API works across relationships (i.e. joins).
class Attribute(models.Model):
name = models.CharField(max_length=20)
class Equipment(models.Model):
name = models.CharField(max_length=20)
attributes = models.ManyToManyField(Attribute)
equips = Equipment.objects.filter(
attributes__name='wheels').exclude(attributes__name='blades')
You can use Q objects in your QuerySet to do more interesting queries.
And keep in mind you can always dump the SQL from a QuerySet like this:
print equips.query.as_sql()
Sometimes you'll want to see the exact SQL being generated to make sure you're using the API correctly.