We are trying to work with legacy DB Tables that were generated outside of Django and are not structured in an ideal way. We also can not modify the existing tables.
The DB uses the same user ID (pk) across all the tables, wether or not there is a record for that user ID. It also uses that ID as a PK on the other tables, rather than rely on them to auto increment their own IDs.
So imagine something like this below:
class Items(models.Model):
user_id = models.ForeignKey('User', db_column='UserID')
class User(models.Model):
user_id = models.IntegerField(primary_key=True)
class UserTypeA(models.Model):
user_id = models.IntegerField(primary_key=True) # Same Value as User
class UserTypeB(models.Model):
user_id = models.IntegerField(primary_key=True) # Same Value as User
What we thought of creating a relationship between Items and UserTypeA (as well as UserTypeB) is to create another field entry that uses the same column as the user_id.
class Items(models.Model):
user_id = models.ForeignKey('User', db_column='UserID')
user_type_a = models.ForeignKey('UserTypeA', db_column='UserID')
user_type_b = models.ForeignKey('UserTypeB', db_column='UserID')
This unfortunately returns a "db_column is already used" type error.
Any thoughts on how to better approach the way what we're trying to do?
A detail to note is that we're only ever reading from this databases (no updates to), so a read-only solution is fine.
Thanks,
-RB
I've solved a similar problem with this (this code should be put before the definition of your Model):
from django.db.models.signals import class_prepared
def remove_field(sender, **kwargs):
if sender.__name__ == "MyModel":
sender._meta.local_fields.remove(sender.myFKField.field)
class_prepared.connect(remove_field)
(Tested in Django 1.5.11)
Django uses local_fields to make the CREATE TABLE query.
So, I've just attached the signal class_prepared and check if sender equals the class I was expecting. If so, I've removed the field from that list.
After doing that, the CREATE TABLE query didn't include the field with same db_column and the error did not ocurr.
However the Model still working properly (with manager methods properly populating the removed field from local_fields), I can't tell the real impact of that.
Related
I am trying to build a tool that, at a simple level, tries to analyse how to buy a flat. DB = POSTGRES
So the model basically is:
class Property(models.Model):
address = CharField(max_length = 200)
price = IntegerField()
user = ForeignKey(User) # user who entered the property in the database
#..
#..
# some more fields that are common across all flats
#However, users might have their own way of analysing
# one user might want to put
estimated_price = IntegerField() # his own estimate of the price, different from the zoopla or rightmove listing price
time_to_purchase = IntegerField() # his own estimate on how long it will take to purchase
# another user might want to put other fields
# might be his purchase process requires sorting or filtering based on these two fields
number_of_bedrooms = IntegerField()
previous_owner_name = CharField()
How do I give such flexiblity to users? They should be able to sort , filter and query their own rows (in the Property table) by these custom fields. The only option I can think of now is the JSONField Postgres field
Any advice? I am surprised this is not solved readily in Django - I am sure lots of other people would have come across this problem already
Thanks
Edit: As the comments point out. JSON field is a better idea in this case.
Simple. Use Relations.
Create a model called attributes.
It will have a foreign key to a Property, a name field and a value field.
Something like,
class Attribute(models.Model):
property = models.ForiegnKey(Property)
name = models.CharField(max_length=50)
value = models.CharField(max_length=150)
Create an object each for all custom attributes of a property.
When using database queries use select_related of prefetch_related for faster response, less db operations.
When I checked group_cover table which is created by Django, there were group_id_id field and group_cover field.
I'd like to change group_id_id to group_id.
models.py
class Group(models.Model):
group_id = models.AutoField(primary_key=True)
group_name = models.CharField(max_length=50, unique=False, blank=False)
class Group_Cover(models.Model):
group_id = models.OneToOneField(Group, primary_key=True) # this create group_id_id
group_cover = models.ImageField(upload_to="/image/group/")
class Group_Member(models.Model):
user_id = models.ForeignKey(User2) # this create user_id_id
group_id = models.ForeignKey(Group) # this create group_id_id
Yeah, if I write,
group = models.OneToOneField(Group, primary_key=True)
It might work, but I may not need "_id" suffix on some field.
I read this document, but owing to my poor English, I couldn't understand the way.
Would you please teach me how to change?
Django adds an _id postix to primary keys that are generated automatically. You generally don't need to worry about them unless using a legacy data base.
Solution 2 would be the one i would recommend for a new project. Solution 1 for legacy databases.
Solution 1
To modify your existing code, use the following db_column attribute as it allows you to name the field in the database.:
group = models.AutoField(primary_key=True, db_column='group_id')
Documentation
Solution 2
To get the same results in a more "Django" way let Django generate the Primary keys automatically then reference the model in the OneToOne and Foreign key fields as shown below.
class Group(models.Model):
group_name = models.CharField(max_length=50, unique=False, blank=False)
class Group_Cover(models.Model):
group = models.OneToOneField(Group)
group_cover = models.ImageField(upload_to="/image/group/")
class Group_Member(models.Model):
user = models.ForeignKey(User2)
group = models.ForeignKey(Group)
Your assumption is correct, you need to rename your fields to not include the _id (i.e group instead of group_id). This will fix your "issue" but more than anything it more accurately represents the relationship/field. You have relationships to a model, not a reference to the id.
_id is an automatic reference provided by django to make it easier to just retrieve the _id from a model.
From the documentation
Behind the scenes, Django appends "_id" to the field name to create its database column name. In the above example, the database table for the Car model will have a manufacturer_id column. (You can change this explicitly by specifying db_column) However, your code should never have to deal with the database column name, unless you write custom SQL. You’ll always deal with the field names of your model object.
You should not worry about _id that is being added in database table. You should not deal with database if you are using ORM in Django. Also, you do not need to specify id unless its special type - group of attributes.
I would do it like this (I believe you do not need that many classes):
class Group(models.Model):
name = models.CharField(max_length=50, unique=False, blank=False)
cover = models.ImageField(upload_to="/image/group/")
users = models.ManyToManyField(User2)
Then you should access attributes with object notation. If you want id, use group.id, if you want to filter object, use Group.objects.filter(id__gt=10) or Group.objects.get(id=1) etc. My model should be doing exactly what you want to achieve.
This question has been asked before, but the answers there do not solve my problem.
I am using a legacy database, nothing can be changed
Here are my django models, with all but the relevant fields stripped off, obviously class meta has Managed=False in my actual code:
class AppCosts(models.Model):
id = models.CharField(primary_key=True)
cost = models.DecimalField()
class AppDefs(models.Model):
id = models.CharField(primary_key=True)
data = models.TextField()
appcost = models.OneToOneField(AppCosts, db_column='id')
class JobHistory(models.Model):
job_name = models.CharField(primary_key=True)
job_application = models.CharField()
appcost = models.OneToOneField(AppCosts, to_field='id', db_column='job_application')
app = models.OneToOneField(AppDefs, to_field='id', db_column='job_application')
The OneToOne fields work fine for querying, and I get the correct result using select_related()
But when I create a new record for the JobHistory table, when I call save(), I get:
DatabaseError: (1110, "Column 'job_application' specified twice")
I am using django 1.4 and I do not quite get how this OneToOneField works. I can't find any example where primary keys are named differently and has this particular semantics
I need the django model that would let me do this SQL:
select job_history.job_name, job_history.job_application, app_costs.cost from job_history, app_costs where job_history.job_application = app_costs.id;
You have defined appcost and app to have the same underlying database column, job_application, which is also the name of another existing field: so three fields share the same column. That makes no sense at all.
OneToOneFields are just foreign keys constrained to a single value on both ends. If you have foreign keys from JobHistory to AppCost and AppDef, then presumably you have actual columns in your database that contain those foreign keys. Those are the values you should be using for db_field for those fields, not "job_application".
Edit I'm glad you said you didn't design this schema, because it is pretty horrible: you won't have any foreign key constraints, for example, which makes referential integrity impossible. But never mind, we can actually achieve what you want, more or less.
There are various issues with that you have, but the main one is that you don't need the separate "job_application" field at all. That is, as I said earlier, the foreign key, so let it be that. Also note it should be an actual foreign key field, not a one-to-one, since there are many histories to one app.
One constraint that we can't achieve easily in Django is to have the same field acting as FK for two tables. But that doesn't really matter, since we can get to AppCosts via AppDefs.
So the models could just look like this:
class AppCosts(models.Model):
app = models.OneToOneField('AppDefs', primary_key=True, db_field='id')
cost = models.DecimalField()
class AppDefs(models.Model):
id = models.CharField(primary_key=True)
data = models.TextField()
class JobHistory(models.Model):
job_name = models.CharField(primary_key=True)
app = models.ForeignKey(AppDefs, db_column='job_application')
Note that I've moved the one-to-one between Costs and Defs onto AppCosts, since it seems to make sense to have the canonical ID in Defs.
Now, given a JobHistory instance, you can do history.app to get the app instance, history.app.cost to get the app cost, and use the history.app_id to get the underlying app ID from the job_application column.
If you wanted to reproduce that SQL output more exactly, something like this would now work:
JobHistory.objects.values_list('job_name', 'app_id', 'app__appcosts__cost')
Consider two django models 'User' and 'BoardGame', the latter has a ManyToMany field 'vote' defined with a custom through table:
class Vote(models.Model):
user = models.ForeignKey(User)
boardgame = models.ForeignKey(BoardGame)
vote = models.IntegerField()
I need to print a two way table having users names on the top, boardgames names on the left column and votes in the middle.
Is there a way to obtain this using django? (Remember that a user might not have voted every single boardgame.)
UPDATE: MORE DETAILS
1) Clearly this can be work out using some lines of python (which probably would result in many queries to the database), but I'm more interested in discovering if there is something directly implemented in django that could do the work. After all a ManyToMany field is nothing but a two way table (in this case with some data associated).
2) A possible 'solution' would be a FULL OUTER JOIN using a raw query, but, again, I am looking for something built-in inside django.
3) More specifically I'm using Class Based View and I was wondering if there exists an appropriate query to associate to queryset parameter of ListView.
Assuming:
class User(models.Model):
...
class BoardGame(models.Model):
users = models.ManyToManyField(User, through='Vote')
...
class Vote(models.Model):
user = models.ForeignKey(User)
boardgame = models.ForeignKey(BoardGame)
vote = models.IntegerField()
would work like this:
from django.db import connections, reset_queries
reset_queries()
users = User.objects.all().prefetch_related('vote_set')
table = []
table.append([''] + list(users))
for board_game in BoardGame.objects.all().prefetch_related('vote_set'):
row = [board_game]
for user in users:
for vote in user.vote_set.all():
if vote in board_game.vote_set.all():
row.append(vote)
break
else:
row.append('')
table.append(row)
len(connection.queries) # Returns always 4
This is not the solution you wanted, but it shows a way to get the table from the database with only 4 queries no matter how many objects you have.
I don't think there is anything in the Django core or generic Class Based Views that will render tables for you, but try django-tables2
There must be a way to do this query through the ORM, but I'm not seeing it.
The Setup
Here's what I'm modelling: one Tenant can occupy multiple rooms and one User can own multiple rooms. So Rooms have an FK to Tenant and an FK to User. Rooms are also maintained by a (possibly distinct) User.
That is, I have these (simplified) models:
class Tenant(models.Model):
name = models.CharField(max_length=100)
class Room(models.Model):
owner = models.ForeignKey(User)
maintainer = models.ForeignKey(User)
tenant = models.ForeignKey(Tenant)
The Problem
Given a Tenant, I want the Users owning a room which they occupy.
The relevant SQL query would be:
SELECT auth_user.id, ...
FROM tenants_tenant, tenants_room, auth_user
WHERE tenants_tenant.id = tenants_room.tenant_id
AND tenants_room.owner_id = auth_user.id;
Getting any individual value off the related User objects can be done with, for example, my_tenant.rooms.values_list('owner__email', flat=True), but getting a full queryset of Users is tripping me up.
Normally one way to solve it would be to set up a ManyToMany field on my Tenant model pointing at User with TenantRoom as the 'through' model. That won't work in this case, though, because the TenantRoom model has a second (unrelated) ForeignKey to User(see "restictions"). Plus it seems like needless clutter on the Tenant model.
Doing my_tenant.rooms.values_list('user', flat=True) gets me close, but returns a ValuesListQuerySet of user IDs rather than a queryset of the actual User objects.
The Question
So: is there a way to get a queryset of the actual model instances, through the ORM, using just one query?
Edit
If there is, in fact, no way to do this directly in one query through the ORM, what is the best (some combination of most performant, most idiomatic, most readable, etc.) way to accomplish what I'm looking for? Here are the options I see:
Subselect
users = User.objects.filter(id__in=my_tenant.rooms.values_list('user'))
Subselect through Python (see Performance considerations for reasoning behind this)
user_ids = id__in=my_tenant.rooms.values_list('user')
users = User.objects.filter(id__in=list(user_ids))
Raw SQL:
User.objects.all("""SELECT auth_user.*
FROM tenants_tenant, tenants_room, auth_user
WHERE tenants_tenant.id = tenants_room.tenant_id
AND tenants_room.owner_id = auth_user.id""")
Others...?
The proper way to do this is with related_name:
class Tenant(models.Model):
name = models.CharField(max_length=100)
class Room(models.Model):
owner = models.ForeignKey(User, related_name='owns')
maintainer = models.ForeignKey(User, related_name='maintains')
tenant = models.ForeignKey(Tenant)
Then you can do this:
jrb = User.objects.create(username='jrb')
bill = User.objects.create(username='bill')
bob = models.Tenant.objects.create(name="Bob")
models.Room.objects.create(owner=jrb, maintainer=bill, tenant=bob)
User.objects.filter(owns__tenant=bob)