I have 4 tables to join; Personnels,Machines and Locations. I want to join these tables and add where clause to end of the ORM query if request body includes filtering data. Here is my models and raw query (I want to write this query in django ORM) and sample if condition for where clause;
Models ;
class Sales(models.Model):
MachineId = models.ForeignKey(Machines,on_delete=models.CASCADE,db_column='MachineId',related_name='%(class)s_Machine')
PersonnelId = models.ForeignKey(Personnels,on_delete=models.CASCADE,db_column='PersonnelId',related_name='%(class)s_Personnel')
LocationId = models.ForeignKey(Locations,on_delete=models.CASCADE,db_column='LocationId',related_name='%(class)s_Location')
class Meta:
db_table = "Sales"
class Machines(models.Model):
Name = models.CharField(max_length=200)
Fee = models.DecimalField(max_digits=10,decimal_places=3)
class Meta:
db_table = "Machines"
class Personnels(models.Model):
name = models.CharField(max_length=200)
surname = models.CharField(max_length=200)
class Meta:
db_table = "Personnels"
class Locations(models.Model):
Latitude = models.FloatField()
Longitude = models.FloatField()
LocationName = models.CharField(max_length=1000)
class Meta:
db_table = "Locations"
As you see I have 4 models. "Sales" table has foreignkeys to others. I want to get all informations in tables with using these foreign keys.(With Inner Join)
query = '''select * from "Sales" as "SL" INNER JOIN "Personnels" as "PL" ON ("SL"."PersonnelId" = "PL"."user_id") INNER JOIN "Machines" as "MC" ON ("SL"."MachineId" = "MC"."id") INNER JOIN "Locations" as "LC" ON ("SL"."LocationId" = "LC"."id") '''
if request.method=='POST':
if request.data['personnel_name'] and request.data['personnel_name'] is not None:
personnel_name = request.data['personnel_name']
condition = '''WHERE "PL"."name" = '{0}' '''.format(personnel_name)
query = query+condition
As it is seen, there are lots of quotes (if I don't write,postgresql makes some trouble) and code is not clean.
My question is, how can I write this query with using django ORM? As you see, I want to add where conditions dynamically. How can I achieve that?
I'm going to use conventional naming, with only class names captilized, and model names singular.
class Sale(models.Model):
machine = models.ForeignKey(Machine, on_delete=models.CASCADE)
person = models.ForeignKey(Person, on_delete=models.CASCADE)
location = models.ForeignKey(Location, on_delete=models.CASCADE)
db_column and db_table is useful if you have to connect the django app use an existing database. If not, django will create sensible table names by default. The table name can be different from the model field name.
To create a join where, use a queryset filter.
Sale.objects.filter(person__name='Jane Janes')
You might not need more joins, since django will perform additional queries when needed, but it can be achieved using select_related, and can give you better performance, since it reduces the total number of sql queries needed.
Sale.objects.filter(person__name='Jane Janes').select_related('machine', 'person', 'location')
It can be useful to inspect the actual SQL that will be performed when you evalute a queryset. You can do this by accessing the QuerySet.query property.
queryset = Sale.objects.select_related('machine').filter(
person__name='Jim', location__name='London')
print(queryset.query)
Related
I want to only execute a prefect_related on my queryset when a boolean value in my database is True.
I have the following models:
from django.db import models
class Map(models.Model):
has_ratings = models.BooleanField()
class Place(models.Model):
name = models.CharField()
map = models.ForeignKey(Map, on_delete=models.CASCADE)
...
class Rating(models.Model):
rating = models.IntegerField()
place = models.ForeignKey(place, on_delete=models.CASCADE)
...
Not every map has the rating functionality turned on. Therefore I only want to conditionally fetch the ratings and prevent the extra query to the Rating table for maps without ratings.
The query below always queries the Rating table.
Place.objects.all().prefetch_related(
Prefetch(
"ratings",
queryset=Rating.objects.all(),
to_attr="rating_list",
)
)
So I tried to add a filter:
Place.objects.all().prefetch_related(
Prefetch(
"ratings",
queryset=Rating.objects.all(),
to_attr="rating_list",
)
).filter(Q(map__has_rating=True))
This doesn't work because it filters all places for a Map with has_rating=False.
Is this doable without loading the map in memory? How can I can I make the prefetch optional based on a database value with Dango querysets?
I have two tables - Players and Buildings. For example:
class Players(models.Model):
username = models.CharField(max_length=500)
buildings = models.CharField(validators=[int_list_validator], default=list, max_length=100)
class Buildings(models.Model):
building_name = models.CharField(max_length=500)
Player can have a lot of buildings, there are in the Player.buildings field as list of ids.
I need to write a query to get building information for the player. In Postgres SQL, I got that query:
SELECT *
FROM players p
LEFT JOIN buildings b on b.id=ANY(buildings) where p.id = %s;
But how I can write that SQL query, using Python and Django ORM?
You should use singular names for your models.
class Player(models.Model):
username = models.CharField(max_length=500)
buildings = models.ManyToManyField(Building)
class Building(models.Model):
building_name = models.CharField(max_length=500)
Now when you want to access the set of buildings for your player you can do the following:
Player.objects.get("pk of player").buildings.all()
So, in Django, we have two models.
model1(models):
pk1 = models.pk(model2, on_delete=models.CASCADE, related_name='fk1')
pk2 = models.pk(model2, on_delete=models.CASCADE, related_name='fk2')
some_field = models.charField()
model2(models)
somefield = models.charfield()
I'd like to create a query to join then and match on either the first primary key or the second
The sql equivalent would be
select *
from model1
join model2 on (model2.id = model1.pk1__id or model2.id = model1.pk2__id)
for now I'm stucked with
Model1.objects.select_related('model2')
which always match on the first pk
I've tried transversing throught the foreign keys
Model1.objects.values('fk1__some_field', 'fk2__some_field')
but as you inspect the query, it shoes that it does two joins, naming the second table something like 't6'
Adding a ForeignKey to both models will give you the capability to filter through both tables and query for results. It can be done like so:
key = models.ForeignKey(ModelReference, on_delete=models.CASCADE)
The ModelReference is the model that the ForeignKey is based on.
On the other hand, if you want to create a more detailed db relationship, you can add a OneToOne or ManyToMany Fields like so:
field = models.OneToOneField(ModelReference, on_delete=models.CASCADE, primary_key=True)
car = models.ManyToManyField(Cars)
If I have the user-group relation as a ManyToMany relationship, and a group-team relation as a ManyToMany relationship, how would I find the user-team relation as a query object?
For example if I had a models.py like this:
class Group(models.Model):
name = CharField(max_length=100)
class User(models.Model):
username = models.CharField(max_length=100)
groups = models.ManyToManyField(Group, blank=True)
class Team(models.Model):
teamname = models.CharField(max_length=100)
groups = models.ManyToManyField(Group, blank=True)
I know I can do it by creating a for loop like follows, but it would be much more useful given django's architecture to have it as a query object.
My ugly solution:
user = User.objects.get(username="Ed")
users_teams = []
user_groups = Group.objects.filter(user=user)
for group in user_groups:
group_teams = Team.objects.filter(group=group)
for team in group_teams:
user_teams.append(team)
So this would give me a list query objects that I could associate with a user, but it isn't as easy to use as a single query set the contains query objects for each user-team relation.
I would prefer something that looks like this:
User.objects.get(username="Ed").groups.team_set.objects.all()
which definitely doesn't work.
Any ideas?
Team.objects.filter(groups__user__username="Ed")
which generates you:
SELECT
`api_team`.`id`,
`api_team`.`teamname`
FROM `api_team`
INNER JOIN `api_team_groups` ON (`api_team`.`id` = `api_team_groups`.`team_id`)
INNER JOIN `api_group` ON (`api_team_groups`.`group_id` = `api_group`.`id`)
INNER JOIN `api_user_groups` ON (`api_group`.`id` = `api_user_groups`.`group_id`)
INNER JOIN `api_user` ON (`api_user_groups`.`user_id` = `api_user`.`id`)
WHERE `api_user`.`username` = 'ed'
Consider below example:
class Customer(models.Model):
first_name = models.CharField()
last_name = models.CharField()
rental_date = models.DateTimeField()
rented_car = models.ForeignKey(Car)
class Car(models.Model):
color = models.CharField()
reg_no = models.IntegerField()
I want to group all rentals by car (assuming that a customer cannot rent more than one car but a customer can rent a car multiple times) and for each group return only the rental with the most recent rental_date and access the name of the customer and the car reg_no.
The SQL would look something like this:
SELECT * FROM Customer c where rental_date = (SELECT max(rental_date) FROM Customer WHERE rented_car_id = c.rented_car_id);
or like this:
SELECT *, max(rental_date) from Customer group by rented_car;
The resulted query-set would then be sorted by customer first_name or car reg_no and other filters may be applied (for example, getting only the blue cars or customers whose name starts with 'A').
Things that I have tried:
Aggregation:
from django.db.models Max
Customer.objects.values('rented_car').annotate(max_start_date=Max('rental_date'))
but this returns a dict containing primary keys of Car objects. I would need values from Customer too. Modifying the query to include a field from Customer (.values('rented_car', 'first_name')) would change the SQL and alter the final result.
The second method I used is raw:
Customer.objects.raw("""SELECT * FROM Customer c where rental_date = (SELECT max(rental_date) FROM Customer WHERE rented_car_id = c.rented_car_id)""")
but this returns a RawQuerySet instance that does not allow further filtering or ordering. Also, Customer.objects.filter(...).raw(...).order_by(...) will not work as the raw method will override the filtering.
Another method that could return a query-set and allow extra filtering is extra:
Customer.objects.filter(...).extra(select={
'customer_instances': """SELECT *, max(rental_date) from Customer group by rented_car"""
})
but his would always return an error (1241, 'Operand should contain 1 column(s)'). Also, from this discussion I found out that the QuerySet.extra(...) will no longer have support and aggregation should be used instead.
You usually don't need custom SQL in Django. If you do find you are in trouble formulating your database queries with the ORM, consider if your database schema could be in a form that makes it easy (and efficient) for you to get your data.
You should probably normalize your rentals data into another table.
I've also normalized Color and Brand into their own models (and therefore tables) here because you will probably wish to seek by it and display a list of available Colors and Brands in your UI. If you allow your users to input Colors in the wild you will find yourself having a 1000 variations of "Blue" spelled in all kind of wrong ways in your Car model. Imagine searching from "Blue", "blue", "blues", "bLue" etc. and displaying that horrid stuff to your users. Eh...
Your car type (Sedan, Pickup, ...) would also, probably, be one table.
class Color(models.Model):
name = models.CharField(max_length=16)
hex = models.CharField(max_length=16)
class Brand(models.Model):
name = models.CharField(max_length=16)
class Car(models.Model):
# make sure your database constraints are thought-out
reg_no = models.CharField(max_length=16, unique=True)
year = models.DateField()
color = models.ForeignKey(Color)
brand = models.ForeignKey(Brand)
class Meta:
order_by = ('reg_no', )
class Customer(models.Model):
first_name = models.CharField(max_length=64)
last_name = models.CharField(max_length=64)
class Meta:
order_by = ('first_name', 'last_name', 'pk')
class Rental(models.Model):
date = models.DateField() # if it's a date, use a DateField
car = models.ForeignKey(Car)
customer = models.ForeignKey(Customer)
class Meta:
# Let's not allow duplicate rentals for the same day
unique_together = ('date', 'car')
order_by = ('date', 'car', 'customer')
# Since your customers are already sorted by names,
# and your cars by 'reg_no', they follow that sorting.
# Your rentals are ordered by 'date' and therefore have the
# most recent date first in the table, so you can just select
# the most recent 'car' instances, which results in you getting
# the most recent 'car' rentals sorted by your dates,
# nice and easy:
Rental.objects.all().distinct('car')
# Pretty easy to search your rentals now.
# If you want to filter just use something like:
Rental.objects.filter(
date__range=(datetime(2016, 1, 1), datetime(2017, 1, 1),
car__brand__icontains='Toyota',
customer__first_name__icontains='John'
)
In Django you can usually figure out what your schema should look like by looking into your product and figuring out what kind of abstractions you have there. You seem to have at least Users, Cars, Rentals and Colors here.
You might also find the django-filter package useful here. It plays well with your UI and possible API, if you will be constructing one.