I want to only execute a prefect_related on my queryset when a boolean value in my database is True.
I have the following models:
from django.db import models
class Map(models.Model):
has_ratings = models.BooleanField()
class Place(models.Model):
name = models.CharField()
map = models.ForeignKey(Map, on_delete=models.CASCADE)
...
class Rating(models.Model):
rating = models.IntegerField()
place = models.ForeignKey(place, on_delete=models.CASCADE)
...
Not every map has the rating functionality turned on. Therefore I only want to conditionally fetch the ratings and prevent the extra query to the Rating table for maps without ratings.
The query below always queries the Rating table.
Place.objects.all().prefetch_related(
Prefetch(
"ratings",
queryset=Rating.objects.all(),
to_attr="rating_list",
)
)
So I tried to add a filter:
Place.objects.all().prefetch_related(
Prefetch(
"ratings",
queryset=Rating.objects.all(),
to_attr="rating_list",
)
).filter(Q(map__has_rating=True))
This doesn't work because it filters all places for a Map with has_rating=False.
Is this doable without loading the map in memory? How can I can I make the prefetch optional based on a database value with Dango querysets?
Related
I have 4 tables to join; Personnels,Machines and Locations. I want to join these tables and add where clause to end of the ORM query if request body includes filtering data. Here is my models and raw query (I want to write this query in django ORM) and sample if condition for where clause;
Models ;
class Sales(models.Model):
MachineId = models.ForeignKey(Machines,on_delete=models.CASCADE,db_column='MachineId',related_name='%(class)s_Machine')
PersonnelId = models.ForeignKey(Personnels,on_delete=models.CASCADE,db_column='PersonnelId',related_name='%(class)s_Personnel')
LocationId = models.ForeignKey(Locations,on_delete=models.CASCADE,db_column='LocationId',related_name='%(class)s_Location')
class Meta:
db_table = "Sales"
class Machines(models.Model):
Name = models.CharField(max_length=200)
Fee = models.DecimalField(max_digits=10,decimal_places=3)
class Meta:
db_table = "Machines"
class Personnels(models.Model):
name = models.CharField(max_length=200)
surname = models.CharField(max_length=200)
class Meta:
db_table = "Personnels"
class Locations(models.Model):
Latitude = models.FloatField()
Longitude = models.FloatField()
LocationName = models.CharField(max_length=1000)
class Meta:
db_table = "Locations"
As you see I have 4 models. "Sales" table has foreignkeys to others. I want to get all informations in tables with using these foreign keys.(With Inner Join)
query = '''select * from "Sales" as "SL" INNER JOIN "Personnels" as "PL" ON ("SL"."PersonnelId" = "PL"."user_id") INNER JOIN "Machines" as "MC" ON ("SL"."MachineId" = "MC"."id") INNER JOIN "Locations" as "LC" ON ("SL"."LocationId" = "LC"."id") '''
if request.method=='POST':
if request.data['personnel_name'] and request.data['personnel_name'] is not None:
personnel_name = request.data['personnel_name']
condition = '''WHERE "PL"."name" = '{0}' '''.format(personnel_name)
query = query+condition
As it is seen, there are lots of quotes (if I don't write,postgresql makes some trouble) and code is not clean.
My question is, how can I write this query with using django ORM? As you see, I want to add where conditions dynamically. How can I achieve that?
I'm going to use conventional naming, with only class names captilized, and model names singular.
class Sale(models.Model):
machine = models.ForeignKey(Machine, on_delete=models.CASCADE)
person = models.ForeignKey(Person, on_delete=models.CASCADE)
location = models.ForeignKey(Location, on_delete=models.CASCADE)
db_column and db_table is useful if you have to connect the django app use an existing database. If not, django will create sensible table names by default. The table name can be different from the model field name.
To create a join where, use a queryset filter.
Sale.objects.filter(person__name='Jane Janes')
You might not need more joins, since django will perform additional queries when needed, but it can be achieved using select_related, and can give you better performance, since it reduces the total number of sql queries needed.
Sale.objects.filter(person__name='Jane Janes').select_related('machine', 'person', 'location')
It can be useful to inspect the actual SQL that will be performed when you evalute a queryset. You can do this by accessing the QuerySet.query property.
queryset = Sale.objects.select_related('machine').filter(
person__name='Jim', location__name='London')
print(queryset.query)
I have a problem with the queries when selecting values and objects. Here is a sample structure:
class Property(models.Model):
name = models.CharField(max_length=70, blank=True, verbose_name="Property Name")
type = models.CharField(max_length=10)
class Agreement(models.Model):
property = models.ForeignKey(Property, on_delete=models.CASCADE, related_name="prop")
renter = models.ForeignKey(User, verbose_name="Kiracı", related_name = "renter01")
Here is the first filter.
qs1 = Agreement.objects.all()
This one returns property and renter as objects. So I can refer the object details such as
for q in qs:
print(q.renter.firstname)
Here is the second filter.
When I need only some fields I use this filter:
qs2 = Agreement.objects.all().values('renter',...)
In that case the query returns the pk value of the renter user; and I cannot use it as object.
Is there a way that I can select certain columns and keep the objects in it as objects?
If you want renters, you should query User, not Agreement.
renters = User.objects.exclude(renter01=None)
(Note, having renter01 as the reverse relation makes no sense; unless you have a good reason, you should keep it as the default, which is agreement_set.)
Let's say I have a product that can have various child products, and the models are defined like this
class Country(models.Model):
name = models.CharField()
class Product(models.Model):
parent = models.ForeignKey(
'self', null=True, blank=True, related_name='children')
name = models.CharField()
countries = models.ManyToManyField(Country)
my goal is to retrieve all products that have one or more child products that are linked to a specific country.
In my use case I need this information as a Queryset. What I have tried is this and it works:
valid_products = []
desired_country = Country.objects.get(name='mycountry')
for product in Product.objects.all():
for child in product.children.all():
countries = child.countries.all()
for country in countries:
if country == desired_country:
valid_products.append(product.id)
desired_queryset = Product.objects.filter(pk__in=valid_products)
This method requires and additional query to convert my result into a queryset and I would like to avoid that.
Is it possible to filter a queryset like this directly with the Django ORM?
You can simply follow the relations using the double-underscore syntax. So:
desired_queryset = Product.objects.filter(children__countries= desired_country)
Consider below example:
class Customer(models.Model):
first_name = models.CharField()
last_name = models.CharField()
rental_date = models.DateTimeField()
rented_car = models.ForeignKey(Car)
class Car(models.Model):
color = models.CharField()
reg_no = models.IntegerField()
I want to group all rentals by car (assuming that a customer cannot rent more than one car but a customer can rent a car multiple times) and for each group return only the rental with the most recent rental_date and access the name of the customer and the car reg_no.
The SQL would look something like this:
SELECT * FROM Customer c where rental_date = (SELECT max(rental_date) FROM Customer WHERE rented_car_id = c.rented_car_id);
or like this:
SELECT *, max(rental_date) from Customer group by rented_car;
The resulted query-set would then be sorted by customer first_name or car reg_no and other filters may be applied (for example, getting only the blue cars or customers whose name starts with 'A').
Things that I have tried:
Aggregation:
from django.db.models Max
Customer.objects.values('rented_car').annotate(max_start_date=Max('rental_date'))
but this returns a dict containing primary keys of Car objects. I would need values from Customer too. Modifying the query to include a field from Customer (.values('rented_car', 'first_name')) would change the SQL and alter the final result.
The second method I used is raw:
Customer.objects.raw("""SELECT * FROM Customer c where rental_date = (SELECT max(rental_date) FROM Customer WHERE rented_car_id = c.rented_car_id)""")
but this returns a RawQuerySet instance that does not allow further filtering or ordering. Also, Customer.objects.filter(...).raw(...).order_by(...) will not work as the raw method will override the filtering.
Another method that could return a query-set and allow extra filtering is extra:
Customer.objects.filter(...).extra(select={
'customer_instances': """SELECT *, max(rental_date) from Customer group by rented_car"""
})
but his would always return an error (1241, 'Operand should contain 1 column(s)'). Also, from this discussion I found out that the QuerySet.extra(...) will no longer have support and aggregation should be used instead.
You usually don't need custom SQL in Django. If you do find you are in trouble formulating your database queries with the ORM, consider if your database schema could be in a form that makes it easy (and efficient) for you to get your data.
You should probably normalize your rentals data into another table.
I've also normalized Color and Brand into their own models (and therefore tables) here because you will probably wish to seek by it and display a list of available Colors and Brands in your UI. If you allow your users to input Colors in the wild you will find yourself having a 1000 variations of "Blue" spelled in all kind of wrong ways in your Car model. Imagine searching from "Blue", "blue", "blues", "bLue" etc. and displaying that horrid stuff to your users. Eh...
Your car type (Sedan, Pickup, ...) would also, probably, be one table.
class Color(models.Model):
name = models.CharField(max_length=16)
hex = models.CharField(max_length=16)
class Brand(models.Model):
name = models.CharField(max_length=16)
class Car(models.Model):
# make sure your database constraints are thought-out
reg_no = models.CharField(max_length=16, unique=True)
year = models.DateField()
color = models.ForeignKey(Color)
brand = models.ForeignKey(Brand)
class Meta:
order_by = ('reg_no', )
class Customer(models.Model):
first_name = models.CharField(max_length=64)
last_name = models.CharField(max_length=64)
class Meta:
order_by = ('first_name', 'last_name', 'pk')
class Rental(models.Model):
date = models.DateField() # if it's a date, use a DateField
car = models.ForeignKey(Car)
customer = models.ForeignKey(Customer)
class Meta:
# Let's not allow duplicate rentals for the same day
unique_together = ('date', 'car')
order_by = ('date', 'car', 'customer')
# Since your customers are already sorted by names,
# and your cars by 'reg_no', they follow that sorting.
# Your rentals are ordered by 'date' and therefore have the
# most recent date first in the table, so you can just select
# the most recent 'car' instances, which results in you getting
# the most recent 'car' rentals sorted by your dates,
# nice and easy:
Rental.objects.all().distinct('car')
# Pretty easy to search your rentals now.
# If you want to filter just use something like:
Rental.objects.filter(
date__range=(datetime(2016, 1, 1), datetime(2017, 1, 1),
car__brand__icontains='Toyota',
customer__first_name__icontains='John'
)
In Django you can usually figure out what your schema should look like by looking into your product and figuring out what kind of abstractions you have there. You seem to have at least Users, Cars, Rentals and Colors here.
You might also find the django-filter package useful here. It plays well with your UI and possible API, if you will be constructing one.
I have following models in my django app:
Product model:
class Product(BaseModel):
company = models.ForeignKey(Company, null=True, blank=True)
title = models.CharField(max_length=128,verbose_name="Product title")
Order model:
class Order(BaseModel):
company = models.ForeignKey(Company)
products = models.ManyToManyField(Product)
I am not able to add multiple same products to an order.
For example, I have product called "Car", how can I add multiple Car objects to single Order?
Every Order may contain multiple Products (same or not).
You need to use third table for that task, for example:
class OrderProduct(BaseModel):
order = models.ForeignKey(Order)
product = models.ForeignKey(Product)
quantity = models.IntegerField()
and then you can use intermediary table https://docs.djangoproject.com/en/dev/topics/db/models/#intermediary-manytomany
class Order(BaseModel):
company = models.ForeignKey(Company)
products = models.ManyToManyField(Product, through='OrderProduct')
Unlike normal many-to-many fields, you can’t use add, create, or assignment (i.e., order.products.add(prod)) to create relationships. You should manually create record in through table:
prod = Product.objects.get(uuid=product)
#order.products.add(prod)
q = product_quantity
order = order # order record must be created before
OrderProduct.objects.create(order=order, product=prod, quantity=q)
check this questions:
How can I add the same object to a ManyToMany field?
adding the same object twice to a ManyToManyField
The many to many relationship with foreign keys works backwards. You don't need a foreign key in your Order model. What you need is a foreign key in the Product model linking to Orders (Every product will have an order) :
class Product(BaseModel):
company = models.ForeignKey(Company, null=True, blank=True)
order = models.ForeignKey(Order)
title = models.CharField(max_length=128,verbose_name="Product title")