Consider below example:
class Customer(models.Model):
first_name = models.CharField()
last_name = models.CharField()
rental_date = models.DateTimeField()
rented_car = models.ForeignKey(Car)
class Car(models.Model):
color = models.CharField()
reg_no = models.IntegerField()
I want to group all rentals by car (assuming that a customer cannot rent more than one car but a customer can rent a car multiple times) and for each group return only the rental with the most recent rental_date and access the name of the customer and the car reg_no.
The SQL would look something like this:
SELECT * FROM Customer c where rental_date = (SELECT max(rental_date) FROM Customer WHERE rented_car_id = c.rented_car_id);
or like this:
SELECT *, max(rental_date) from Customer group by rented_car;
The resulted query-set would then be sorted by customer first_name or car reg_no and other filters may be applied (for example, getting only the blue cars or customers whose name starts with 'A').
Things that I have tried:
Aggregation:
from django.db.models Max
Customer.objects.values('rented_car').annotate(max_start_date=Max('rental_date'))
but this returns a dict containing primary keys of Car objects. I would need values from Customer too. Modifying the query to include a field from Customer (.values('rented_car', 'first_name')) would change the SQL and alter the final result.
The second method I used is raw:
Customer.objects.raw("""SELECT * FROM Customer c where rental_date = (SELECT max(rental_date) FROM Customer WHERE rented_car_id = c.rented_car_id)""")
but this returns a RawQuerySet instance that does not allow further filtering or ordering. Also, Customer.objects.filter(...).raw(...).order_by(...) will not work as the raw method will override the filtering.
Another method that could return a query-set and allow extra filtering is extra:
Customer.objects.filter(...).extra(select={
'customer_instances': """SELECT *, max(rental_date) from Customer group by rented_car"""
})
but his would always return an error (1241, 'Operand should contain 1 column(s)'). Also, from this discussion I found out that the QuerySet.extra(...) will no longer have support and aggregation should be used instead.
You usually don't need custom SQL in Django. If you do find you are in trouble formulating your database queries with the ORM, consider if your database schema could be in a form that makes it easy (and efficient) for you to get your data.
You should probably normalize your rentals data into another table.
I've also normalized Color and Brand into their own models (and therefore tables) here because you will probably wish to seek by it and display a list of available Colors and Brands in your UI. If you allow your users to input Colors in the wild you will find yourself having a 1000 variations of "Blue" spelled in all kind of wrong ways in your Car model. Imagine searching from "Blue", "blue", "blues", "bLue" etc. and displaying that horrid stuff to your users. Eh...
Your car type (Sedan, Pickup, ...) would also, probably, be one table.
class Color(models.Model):
name = models.CharField(max_length=16)
hex = models.CharField(max_length=16)
class Brand(models.Model):
name = models.CharField(max_length=16)
class Car(models.Model):
# make sure your database constraints are thought-out
reg_no = models.CharField(max_length=16, unique=True)
year = models.DateField()
color = models.ForeignKey(Color)
brand = models.ForeignKey(Brand)
class Meta:
order_by = ('reg_no', )
class Customer(models.Model):
first_name = models.CharField(max_length=64)
last_name = models.CharField(max_length=64)
class Meta:
order_by = ('first_name', 'last_name', 'pk')
class Rental(models.Model):
date = models.DateField() # if it's a date, use a DateField
car = models.ForeignKey(Car)
customer = models.ForeignKey(Customer)
class Meta:
# Let's not allow duplicate rentals for the same day
unique_together = ('date', 'car')
order_by = ('date', 'car', 'customer')
# Since your customers are already sorted by names,
# and your cars by 'reg_no', they follow that sorting.
# Your rentals are ordered by 'date' and therefore have the
# most recent date first in the table, so you can just select
# the most recent 'car' instances, which results in you getting
# the most recent 'car' rentals sorted by your dates,
# nice and easy:
Rental.objects.all().distinct('car')
# Pretty easy to search your rentals now.
# If you want to filter just use something like:
Rental.objects.filter(
date__range=(datetime(2016, 1, 1), datetime(2017, 1, 1),
car__brand__icontains='Toyota',
customer__first_name__icontains='John'
)
In Django you can usually figure out what your schema should look like by looking into your product and figuring out what kind of abstractions you have there. You seem to have at least Users, Cars, Rentals and Colors here.
You might also find the django-filter package useful here. It plays well with your UI and possible API, if you will be constructing one.
Related
I want to only execute a prefect_related on my queryset when a boolean value in my database is True.
I have the following models:
from django.db import models
class Map(models.Model):
has_ratings = models.BooleanField()
class Place(models.Model):
name = models.CharField()
map = models.ForeignKey(Map, on_delete=models.CASCADE)
...
class Rating(models.Model):
rating = models.IntegerField()
place = models.ForeignKey(place, on_delete=models.CASCADE)
...
Not every map has the rating functionality turned on. Therefore I only want to conditionally fetch the ratings and prevent the extra query to the Rating table for maps without ratings.
The query below always queries the Rating table.
Place.objects.all().prefetch_related(
Prefetch(
"ratings",
queryset=Rating.objects.all(),
to_attr="rating_list",
)
)
So I tried to add a filter:
Place.objects.all().prefetch_related(
Prefetch(
"ratings",
queryset=Rating.objects.all(),
to_attr="rating_list",
)
).filter(Q(map__has_rating=True))
This doesn't work because it filters all places for a Map with has_rating=False.
Is this doable without loading the map in memory? How can I can I make the prefetch optional based on a database value with Dango querysets?
I'm expanding an existing django application with new functionality.
The problem is essentially as follows.
Sorry if the equivalent of this question has already been asked, I'm not sure what to search for to find the solution.
I have a 'store' model that contains a ManyToMany relationship to a 'departments' model, not all stores have all the same departments.
I need to track daily department volumes for each store, and I need a user form to enter all this information.
I don't know if I should create one model with all possible departments, or if I should create one model with one department and weight, and then create as many objects as departments for each day.
Scale ability in this application will be very important going forward, and I'd like to start right.
Code Below.
Store departments
class Department(models.Model):
id = models.AutoField(primary_key=True)
name = models.CharField(max_length=255, unique = True)
short_name = models.CharField(max_length=32)
description = models.CharField(max_length=255)
def __str__(self):
return self.name;
Volume Data Options (Model In Question)
Option 1, one model with everything:
class VolumeData(models.Model):
id = models.AutoField(primary_key=True)
store = models.ForeignKey(stores.models.Store, on_delete=models.DO_NOTHING)
date = models.DateField(auto_now_add=True,null=False,blank=False)
produce = models.DecimalField(decimal_places=2,Null=True)
dairy = models.DecimalField(decimal_places=2,Null=True)
deli = models.DecimalField(decimal_places=2,Null=True)
meat = models.DecimalField(decimal_places=2,Null=True)
seafood = models.DecimalField(decimal_places=2,Null=True)
coffee = models.DecimalField(decimal_places=2,Null=True)
frozen = models.DecimalField(decimal_places=2,Null=True)
Option 2, one model but I need more objects.
class VolumeData(models.Model):
id = models.AutoField(primary_key=True)
store = models.ForeignKey(stores.models.Store, on_delete=models.DO_NOTHING)
date = models.DateField(auto_now_add=True,null=False,blank=False)
department = ForeignKey(Departmnet, blank=False)
I feel like option 2 will be more flexible, but I'm worried about the number of extra objects it will create.
Option 1 however, will have a lot of nulls that I don't need, and I'm not sure if that's worse, it also bakes in the departments, which may be complicated?
The Department list won't be very dynamic, I expect to update it less than once every year, and it's unlikely that an end user will ever need to modify that information.
I like option 2 as well, but I think you need to go a bit further and create a class for different items that each department sells:
class Items(models.Model):
id = models.AutoField(primary_key=True)
product_SKU = models.CharField(max_length=15) # set max_length as needed
product_name = models.CharField(max_length=50) # set max_length as needed
number_sold = models.DecimalField(decimal_places=2,Null=True)
with this added depth of information, you'll be able to incorporate a deeper level of reporting, forecasting with AI, machine learning, etc. Just my two cents.
I have 4 tables to join; Personnels,Machines and Locations. I want to join these tables and add where clause to end of the ORM query if request body includes filtering data. Here is my models and raw query (I want to write this query in django ORM) and sample if condition for where clause;
Models ;
class Sales(models.Model):
MachineId = models.ForeignKey(Machines,on_delete=models.CASCADE,db_column='MachineId',related_name='%(class)s_Machine')
PersonnelId = models.ForeignKey(Personnels,on_delete=models.CASCADE,db_column='PersonnelId',related_name='%(class)s_Personnel')
LocationId = models.ForeignKey(Locations,on_delete=models.CASCADE,db_column='LocationId',related_name='%(class)s_Location')
class Meta:
db_table = "Sales"
class Machines(models.Model):
Name = models.CharField(max_length=200)
Fee = models.DecimalField(max_digits=10,decimal_places=3)
class Meta:
db_table = "Machines"
class Personnels(models.Model):
name = models.CharField(max_length=200)
surname = models.CharField(max_length=200)
class Meta:
db_table = "Personnels"
class Locations(models.Model):
Latitude = models.FloatField()
Longitude = models.FloatField()
LocationName = models.CharField(max_length=1000)
class Meta:
db_table = "Locations"
As you see I have 4 models. "Sales" table has foreignkeys to others. I want to get all informations in tables with using these foreign keys.(With Inner Join)
query = '''select * from "Sales" as "SL" INNER JOIN "Personnels" as "PL" ON ("SL"."PersonnelId" = "PL"."user_id") INNER JOIN "Machines" as "MC" ON ("SL"."MachineId" = "MC"."id") INNER JOIN "Locations" as "LC" ON ("SL"."LocationId" = "LC"."id") '''
if request.method=='POST':
if request.data['personnel_name'] and request.data['personnel_name'] is not None:
personnel_name = request.data['personnel_name']
condition = '''WHERE "PL"."name" = '{0}' '''.format(personnel_name)
query = query+condition
As it is seen, there are lots of quotes (if I don't write,postgresql makes some trouble) and code is not clean.
My question is, how can I write this query with using django ORM? As you see, I want to add where conditions dynamically. How can I achieve that?
I'm going to use conventional naming, with only class names captilized, and model names singular.
class Sale(models.Model):
machine = models.ForeignKey(Machine, on_delete=models.CASCADE)
person = models.ForeignKey(Person, on_delete=models.CASCADE)
location = models.ForeignKey(Location, on_delete=models.CASCADE)
db_column and db_table is useful if you have to connect the django app use an existing database. If not, django will create sensible table names by default. The table name can be different from the model field name.
To create a join where, use a queryset filter.
Sale.objects.filter(person__name='Jane Janes')
You might not need more joins, since django will perform additional queries when needed, but it can be achieved using select_related, and can give you better performance, since it reduces the total number of sql queries needed.
Sale.objects.filter(person__name='Jane Janes').select_related('machine', 'person', 'location')
It can be useful to inspect the actual SQL that will be performed when you evalute a queryset. You can do this by accessing the QuerySet.query property.
queryset = Sale.objects.select_related('machine').filter(
person__name='Jim', location__name='London')
print(queryset.query)
I am doing first steps in Django and currently can't resolve further difficulties. Imagine we have next data scheme:
class Categories(models.Model):
name = models.CharField()
class Inventories(models.Model):
name = models.CharField(unique = True)
unit = models.CharField()
category = models.ForeignKey(Categories)
class Suppliers(models.Model):
name = models.CharField()
class InventoryReceiving (models.Model):
name = models.ForeignKey(Inventories, related_name='item_received')
quantity = models.DecimalField(max_digits=6, decimal_places=2)
supplier = models.ForeignKey(Suppliers, related_name = 'item_supplier')
I would like to group_by names in InventoryReceiving to get distinct values and aggregated quantity fields. Then track back relations and get a single grouped_by name table with human readable name, unit, category, supplier and quantity labels.
I came up with an expression which return name_id (I need names from the related table) and sum:
inventory_list = InventoryReceiving.objects\
.values('name')\
.annotate(agg_sum=Sum('quantity'))
I think you can just add the rest of the related data you need.
InventoryReceiving.objects.values(
'name__name', #Inventories.name
'name__unit', #Inventories.unit
'name__category__name', #Inventories.category.name
'supplier__name' #Suppliers.name
).annotate(agg_sum=Sum('quantity'))
I have following models in my django app:
Product model:
class Product(BaseModel):
company = models.ForeignKey(Company, null=True, blank=True)
title = models.CharField(max_length=128,verbose_name="Product title")
Order model:
class Order(BaseModel):
company = models.ForeignKey(Company)
products = models.ManyToManyField(Product)
I am not able to add multiple same products to an order.
For example, I have product called "Car", how can I add multiple Car objects to single Order?
Every Order may contain multiple Products (same or not).
You need to use third table for that task, for example:
class OrderProduct(BaseModel):
order = models.ForeignKey(Order)
product = models.ForeignKey(Product)
quantity = models.IntegerField()
and then you can use intermediary table https://docs.djangoproject.com/en/dev/topics/db/models/#intermediary-manytomany
class Order(BaseModel):
company = models.ForeignKey(Company)
products = models.ManyToManyField(Product, through='OrderProduct')
Unlike normal many-to-many fields, you can’t use add, create, or assignment (i.e., order.products.add(prod)) to create relationships. You should manually create record in through table:
prod = Product.objects.get(uuid=product)
#order.products.add(prod)
q = product_quantity
order = order # order record must be created before
OrderProduct.objects.create(order=order, product=prod, quantity=q)
check this questions:
How can I add the same object to a ManyToMany field?
adding the same object twice to a ManyToManyField
The many to many relationship with foreign keys works backwards. You don't need a foreign key in your Order model. What you need is a foreign key in the Product model linking to Orders (Every product will have an order) :
class Product(BaseModel):
company = models.ForeignKey(Company, null=True, blank=True)
order = models.ForeignKey(Order)
title = models.CharField(max_length=128,verbose_name="Product title")