I am making a little Time Attendance Application in Django.
I'm experimenting with models in my app. I can't figure out how I can do this:
start_time datetime NOT NULL,
finish_time datetime NULL,
duration int(11) GENERATED AS (TIMESTAMPDIFF(MINUTE, start_time, end_time)) STORED NULL,
In django.
So far, I've made a table called employees with all the employees' details:
class employees(models.Model):
employee_id = models.AutoField(primary_key=True)
first_name = models.CharField(max_length=15)
last_name = models.CharField(max_lenth=20)
user_pic_one = models.ImageField()
user_pic_two = models.ImageField()
user_pic_three = models.ImageField()
age = models.IntegerField()
national_id = models.CharField(max_length=15)
join_date = models.DateField()
pay_structure = models.CharField()
What I want to do is,
Make a new table that has the 5 columns.
employee_id (as a foreign key from the employees class we just made)
start_time = models.TimeField()
end_time = models.TimeField()
duration = models.IntegerField()
date = models.DateField(default=date.today)
So the only two things that I want to know are:
How to do the foreign key stuff to verify the employee id in the later table from the employees table.
and
calculate the time duration in minutes from the start_time to the end_time.
Thanks :)
To the best of my knowledge, at the moment of writing, Django has no builtin generated columns (as in a way to create columns that are calculated at the database side). Usually these are not necessary anyway, since we can annotate the queryset.
We can for example define a manager like:
from django.db.models import DurationField, ExpressionWrapper, F
class RegistrationManager(models.Manager):
def get_queryset(self):
return super().get_queryset().annotate(
duration=ExpressionWrapper(
F('end_time')-F('start_time'),
output_field=DurationField(null=True)
)
)
Then in our "Registration" model could then look like:
class Registration(models.Model):
employee = models.ForeignKey(Employee, on_delete=models.CASCADE)
start_time = models.DateTimeField()
end_time = models.DateTimeField(null=True)
objects = RegistrationManager()
Each time you access Registration.objects..., Django will annotate the model with an extra column duration that contains the difference between end_time and start_time. It will use a DurationField [Django-doc] for that, and thus the attributes will be timedeltas.
You can for example filter the Registration objects on the employee and duration with:
from datetime import timedelta
Registration.objects.filter(employee_id=14, duration__gte=timedelta(minutes=15))
Django will automatically add foreign key constraints on the ForeignKeyField [Django-doc]. You furthermore should specify what on_delete=... trigger [Django-doc] you want to use. We specified a foreign key to the Employee model, so Django will create a column with the name employee_id that stores the primary key of the Employee to which we refer. You can use some_registration.employee to load the related Employee object in memory. This will usually require an extra query (unless you use .select_related(..) [Django-doc] or .prefetch_related(..) [Django-doc]).
Note: Model names are normally singular and written in CamelCase, so Employee instead of employees.
Related
Django documentation states to check the ORM before writing raw SQL, but I am not aware of a resource that explains how to perform inner joins between tables without foreign keys, something that would be relatively simple to execute in a few lines of SQL.
Many of the tables in my database need to be joined by time-related properties or intervals. For example, in this basic example I need to first take a subset of change_events, and then join a different table ais_data on two separate columns:
models.py
class AisData(models.Model):
navstatus = models.BigIntegerField(blank=True, null=True)
start_ts = models.DateTimeField()
end_ts = models.DateTimeField(blank=True, null=True)
destination = models.TextField(blank=True, null=True)
draught = models.FloatField(blank=True, null=True)
geom = models.PointField(srid=4326)
imo = models.BigIntegerField(primary_key=True)
class Meta:
managed = False
db_table = 'ais_data'
unique_together = (('imo', 'start_ts'),)
class ChangeEvent(models.Model):
event_id = models.BigIntegerField(primary_key=True)
imo = models.BigIntegerField(blank=True, null=True)
timestamp = models.DateTimeField(blank=True, null=True)
class Meta:
managed = False
db_table = 'change_event'
First I take a subset of change_events which returns a QuerySet object. Using the result of this query, I need to get all of the records in ais_data that match on imo and timestamp - so the raw SQL would look exactly like this:
WITH filtered_change_events AS (
SELECT * FROM change_events WHERE timestamp BETWEEN now() - interval '1' day and now()
)
SELECT fce.*, ad.geom FROM filtered_change_events fce JOIN ais_data ad ON fce.imo = ad.imo AND fce.timestamp = ad.start_ts
views.py
from .models import ChangeEvent
from .models import AisData
from datetime import datetime, timedelta
start_period = datetime.now() - timedelta(hours=24)
end_period = datetime.now()
subset_change_events = ChangeEvent.filter(timestamp__gte=start_period,
timestamp__lte=end_period)
#inner join
subset_change_events.filter()?
How would one write this relatively simple query using the language of Django ORM? I am finding it difficult to make a simple inner join on two columns in without using a foreign key? Any advice or links to resources would be helpful.
I m working on hotel reservation app, in which i have a model for "RoomType" that contains a field for number of available rooms per day ...
I created a new model For Saving BookedDates (from, to), with ForiegnKey for the RoomType model...
Every time the user books a room (from, to), a signal will be sent the RoomType model to be saved in ManyToManyField in it ...
Now the end users will search for available rooms in specific date(from, to)
So i created a query to filter rooms available in this dates and to check if it is reserved in the specific date more times than the numberOfAvailableRoom per day..
My Problem Is that when i make a query to filter RoomTypes i .exclude() through it A Query For BookedDates models in which numberOfAvailableRoom per day field is less than number of time specific room is booked in that date ...
but actually the result of this excludes any roomType that is booked in BookedDates models!
models.py
class BookedDates(models.Model):
hotel = models.ForeignKey(Hotel, on_delete=models.CASCADE, null=True)
room_type = models.ForeignKey(
'RoomType', on_delete=models.CASCADE, null=True)
booked_from = models.DateField(null=True)
booked_to = models.DateField(null=True)
def __str__(self):
return '{} /\ {} /\ {}'.format(self.room_type, self.hotel, self.booked_from)
class RoomType(models.Model):
hotel = models.ForeignKey(Hotel, on_delete=models.CASCADE)
room_type = models.ForeignKey(RoomTypesNames, on_delete=models.CASCADE)
bookedDates = models.ManyToManyField(
BookedDates)
number_of_rooms = models.PositiveIntegerField()
views.py
roomsByDate = RoomType.objects.annotate(modulo=numAdult % F('room_capacity')).filter(
q_city, q_room_capacity1, modulo=0
).exclude(number_of_rooms__lte = BookedDates.objects.filter(room_type=F('room_type'),
booked_from__in = day).count()) ## my error is that the exclude , excludes any room type in the BookedDates model, even if it is reserved in this date number of times less than number_of_rooms available per day
Any Another Approach For Creating This Query In The Right Way Will Be Welcome
Thanks in advance
I believe you just need another annotation to calculate the number of bookings for that room. You can make use of the reverse foreign key path / related_name which in this case is roomtype_set I think.
roomsByDate = RoomType.objects.annotate(
modulo=numAdult % F('room_capacity')
).filter(
q_city, q_room_capacity1, modulo=0
).annotate(
times_booked=Count("roomtype_set__id", filter=Q(booked_from__in=day), distinct=True)
).exclude(
number_of_rooms__lte=F('times_booked')
)
Thanks For #schillingt
updating for his answer, Actually when i search by the related_name for the model, the query filters the times of its existence in the other BookedDates model regardless to the date_from query in that model ...
So,
I made the query through the ManyToManyField itself to count times of thay bookedDates field only
roomsByDate = RoomType.objects.annotate(
modulo=numAdult % F('room_capacity')
).filter(
q_city, q_room_capacity1, modulo=0
).annotate(
times_booked=Count("bookedDates ", filter=Q(booked_from__in=day), distinct=True)
).exclude(
number_of_rooms__lte=F('times_booked')
)
Currently my filter works so I can filter on name.
However I also want to filter on releases (So display game that has release.date between a certain range)
How can I do that?
filters.py
class ReleaseFilter(django_filters.FilterSet):
date = django_filters.DateFromToRangeFilter(field_name='date')
class Meta:
model = Release
fields = ['date']
class GameFilter(django_filters.FilterSet):
name = django_filters.CharFilter(lookup_expr='icontains')
releases = ReleaseFilter()
class Meta:
model = Game
fields = ['releases']
models.py
class Game(models.Model):
name = models.CharField(max_length=255)
class Release(models.Model):
game = models.ForeignKey(Game, related_name='releases', on_delete=models.CASCADE)
date = models.DateField()
What you can do is define the filter field with distinct=True:
This option can be used to eliminate duplicate results when using filters that span relationships.
Example:
release_date = django_filters.DateFromToRangeFilter(field_name='releases__date', distinct=True)
Full working example on Repl.it. Query result example: https://drf-filters-distinct--frankie567.repl.co/games/?release_date_after=2019-01-01&release_date_before=2019-12-31
releases__date = django_filters.DateFromToRangeFilter()
Did the trick for me
But the problem is if I sort on it, I see duplicates in the game list. Since it outputs a game for every release date
You can use built-in model filter to do that
Like as (for greater or equal than date)
Game.objects.filter(release__date__gte=date)
I'm using Django 1.11.6, python 3.4.2, postgresql, PyCharm 4.5.2, and windows 10 (only for development purposes).
The goal is to utilize the 'Lookups that span relationships' from the Django docs.
# models
class AlphaType(models.Model):
page_type = models.CharField(max_length=50, primary_key=True, null=False)
created_on = models.DateTimeField(auto_now_add=True)
modified_on = models.DateTimeField(auto_now=True)
class AlphaBoard(models.Model):
title = models.CharField(max_length=50)
alpha_text = models.TextField(max_length=30000)
created_on = models.DateTimeField(auto_now_add=True)
modified_on = models.DateTimeField(auto_now=True)
fk_page_type = models.ForeignKey(AlphaType, on_delete=models.CASCADE, default='general')
#views
....
q = AlphaBoard.objects.filter(fk_page_type__page_type='general')
print(q.query)
....
Just fyi, the tables have the app name prepended to the model name and the foreign key has 'id' appended to the foreign key column name.
Result of the query print.
SELECT
"alpha_alphaboard"."id", "alpha_alphaboard"."title",
"alpha_alphaboard"."alpha_text", "alpha_alphaboard"."created_on",
"alpha_alphaboard"."modified_on", "alpha_alphaboard"."fk_page_type_id"
FROM
"alpha_alphaboard"
WHERE
"alpha_alphaboard"."fk_page_type_id" = "general"
What I was expecting.
SELECT
"alpha_alphaboard"."id", "alpha_alphaboard"."title",
"alpha_alphaboard"."alpha_text", "alpha_alphaboard"."created_on",
"alpha_alphaboard"."modified_on", "alpha_alphaboard"."fk_page_type_id"
FROM
"alpha_alphaboard"
INNER JOIN "alpha_alphaboard" ON "alpha_alphatype"
"alpha_alphaboard"."fk_page_type_id" = "alpha_alphatype"."page_type"
WHERE
"alpha_alphatype"."page_type" = "general"
Questions
Why is the query ignoring the page_type relation from the filter? Look at the result of the printed query and the filter within the views. I should also add that I had a related_name="fk_page_type" within the AlphaBoard.fk_page_type, but I removed it. So a follow up question is why is it still picking up the related_name?
How do you use the "relationship" from the docs to get the expected?
Is there a way to specify the join type?
Since page_type is the primary key of the AlphaType model and its value is just written in the fk_page_type column of the AlphaBoard table, no join is needed:
q = AlphaBoard.objects.filter(fk_page_type__page_type='general')
is the same as
q = AlphaBoard.objects.filter(fk_page_type_id='general')
the field of the related model you are using in your filter is the exact foreign key value that is written in the primary table.
As for the related_name, it is used to access the reverse relation:
class AlphaBoard(models.Model):
fk_page_type = models.ForeignKey(AlphaType, on_delete=models.CASCADE, related_name='boards')
t = AlphaType(...)
boards = t.boards.all() # gives all boards with type t
boards = t.alphaboard_set.all() # default: lowermodelname_set
Consider below example:
class Customer(models.Model):
first_name = models.CharField()
last_name = models.CharField()
rental_date = models.DateTimeField()
rented_car = models.ForeignKey(Car)
class Car(models.Model):
color = models.CharField()
reg_no = models.IntegerField()
I want to group all rentals by car (assuming that a customer cannot rent more than one car but a customer can rent a car multiple times) and for each group return only the rental with the most recent rental_date and access the name of the customer and the car reg_no.
The SQL would look something like this:
SELECT * FROM Customer c where rental_date = (SELECT max(rental_date) FROM Customer WHERE rented_car_id = c.rented_car_id);
or like this:
SELECT *, max(rental_date) from Customer group by rented_car;
The resulted query-set would then be sorted by customer first_name or car reg_no and other filters may be applied (for example, getting only the blue cars or customers whose name starts with 'A').
Things that I have tried:
Aggregation:
from django.db.models Max
Customer.objects.values('rented_car').annotate(max_start_date=Max('rental_date'))
but this returns a dict containing primary keys of Car objects. I would need values from Customer too. Modifying the query to include a field from Customer (.values('rented_car', 'first_name')) would change the SQL and alter the final result.
The second method I used is raw:
Customer.objects.raw("""SELECT * FROM Customer c where rental_date = (SELECT max(rental_date) FROM Customer WHERE rented_car_id = c.rented_car_id)""")
but this returns a RawQuerySet instance that does not allow further filtering or ordering. Also, Customer.objects.filter(...).raw(...).order_by(...) will not work as the raw method will override the filtering.
Another method that could return a query-set and allow extra filtering is extra:
Customer.objects.filter(...).extra(select={
'customer_instances': """SELECT *, max(rental_date) from Customer group by rented_car"""
})
but his would always return an error (1241, 'Operand should contain 1 column(s)'). Also, from this discussion I found out that the QuerySet.extra(...) will no longer have support and aggregation should be used instead.
You usually don't need custom SQL in Django. If you do find you are in trouble formulating your database queries with the ORM, consider if your database schema could be in a form that makes it easy (and efficient) for you to get your data.
You should probably normalize your rentals data into another table.
I've also normalized Color and Brand into their own models (and therefore tables) here because you will probably wish to seek by it and display a list of available Colors and Brands in your UI. If you allow your users to input Colors in the wild you will find yourself having a 1000 variations of "Blue" spelled in all kind of wrong ways in your Car model. Imagine searching from "Blue", "blue", "blues", "bLue" etc. and displaying that horrid stuff to your users. Eh...
Your car type (Sedan, Pickup, ...) would also, probably, be one table.
class Color(models.Model):
name = models.CharField(max_length=16)
hex = models.CharField(max_length=16)
class Brand(models.Model):
name = models.CharField(max_length=16)
class Car(models.Model):
# make sure your database constraints are thought-out
reg_no = models.CharField(max_length=16, unique=True)
year = models.DateField()
color = models.ForeignKey(Color)
brand = models.ForeignKey(Brand)
class Meta:
order_by = ('reg_no', )
class Customer(models.Model):
first_name = models.CharField(max_length=64)
last_name = models.CharField(max_length=64)
class Meta:
order_by = ('first_name', 'last_name', 'pk')
class Rental(models.Model):
date = models.DateField() # if it's a date, use a DateField
car = models.ForeignKey(Car)
customer = models.ForeignKey(Customer)
class Meta:
# Let's not allow duplicate rentals for the same day
unique_together = ('date', 'car')
order_by = ('date', 'car', 'customer')
# Since your customers are already sorted by names,
# and your cars by 'reg_no', they follow that sorting.
# Your rentals are ordered by 'date' and therefore have the
# most recent date first in the table, so you can just select
# the most recent 'car' instances, which results in you getting
# the most recent 'car' rentals sorted by your dates,
# nice and easy:
Rental.objects.all().distinct('car')
# Pretty easy to search your rentals now.
# If you want to filter just use something like:
Rental.objects.filter(
date__range=(datetime(2016, 1, 1), datetime(2017, 1, 1),
car__brand__icontains='Toyota',
customer__first_name__icontains='John'
)
In Django you can usually figure out what your schema should look like by looking into your product and figuring out what kind of abstractions you have there. You seem to have at least Users, Cars, Rentals and Colors here.
You might also find the django-filter package useful here. It plays well with your UI and possible API, if you will be constructing one.