Im trying to join two tables together before being sent to a view, using _set in a view causes 100s of queries which is highly inefficient.
example structure sites.models.py
class SiteData(models.Model):
location = models.CharField(max_length=50)
site_type = models.ForeignKey(SiteTypes, verbose_name="Site Type", \
on_delete=models.PROTECT)
bgp_as = models.CharField(max_length=6, verbose_name="BGP AS Number")
opening_date = models.DateField(verbose_name="Site opening date")
last_hw_refresh_date = models.DateField(verbose_name="Date of latest hardware refresh", \
blank=True, null=True)
is_live = models.BooleanField(default=False, verbose_name="Is this a live site?")
example structure config.models.py
class SiteSubnets(models.Model):
site_data = models.ForeignKey(SiteData, verbose_name="Location", \
on_delete=models.PROTECT, blank=True, null=True)
subnet = models.GenericIPAddressField(protocol='IPv4', \
verbose_name="Subnet", blank=True, null=True)
subnet_type = models.ForeignKey(SubnetTypes, verbose_name="Subnet Type")
vlan_id = models.IntegerField(verbose_name="Vlan ID", blank=True, null=True)
peer_desc = models.IntegerField(verbose_name="Peer description", blank=True, null=True)
site_ip = models.BooleanField(default=False, verbose_name="Is this a site supernet IP?")
class Meta:
verbose_name = "Site Subnets"
verbose_name_plural = "Site Subnets"
Queries:
site_subnets = SiteSubnets.objects.only('subnet').filter(site_ip=True)
site_data = SiteData.objects.only('location','is_live','bgp_as','postcode','opening_date','live_link_type')
Desired Outcome example:
Location | Subnet | BGP AS
---------------------------------
London | 10.10.10.0 | 65001
Manchester | 10.10.20.0 | 65002
...
I cant do a select_related without having the SitesSubnet table as the main table, as when I do it on site data, I get
django.core.exceptions.FieldError: Invalid field name(s) given in select_related: 'site_subnets'. Choices are: site_type
If I use the SiteSubnet as the main table, if a Site does not have a SiteSubnet.site_up I wont get the Site info. displayed
does anyone know a way around this that will display all the data and not run n+1 queries?
EDIT:
prefetch also fails with the below error:
AttributeError: Cannot find 'site_subnets_set' on SiteData object, 'site_subnets_set' is an invalid parameter to prefetch_related()
I would be sending the data to a template to be accessed in a loop i.e
<table>
<tr>
<td>Location</td>
<td>Subnet</td>
<td>BGP AS</td>
<tr>
{%for site in sitedata %}
<tr>
<td>{{ site.location }}</td>
<td>{{ site.subnet }}</td>
<td>{{ site.bg_as }}</td>
<tr>
{% endfor %}
Thanks
You can use prefetch_related to prefetch sitesubnets for the sitedata queryset.
SiteData.objects.prefetch_related('sitesubnets_set')
Related
I am working on a Django project with 2 Models; Customer and Deposit and I want to display a list of Customers with their names, deposited dated, account number, and Total Deposited within the year so how do I do it the right way.
See what I have tried but Couldn't get the Customers' name in my Django Templates.
Models:
class Customer(models.Model):
surname = models.CharField(max_length=10, null=True)
othernames = models.CharField(max_length=20, null=True)
account_number = models.CharField(max_length=10, null=True)
address = models.CharField(max_length=50, null=True)
phone = models.CharField(max_length=11, null=True)
date = models.DateTimeField(auto_now_add=True, null=True)
#Get the url path of the view
def get_absolute_url(self):
return reverse('customer_create', args=[self.id])
#Making Sure Django Display the name of our Models as it is without Pluralizing
class Meta:
verbose_name_plural = 'Customer'
#
def __str__(self):
return f'{self.surname} {self.othernames} - {self.account_number}'
class Deposit(models.Model):
customer = models.ForeignKey(Customer, on_delete=models.CASCADE, null=True)
acct = models.CharField(max_length=6, null=True)
staff = models.ForeignKey(User, on_delete=models.CASCADE, null=True)
deposit_amount = models.PositiveIntegerField(null=True)
date = models.DateTimeField(auto_now_add=True)
def get_absolute_url(self):
return reverse('create_account', args=[self.id])
def __str__(self):
return f'{self.customer} Deposited {self.deposit_amount} by {self.staff.username}'
here is my view code:
def customer_list(request):
#Get Current Date
current_date = datetime.now().date()
#Get Current Month Name from Calendar
current_month_name = calendar.month_name[date.today().month]
group_deposits = Deposit.objects.filter(date__year=current_date.year).order_by('acct')
grouped_customer_deposit = group_deposits.values('acct').annotate(total `=Sum('deposit_amount')).order_by()`
context = { 'customers':grouped_customer_deposit,}
Here is how I tried to display the result in Django Template:
{% for deposit in customers %}
<tr>
<td>{{ forloop.counter }}</td>
<td>{{ deposit.acct }}</td>
<td>{{ deposit.customer.surname }}</td>
<td>{{ deposit.total }}</td>
<td>{{ customer.deposit.date }}</td>
<th scope="row"><a class="btn btn-info btn-sm" href=" ">Deposit</a></th>
</tr>
{% endfor %}
Someone should graciously help with the most efficient way of getting the Total Deposit for each customer with their names, account number, and date deposited. Thank in anticipation for your kind answers.
As you are using the values method:
grouped_customer_deposit = group_deposits.values('acct').annotate(total `=Sum('deposit_amount')).order_by()`
Django will return a queryset that returns dictionaries rather than model instances. See Django Queryset API reference for more details.
This means that you cannot reference your model relationships as you are trying to do here:
<td>{{ deposit.customer.surname }}</td>
as that information will not be in the dictionary passed to the template (you can do a print of grouped_customer_deposit in your view, or display it in your template -- {{ customers }} -- to see what I mean).
Ergo, if you remove your call to values() then your template will receive model instances which should solve your problem, and make your template work as-is.
grouped_customer_deposit = group_deposits.annotate(total `=Sum('deposit_amount')).order_by()`
(Although your syntax for your annotate is not correct, so I'm assuming that was a mistake when you pasted your code here.)
I have gathered data from two tables that is Orders and Purchases (joined) and saved that in dictionary, when I'm printing the qty on the console its giving me the value but when I'm trying to print the values on template it not showing up however columns from the orders table like first name last name status address is shown on the template but qty and product from the purchases table is not showing up on the template but can be easily printed on console
can somebody help???
Models.py
Dispatcher = models.ForeignKey(Dispatchers , null=True, on_delete=models.SET_NULL )
Firstname = models.CharField(max_length=200 , null=True)
Lastname = models.CharField(max_length=200 , null=True)
Email = models.CharField(max_length=200 , null=True)
Address = models.CharField(max_length=200 , null=True)
Country = models.CharField(max_length=200 , null=True)
State = models.CharField(max_length=200 , null=True)
status = models.CharField(max_length=200 , null=True)
def __str__(self):
return self.Lastname
class purchases(models.Model):
orders_id = models.ForeignKey(Orders , null=True, on_delete=models.SET_NULL )
product = models.CharField(max_length=200 , null=True)
qty = models.CharField(max_length=200 , null=True)
price = models.CharField(max_length=200 , null=True)
def __str__(self):
return self.product
views.py file
dispatcherOrders = {}
orders = request.user.dispatchers.orders_set.all()
for order in orders:
dispatcherOrders[order] = purchases.objects.get(orders_id = order.id)
print(dispatcherOrders[order].qty)
return render(request , 'dispatcherOrders.html',{'dispatcherOrders' : dispatcherOrders})
dispatcherOrders.html
{%for Orders in dispatcherOrders%}
<tr>
<th scope="row">1</th>
<td>{{Orders.Firstname}}</td>
<td>{{Orders.Lastname}}</td>
<td>{{Orders.Address}}</td>
<td>{{Orders.product}}</td>
<td>{{Orders.qty}}</td>
<td>{{Orders.status}}</td>
</tr>
{% endfor %}
In your code, dispatcherOrders dictionary keys are orders and dictionary values are purchases:
dispatcherOrders[order] = purchases.objects.get(orders_id = order.id)
yet you are iterating only on dictionary keys, the orders:
{%for Orders in dispatcherOrders%}
To get the info from purchases, iterate the dictionary items too.
So your template code should be similar to
{%for Orders, Purchase in dispatcherOrders.items %}
<tr>
<th scope="row">1</th>
<td>{{Orders.Firstname}}</td>
<td>{{Orders.Lastname}}</td>
<td>{{Orders.Address}}</td>
<td>{{Purchase.product}}</td>
<td>{{Purchase.qty}}</td>
<td>{{Purchase.status}}</td>
</tr>
{% endfor %}
Here's how .items work: https://stackoverflow.com/a/6285769/3694363
I am trying to get details of two querysets and display on my datatable. I can display details of first queryset but not of the second which is also dependent on the first queryset.
My views.py function;
def truck_list(request):
q_data = quar_contacts.objects.all().filter(.....)
hot_details = []
for d in q_data:
h_details = truck_contacts.objects.filter(p_contacts=d.id)
hot_details.append(h_details)
my_list_data = zip(q_data, hot_details)
data = {'quar_data': my_list_data}
return render(request, 'syst/truck_list.html', data)
I can print out details of the zip list as a tuple i.e data = tuple(my_list_data) and see the data.
Then my truck_list template on the datatable body section :
<tbody>
{% for q_data, hot_details in quar_data %}
<tr>
<td>{{ q_data.last_name }}</td>
<td>{{ q_data.first_name }}</td>
<td>{{ q_data.age }}</td>
<td>{{ q_data.sex }}</td>
<td>{{ q_data.phone_number }}</td>
<td>{{ hot_details.hotel_name}}</td>
<td>{{ hot_details.date_check_in }}</td>
<td>{{ hot_details.created_by }}</td>
</tr>
{% endfor %}
</tbody>
How can i get the hot_details (hotel_name and date_checked_in) displayed on my datatable. How can I loop for each contact what hotel details are there.
Is there an easier way of achieving this?
My Models:
class quar_contacts(models.Model):
first_name = models.CharField(max_length=50)
middle_name = models.CharField(max_length=50, blank=True)
last_name = models.CharField(max_length=50)
sex = models.CharField(max_length=50)
dob = models.DateField(default=date.today)
passport_number = models.CharField(max_length=50, blank=True)
created_by = models.ForeignKey(User, on_delete=models.DO_NOTHING, related_name='quar_updated_by')
class truck_contacts(models.Model):
patient_contacts = models.ForeignKey(quar_contacts, on_delete=models.DO_NOTHING, related_name='truck_contact')
vehicle_registration = models.CharField(max_length=50, blank=True)
company_name = models.CharField(max_length=50, blank=True)
hotel = models.CharField(max_length=50, blank=True)
hotel_town = models.CharField(max_length=50, blank=True)
date_check_in = models.DateField(default=date.today)
date_check_out = models.DateField(default=date.today)
Also, how do you get the 'created_by' value which is a foreign key of the Users table?
You can use annotate to add additional fields to each object from quar_contacts queryset. I don't know your db schema, but if "quar contact" has a foreign key to "truck contact" it can be something like that:
from django.db.models import F
q_data = quar_contacts.objects.all().filter(.....).annotate(
hotel_name=F("your_relation_name__hotel_name"),
date_check_in=F("your_relation_name__date_check_in"),
)
Then you can access hotel_name and date_check_in just like other fields from this objects.
Please let me know if it's helpful. If you have different relation between those models I'll update my answer.
Using "annotate" I have managed to to add the fields the other way. getting 'truck_contacts' queryset then look for the parent field values 'quar_contacts'. Thanks to #rafaljusiak idea above.
all_data = truck_contacts.objects.all().annotate(
first_name=F("patient_contacts__first_name"),
last_name=F("patient_contacts__last_name"),
sex=F("patient_contacts__sex"),
age=F("patient_contacts__dob"),
passport_number=F("patient_contacts__passport_number"),
phone_number=F("patient_contacts__phone_number"),
created_by=F("patient_contacts__created_by"),
)
But am still struggling how to display 'created by' being a foreign key of Users table
I have two tables one called called PlayerLkup, which contains info about the player and serves as a lookup table. I have another table (called BattingStats) which contains a player's stats (and the playerid). The BattingStats table is a one-to-many (playerid is listed multiple times, one time for each season they played).
My data from PlayerLkup is displaying fine, and it is using the playerid in the URL address to retreive a specific player. My question is how do I use the data from my BattingStats model/table onto that same page? I'm assuming my views page is where the work needs to be done. Is there a way to have multiple models passed into one view? I've tried same url, different views. It didnt seem to work for me. What do I need to do? Any help here would be appreciated.
I posted this question before (and have since deleted it) but someone mistakenly tagged it as a duplicate so it didn't receive any attention.
models.py
class BattingStats(models.Model):
playerid = models.CharField(db_column='playerID', max_length=9)
year = models.IntegerField(db_column='Year', blank=True, null=True)
g = models.IntegerField(db_column='G', blank=True, null=True)
ab = models.IntegerField(db_column='AB', blank=True, null=True)
r = models.IntegerField(db_column='R', blank=True, null=True)
hr = models.IntegerField(db_column='HR', blank=True, null=True)
rbi = models.IntegerField(db_column='RBI', blank=True, null=True)
sb = models.IntegerField(db_column='SB', blank=True, null=True)
class PlayerLkup(models.Model):
playerid = models.CharField(db_column='playerID', primary_key=True, max_length=255)
birthyear = models.IntegerField(db_column='birthYear', blank=True, null=True)
birthmonth = models.IntegerField(db_column='birthMonth', blank=True, null=True)
birthday = models.IntegerField(db_column='birthDay', blank=True, null=True)
birthstate = models.CharField(db_column='birthState', max_length=255, blank=True, null=True)
birthcity = models.CharField(db_column='birthCity', max_length=255, blank=True, null=True)
namefirst = models.CharField(db_column='nameFirst', max_length=255, blank=True, null=True)
namelast = models.CharField(db_column='nameLast', max_length=255, blank=True, null=True)
weight = models.IntegerField(blank=True, null=True)
height = models.IntegerField(blank=True, null=True)
views.py
def player_info(request, playerid):
playerdata = PlayerLkup.objects.get(playerid=playerid)
return render(request, 'careerstats/playerpage.html', {'playerdata': playerdata})
urls.py
urlpatterns = [
path('player/<str:playerid>/', views.player_info, name='player_info'),
You can do whatever you like in the view, and pass as many models as you like in the context.
But in your case you don't actually need to change the view, but you should change the models. It looks like BattingStats should have a relationship to PlayerLkup, via the playerid field. You should make this a ForeignKey (and call it player) - note, you don't actually need to change the underlying tables at all, which may be important since this appears to be a legacy db.
class BattingStats(models.Model):
player = models.ForeignKey('PlayerLkup', db_column='playerID')
Now in your template you can do:
{{ playerdata.namefirst }}
{% for stat in playerdata.battingstats_set.all %}
{{ stat.year }}
{{ stat.g }}
...
{% endfor %}
(Also, please give better names to your fields. They don't need to be the same as the db column, that's what the db_column attribute is for. There's no reason to use single-character field names like g.)
Just in case you don't want to change your model (I would) you can:
def player_info(request, playerid):
playerdata = PlayerLkup.objects.get(playerid=playerid)
battingdata = BattingStats.objects.get(playerid=playerid)
data = {
'playerdata': playerdata,
'battingdata': battingdata
}
return render(request, 'careerstats/playerpage.html', data)
In your template, you will both playerdata and battingdata available as template variables.
I have the following Models in Django.
from django.db import models
#Show DB Table Model
class Shows(models.Model):
show_key = models.CharField(primary_key=True, max_length=7)
show_date = models.DateField(blank=True, null=True)
show_venue = models.CharField(max_length=50, blank=True, null=True)
show_city = models.CharField(max_length=50, blank=True, null=True)
show_state = models.CharField(max_length=3, blank=True, null=True)
show_country = models.CharField(max_length=3, blank=True, null=True)
class Meta:
managed = False
db_table = 'shows'
#Songs DB Table Model
class Songs(models.Model):
song_key = models.CharField(primary_key=True, max_length=8)
show_key = models.ForeignKey('Shows', models.DO_NOTHING, db_column='show_key', blank=True, null=True)
song_name = models.CharField(max_length=100, blank=True, null=True)
song_set = models.CharField(max_length=20, blank=True, null=True)
song_track = models.IntegerField(blank=True, null=True)
song_encore = models.IntegerField(blank=True, null=True)
song_segue = models.CharField(max_length=1, blank=True, null=True)
song_notes = models.CharField(max_length=100, blank=True, null=True)
song_cover = models.CharField(max_length=50, blank=True, null=True)
song_with_guest = models.CharField(max_length=50, blank=True, null=True)
class Meta:
managed = False
db_table = 'songs'
I am trying make a query that will find all objects meeting a certain criteria, ie:
Shows.objects.filter(show_date__year=2000)
This above query would return multiple objects.
I need to take it a step further and pull all of the information from the Songs table/model relating to the filtered Show objects. The models are related in the sense that the "show_key" is a primary key / foreign key relationship and is one to many.
I also need to package all of the found data up into a usable form that I can iterate through and send to a jinja2 template.
For example:
{% for item in query_results %}
<ul>
<li>item.show_date</li>
<li>item.show_venue</li>
<li>item.show_city</li>
<li>item.show_state</li>
<li>item.show_country</li>
</ul>
<ul>
{% for song in item %}
<li>song.song_name</li>
<li>song.song_set</li>
<li>song.song_track</li>
<li>song.song_encore</li>
<li>song.song_segue</li>
<li>song.song_notes</li>
</ul>
{% endfor %}
Thanks in advance. Brent
Seems like what you're trying to do is follow a FK relationship backwards.
This is what it should look like in the template:
{% for show in query_results %}
<ul>
<li>show.show_date</li>
<li>show.show_venue</li>
<li>show.show_city</li>
<li>show.show_state</li>
<li>show.show_country</li>
</ul>
<ul>
{% for song in show.entry_set.all %}
<li>song.song_name</li>
<li>song.song_set</li>
<li>song.song_track</li>
<li>song.song_encore</li>
<li>song.song_segue</li>
<li>song.song_notes</li>
</ul>
{% endfor %}
This will actually force jango to issue one SQL query for every show. If you have too many it can be a pain. To avoid this you can tell Django to select related songs data when it queries for the shows. This can save you a lot of SQL queries.
Shows.objects.select_related(Songs).filter(show_date__year=2000)
I finally figured it out!
First I queried the Shows model/table and saved the results to query_results:
query_results = Shows.objects.filter(show_date__year=2018)
Then in my Jinja Template
{% for show in query_results %}
<ul>
<li>{{show.show_date}}</li>
<li>{{show.show_venue}}</li>
<li>{{show.show_city}}</li>
<li>{{show.show_state}}</li>
<li>{{show.show_country}}</li>
</ul>
<ul>
{% for song in show.songs_set.all %} #This is the key, calling the related "shows_set.all" This grabs the related objects from the Songs table/model
<li>{{song.song_name}}</li>
<li>{{song.song_set}}</li>
<li>{{song.song_track}}</li>
<li>{{song.song_encore}}</li>
<li>{{song.song_segue}}</li>
<li>{{song.song_notes}}</li>
{% endfor %}
</ul>
{% endfor %}