Develop Reference

How to change join and group by SQL to ORM in Django

I'm new in Django. So, I want to join two models which are company and client and count the number of clients for each of the company. Here the SQL
SELECT Company_company.name, count(Client_client.cid)
FROM Company_company
LEFT JOIN Client_client
ON Company_company.comid = Client_client.comid_id
GROUP BY Company_company.name;
But since in Django, we use ORM. So I'm a little bit confusing since I'm a beginner. I already refer few SQL to ORM converter website such as Django ORM and do some try and error. But, I didn't know where the problem since I want the output from the ORM to be classified into a different array. Here is my code:
labels = []
data = []
queryClientCompany = client.objects.values('comid').annotate(c=Count('cid')).values('comid__name','c')
for comp in queryClientCompany:
labels.append(comp.comid__name)
data.append(comp.c)
Here some of the relevant things in the client and company models:
class client (models.Model):
#client info
cid = models.AutoField(primary_key = True)
comid = models.ForeignKey(company,related_name='companys',
on_delete = models.DO_NOTHING,verbose_name="Company",null = True, blank = True)
class company(models.Model):
comid = models.AutoField(_('Company'),primary_key = True)
#company info
name = models.CharField(_('Company Name'),max_length = 50)
The error stated that the comid__name is not defined. So actually how to append the result? I hope someone can help me. Thank you for helping in advanced.

You should query from the opposite side to perform the LEFT OUTER JOIN between company and client (and not client and company):
from django.db.models import Count
labels = []
data = []
queryClientCompany = company.objects.annotate(
c=Count('companys__cid')
)
for comp in queryClientCompany:
labels.append(comp.name)
data.append(comp.c)
The companys part is due to the related_name='copanys', but it does not make much sense to name this relation that way. The related_name=… parameter [Django-doc] specifies how to access the Clients for a given Company, so clients is a more appropriate value for the related_name:
class client (models.Model):
cid = models.AutoField(primary_key=True)
comid = models.ForeignKey(
company,
related_name='clients',
on_delete = models.DO_NOTHING,
verbose_name="Company",
null = True,
blank = True
)
then the query is:
from django.db.models import Count
labels = []
data = []
queryClientCompany = company.objects.annotate(
c=Count('clients__cid')
)
for comp in queryClientCompany:
labels.append(comp.name)
data.append(comp.c)

peewee : how to filter data in a table based on values across rows in another table

Considering following peewee backed python code, how to filter data in a table based on values across rows in another table
e.g.
if I want to get a note in Note table having all java, lambda, generics tags in customtags
#!/usr/bin/env python3
import peewee
import datetime
db = peewee.SqliteDatabase('test.db')
class Note(peewee.Model):
id = peewee.AutoField()
name = peewee.CharField()
text = peewee.CharField()
created = peewee.DateField(default=datetime.date.today)
class Meta:
database = db
db_table = 'notes'
class CustomTag(peewee.Model):
id = peewee.AutoField()
note = peewee.ForeignKeyField(Note, backref='notes')
tag = peewee.CharField()
class Meta:
database = db
db_table = 'customtags'
indexes = ((("note_id", "tag"), True),)
Note.drop_table()
CustomTag.drop_table()
Note.create_table()
CustomTag.create_table()
note1 = Note.create(name="note1",text='Java 8 lambda with generics')
note1.save()
CustomTag.insert(note=note1, tag='java').on_conflict_ignore(True)
CustomTag.insert(note=note1, tag='lambda').on_conflict_ignore(True)
CustomTag.insert(note=note1, tag='generics').on_conflict_ignore(True)
note2 = Note.create(name="note2",text='Core Java concepts',
created=datetime.date(2018, 10, 20))
note2.save()
CustomTag.insert(note=note2, tag='java').on_conflict_ignore(True)
note3 = Note.create(name="note3",text='Java collection framework',
created=datetime.date(2018, 10, 22))
note3.save()
CustomTag.insert(note=note3, tag='java').on_conflict_ignore(True)
note4 = Note.create(name="note4",text='Java NIO packageJava nio package')
note4.save()
CustomTag.insert(note=note4, tag='java').on_conflict_ignore(True)
notes = Note.select().join(CustomTag, peewee.JOIN.LEFT_OUTER,on=(Note.id == CustomTag.note_id)).order_by(Note.name)
for note in notes:
print('{} with text {} on {}'.format(note.name, note.text, note.created))
I am really not having idea how to modify my code the get the data above mentioned, I know that issue is in following code
notes = Note.select().join(CustomTag, peewee.JOIN.LEFT_OUTER,on=(Note.id == CustomTag.note_id)).order_by(Note.name)

e.g. if I want to get a note in Note table having all java, lambda, generics tags in customtags
tags = ['foo', 'bar', 'baz']
query = (Note
.select()
.join(CustomTag)
.where(CustomTag.tag.in_(tags))
.group_by(Note)
.having(fn.COUNT(CustomTag.id) == len(tags)))

It seems execute() is missing on your insert statements. it should be corrected as of each line...
CustomTag.insert(note=note1, tag='java').on_conflict_ignore(True).execute()
I've included a bit improved version of coleifer answer here, results includes list of tags as well...
from peewee import fn
filter_tags = ['java', 'lambda', 'generics']
notes = (Note
.select(Note, fn.GROUP_CONCAT(CustomTag.tag,',').alias('tags'))
.join(CustomTag)
.where(CustomTag.tag.in_(filter_tags))
.group_by(Note)
.having(fn.COUNT(CustomTag.id) >= len(filter_tags)))
for note in notes:
print(f'name : {note.name} text: {note.text} tags: {note.tags} created : {note.created}')

How do I display Django data from a related model of a related model?

I am trying to display data from several models that are related together through a QuerySet. My ultimate goal is to display some information from the Site model, and some information from the Ppack model, based on a date range filter of the sw_delivery_date in the Site model.
Here are my models:
class Site(models.Model):
mnemonic = models.CharField(max_length = 5)
site_name = models.CharField(max_length = 100)
assigned_tech = models.ForeignKey('Person', on_delete=models.CASCADE, null = True, blank = True)
hw_handoff_date = models.DateField(null = True, blank = True)
sw_delivery_date = models.DateField(null = True, blank = True)
go_live_date = models.DateField(null = True, blank = True)
web_url = models.CharField(max_length = 100, null = True, blank = True)
idp_url = models.CharField(max_length = 100, null = True, blank = True)
def __str__(self):
return '(' + self.mnemonic + ') ' + self.site_name
class Ring(models.Model):
ring = models.IntegerField()
def __str__(self):
return "6." + str(self.ring)
class Ppack(models.Model):
ppack = models.IntegerField()
ring = models.ForeignKey('Ring', on_delete=models.CASCADE)
def __str__(self):
return str(self.ring) + " pp" + str(self.ppack)
class Code_Release(models.Model):
Inhouse = 'I'
Test = 'T'
Live = 'L'
Ring_Location_Choices = (
(Inhouse, 'Inhouse'),
(Test, 'Test'),
(Live, 'Live'),
)
site_id = models.ForeignKey('Site', on_delete=models.CASCADE)
type = models.CharField(max_length = 1, choices = Ring_Location_Choices, blank = True, null = True)
release = models.ForeignKey('Ppack', on_delete=models.CASCADE)
def __str__(self):
return "site:" + str(self.site_id) + ", " + self.type + " = " + str(self.release)
If I use the following,
today = datetime.date.today()
future = datetime.timedelta(days=60)
new_deliveries = Site.objects.select_related().filter(sw_delivery_date__range=[today, (today + future)])
I can get all of the objects in the Site model that meet my criteria, however, because there is no relation from Site to Code_Release (there's a one-to-many coming the other way), I can't get at the Code_Release data.
If I run a for loop, I can iterate through every Site returned from the above query, and select the data from the Code_Release model, which allows me to get the related data from the Ppack and Ring models.
site_itl = {}
itl = {}
for delivery in new_deliveries:
releases = Code_Release.objects.select_related().filter(site_id = delivery.id)
for rel in releases:
itl[rel.id] = rel.release
site_itl[delivery.id] = itl
But, that seems overly complex to me, with multiple database hits and possibly a difficult time parsing through that in the template.
Based on that, I was thinking that I needed to select from the Code_Release model. That relates back to both the Site model and the Ppack model (which relates to the Ring model). I've struggled to make the right query / access the data in this way that accomplishes what I want, but I feel this is the right way to go.
How would I best accomplish this?

You can use RelatedManager here. When you declare ForeignKey, Django allows you to access reverse relationship. To be specific, let's say that you have multiple code releases that are pointing to one specific site. You can access them all via site object by using <your_model_name_lowercase>_set attribute. So in your case:
site.code_release_set.all()
will return QuerySet of all code release objects that have ForeignKey to object site

You can access the Releases from a Site object. First, you can put a related_name to have a friendly name of the reverse relation between the models:
site_id = models.ForeignKey('Site', on_delete=models.CASCADE, related_name="releases")
and then, from a Site object you can make normal queries to Release model:
site.releases.all()
site.releases.filter(...)
...

How to convert results of Model queries into xlsx using pandas in python?

I have a standart model's collection in my project like this one:
class Dimension(models.Model):
dimension_id = MyCharField(max_length=1024, primary_key=True)
name = MyCharField(max_length=255, null = False, unique=True)
external_flg = models.BooleanField(default = False)
ext_owner = MyCharField(max_length=30, null = True)
ext_table_name = MyCharField(max_length=30, null = True)
ext_start_date_column_name = MyCharField(max_length=30, null = True)
ext_end_date_column_name = MyCharField(max_length=30, null = True)
ext_id_column_name = MyCharField(max_length=30, null = True)
ext_name_column_name = MyCharField(max_length=30, null = True)
ext_where_codition = MyCharField(max_length=512, null = True)
def save(self):
cnt =self.__class__.objects.filter(name=self.name).count()
if cnt==0:
if self.pk:
super(Dimension, self).save()
else:
self.dimension_id = getUid()
super(Dimension, self).save()
else:
raise DimensionUniqueError(self.name)
At this moment, I have to implement a button which will import data from our models to xlsx files and download it automatically on the client side.
We're planning to use pandas for sql to xlsx conversion, but I can't understand how to implement interaction with pandas for Models. For now I implemented it this way:
import pandas as ps
class Excel:
def __init__(self, model_name):
self.model_name = model_name
def sql_to_xlsx():
elements = self.model_name.objects.all()
filter = self.request.GET.get('filter', None)
if filter is not None:
elements = elements.filter(filter_field=filter)
columns = [desc[0] for desc in elements]
data = [desc[1:] for desc in elements]
df = ps.DataFrame(list(data), columns)
writer = ps.ExcelWriter('converted.xlsx')
df.to_excel(writer, sheet_name='converted')
return writer.save()
def get(self, request, *args, **kwargs):
document = self.sql_to_xlsx()
_file = open(document.filename, 'r')
response = HttpResponse(_file, content_type='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet')
response['Content-Disposition'] = 'attachment; filename=%s' % document.filename.split('/')[-1] # Here will return a full path, that's why probably you will need a split to get only the filename
add_never_cache_headers(response=response) # To avoid download the same file with out of date data.
return response
but it's not a correct logic what I would have expected to have. I have a feeling that it's not correct way to do.
Could you please help me to try out how to implement needed logic for our models?
Thank you!

I don't think you need pandas to write to excel.
I would use openpyxl for writing to excel. Just following the example on their page should be good enough for you. You just have to loop through your model and use openpyxl to write to specific rows and columns that you want.

Django - Reference data from another model using a foreign key

I'm new to Django so please tell me if I'm not on the right track.
I have a Django project that I'm building and just wanted to ask what is the correct Django way to retrieve data from one model and use it in another.
I have a for loop to assign the required fields to variables but I was looking for a cleaner solution and if one exists, one that uses the foreign key.
Here's an example of the code involved:
#MODELS
class Class1(models.Model):
tag_number = models.CharField(max_length = 300, null = True, blank = True)
example1 = models.CharField(max_length = 300, null = True, blank = True)
example2 = models.CharField(max_length = 300, null = True, blank = True)
class Class2(models.Model):
tag = models.ForeignKey(Class1, related_name = 'tag_foreignkey')
example3 = models.CharField(max_length = 300, null = True, blank = True)
example4 = models.CharField(max_length = 300, null = True, blank = True)
#VIEWS - This view is for Class2 but referencing fields from Class1
tag_number = str(instrumentform.cleaned_data['tag'])
query_results = Class1.objects.filter(tag_number = tag_number)
for query_result in query_results:
example5 = query_result.example1
example6 = query_result.example2
The above works but I assume it's not the Django way of doing things and is not taking advantage of the foreign key.
If someone could give me a nudge in the right direction that would be greatly appreciated.

Still, not hundred percent know what you want, since there are some missing info out there.
For you Class1, You should do what you have done to Class2, using foreign key to store tag.
For you code at bottom, there is a easy way to do it. (Assume you have used foreign key)
tag_number = int(instrumentform.cleaned_data['tag'])
for query_result in Class1.objects.filter(tag = tag_number).values_list('example1', 'exmaple2'):
example5, example6 = query_result

I'm not sure what you are trying to achieve here. I am imagining that PstCalc = Class1
Nevertheless, if you have
c1 = Class1()
c2 = Class2()
c3 = Class2()
and
c2.tag = c1
c3.tag = c1
using the foreignkey you would have:
c1.class2_set = (c2, c3) <- this would be a queryset, not a list
c2.tag = c1 => c2.tag.example1 = c1.example1
...hope I made my self understood :)