Develop Reference

How to use prefetch_related on two M2M values?

I want to prefetch_related to two level of M2M values,
Here is my models.py
class A(models.Model):
name = models.CharField(max_length=40)
b = models.ManyToManyField('B')
class B(models.Model):
name = models.CharField(max_length=40)
c = models.ManyToManyField('C')
class C(models.Model):
name = models.CharField(max_length=40)
d = models.ManyToManyField('D')
And my ORM is
a_obj = A.objects.all().prefetch_related('a__b__c')
And I am trying to access the values like below,
Method A:
for each_obj in a_obj:
print(each_obj.a__b__c)
Method B:
for each_obj in a_obj:
print(each_obj.a.all())
Method A throws an error saying No such value a__b__b for A found
Method B doesn't throw any error, but the number of queries increases to the length of a_obj.
Is there a way to access a__b__c in a single query?

You load both the related B and C models with .prefetch_related(…) [Django-doc]:
a_objs = A.objects.prefetch_related('b__c')
But here .prefetch_related(…) does not change how the items look, it simply loads items. You thus can access these with:
for a in a_objs:
for b in a.b.all():
for c in b.c.all():
print(f'{a} {b} {c}')
You this still access the items in the same way, but here Django will already load the objects in advance to prevent extra queries.

How to join tables in Django 1.8

I have tried to access joined data in my django template but nothing works, a little help is deeply appreciated.
Model1():
project_code = Foreignkey(another table1)
shot_code = charfield(primary_key = True)
shot_name = charfield()
sequence_name = Integerfield()
Model2():
vendor_id = Foreignkey(another table2)
shot_code = Foreignkey(Model1, on_delete= models.CASCADE)
shot_rate = integerfield()
shot_bid = integerfield()
I wanted to display
Select * from Model1 a, Model2 b, where a.shot_code = b.shot_code
and model1.project_code = "XXX"
and columns to be accessed in template are
1. Shot code
2. Shot name
3. Sequence name
4. Shot rate
5. Shot bid
6. Vendor id
I tried the following method
1. Using Select_related
result = only values of model2 is displayed
unable to access model1's data,
error = 'QuerySet' object has no attribute model1

Do you expect this to return one or multiple instances? The best way to do this would be still with select_related, e.g.:
Model2.objects.filter(shot_code__project_code=<your value>).select_related("shot_code")
For queryset with multiple Model2 instances, or add .get() at the end if you expect only single instance.
Alternatively, you can add .values() and instead of operating on two related models, get dict-like join result (although note that you won't be able to reuse shot_code straightforward, as it would clash with your foreign key name):
Model2.objects.filter(shot_code__project_code=<your value>).annotate(
sequence_name=F("shot_code__sequence_name"),
shot_name=F("shot_code__shot_name"),
real_shot_code=F("shot_code__shot_code")
).values(
"sequence_name", "shot_name", "real_shot_code", "shot_rate", "shot_bid", "vendor_id"
)
And as always, I recommend to refrain from naming your ForeignKey as vendor_id, since it will place the real id under the vendor_id_id, and naming will be a bit unclear.

You can use object of Model1 query set in Model2 and get the data see below example:
model1obj = Model1.objects.get(project_code = "XXX")
model2obj = Model2.objects.get(shot_code = model1obj)
# now access all the fields using model1obj and model2obj

One to one class not updating in django: Gives Integrity Error

I am relatively new to Django and have been kinda struggling.
These are my three models below:
class Site(models.Model):
siteID = models.CharField(max_length=255, primary_key=True)
class EndDevice(models.Model):
class Meta:
unique_together = ("edevID", "siteID")
edevID = models.CharField(max_length=255)
siteID = models.ForeignKey(Site, on_delete=models.CASCADE)
deviceCategory = models.BigIntegerField()
class ThirdCombi(models.Model):
siteID = models.OneToOneField(Site, on_delete=models.CASCADE, primary_key=True)
endDevice = models.TextField()
I am trying to make a table where one siteID displays all the edevID, which is the third model here. This does work using the following serializers.py
class CombiSerializer(serializers.ModelSerializer):
class Meta:
model = ThirdCombi
fields = ("siteID", "endDevice")
def serialize(devices):
d_list = []
fields = ['edevID', 'siteID', 'deviceCategory']
for device in devices:
d_list.append(model_to_dict(device, fields=fields))
return d_list
And the views.py as follow:
class CombiView(generics.RetrieveUpdateDestroyAPIView):
queryset = ThirdCombi.objects.all()
serializer_class = CombiSerializer
def get(self, request, *args, **kwargs):
try:
s1 = Site.objects.get(siteID=kwargs["pk"])
devices = EndDevice.objects.filter(siteID=s1)
a_site, created = ThirdCombi.objects.get_or_create(siteID=s1, endDevice=CombiSerializer.serialize(devices))
return Response(CombiSerializer(a_site).data)
except Site.DoesNotExist:
return Response(
data={
"message": "Site with id: {} does not exist".format(kwargs["pk"])},
status=status.HTTP_404_NOT_FOUND)
But once there is an update in EndDevice and I reload the page it gives me an integrity error and if I put an exception to integrity error, I cannot see the changes made in EndDevice reflected in ThirdCombi. I know why there is an integrity error because the siteID already exists and it tries to make a new one. I am not sure how to clear the old one in order to avoid the integrity error.
Any help will be appreciated in order to update the third table. Thanks.

In Short: Use update_or_create instead of get_or_create.
ThirdCombi.objects.update_or_create(siteID=s1, defaults={"endDevice":CombiSerializer.serialize(devices)})
Go to bottom for full updated code.
I know why there is an integrity error because the siteID already exists and it tries to make a new one.
You are right.
get_or_create: creates a new object if it doesn't find one with the passed parameters.
OneToOneField: If you have a model Site and another model ThirdCombi with a OneToOneField on Site, it means that ThirdCombi can only have utmost one object per Site
The problem is you are trying to create a new ThirdCombi with every get request.
Please observe:
Initially, there is a Site s1 with 2 devices.
s1 = Site.objects.get(siteID=kwargs["pk"])
devices = EndDevice.objects.filter(siteID=s1) #2 devices
a_site, created = ThirdCombi.objects.get_or_create(siteID=s1, endDevice=CombiSerializer.serialize(devices)) #2 devices serialized "d1, d2"
Now, get_or_create creates a new ThirdCombi object for s1, since there is no ThirdCombi object for s1 initially. Let the created object be third_combi1.
But once there is an update in EndDevice and I reload the page
See, you have updated the EndDevice. Now again observe:
We already have Site: s1, EndDevices: d1, d2 and ThirdCombi: third_combi1 for s1.
We cannot have another third_combi2 for s1.
Since you updated EndDevices, let the current updated EndDevices be: d1, d2, d3
s1 = Site.objects.get(siteID=kwargs["pk"])
devices = EndDevice.objects.filter(siteID=s1) #3 devices
a_site, created = ThirdCombi.objects.get_or_create(siteID=s1, endDevice=CombiSerializer.serialize(devices)) #2 devices serialized "d1, d2, d3"
The problem here is get_or_create doesn't find an object with "d1, d2, d3". Hence it tries to create another third_combi2 for 's1' with d1, d2, d3. Hence the integrity error.
Instead, use update_or_create.
update_or_create: update if an object with the given parameters exist, else create.
Your final code should be:
s1 = Site.objects.get(siteID=kwargs["pk"])
devices = EndDevice.objects.filter(siteID=s1) #2 devices
a_site, created = ThirdCombi.objects.update_or_create(siteID=s1, defaults={"endDevice":CombiSerializer.serialize(devices)})
Here, if there is a ThirdCombi object with siteId=s1, then update its endDevice field with the provided ones. Else create a new ThirdCombi object with the given data.

SQLalchemy find id and use it to lookup other information

I'm making a simple lookup application for Japanese characters (Kanji), where the user can search the database using any of the information available.
My database structure
Kanji:
id
character (A kanji like 頑)
heisig6 (a number indicating the order of showing Kanji)
kanjiorigin (a number indicating the order of showing Kanji)
MeaningEN (1 kanji_id can have multiple entries with different meanings):
kanji_id (FOREIGN KEY(kanji_id) REFERENCES "Kanji" (id)
meaning
User handling
The user can choose to search by 'id', 'character', 'heisig6', 'kanjiorigin' or 'meaning' and it should then return all information in all those fields. (All fields return only 1 result, except meanings, which can return multiple results)
Code, EDIT 4+5: my code with thanks to #ApolloFortyNine and #sqlalchemy on IRC, EDIT 6: join --> outerjoin (otherwise won't find information that has no Origins)
import sqlalchemy as sqla
import sqlalchemy.orm as sqlo
from tableclass import TableKanji, TableMeaningEN, TableMisc, TableOriginKanji # See tableclass.py
# Searches database with argument search method
class SearchDatabase():
def __init__(self):
#self.db_name = "sqlite:///Kanji_story.db"
self.engine = sqla.create_engine("sqlite:///Kanji.db", echo=True)
# Bind the engine to the metadata of the Base class so that the
# declaratives can be accessed through a DBSession instance
tc.sqla_base.metadata.bind = self.engine
# For making sessions to connect to db
self.db_session = sqlo.sessionmaker(bind=self.engine)
def retrieve(self, s_input, s_method):
# s_input: search input
# s_method: search method
print("\nRetrieving results with input: {} and method: {}".format(s_input, s_method))
data = [] # Data to return
# User searches on non-empty string
if s_input:
session = self.db_session()
# Find id in other table than Kanji
if s_method == 'meaning':
s_table = TableMeaningEN # 'MeaningEN'
elif s_method == 'okanji':
s_table = TableOriginKanji # 'OriginKanji'
else:
s_table = TableKanji # 'Kanji'
result = session.query(TableKanji).outerjoin(TableMeaningEN).outerjoin(
(TableOriginKanji, TableKanji.origin_kanji)
).filter(getattr(s_table, s_method) == s_input).all()
print("result: {}".format(result))
for r in result:
print("r: {}".format(r))
meanings = [m.meaning for m in r.meaning_en]
print(meanings)
# TODO transform into origin kanji's
origins = [str(o.okanji_id) for o in r.okanji_id]
print(origins)
data.append({'character': r.character, 'meanings': meanings,
'indexes': [r.id, r.heisig6, r.kanjiorigin], 'origins': origins})
session.close()
if not data:
data = [{'character': 'X', 'meanings': ['invalid', 'search', 'result']}]
return(data)
Question EDIT 4+5
Is this an efficient query?: result = session.query(TableKanji).join(TableMeaningEN).filter(getattr(s_table, s_method) == s_input).all() (The .join statement is necessary, because otherwise e.g. session.query(TableKanji).filter(TableMeaningEN.meaning == 'love').all() returns all the meanings in my database for some reason? So is this either the right query or is my relationship() in my tableclass.py not properly defined?
fixed (see lambda: in tableclass.py) kanji = relationship("TableKanji", foreign_keys=[kanji_id], back_populates="OriginKanji") <-- what is wrong about this? It gives the error:
File "/path/python3.5/site-packages/sqlalchemy/orm/mapper.py", line 1805, in get_property
"Mapper '%s' has no property '%s'" % (self, key))
sqlalchemy.exc.InvalidRequestError: Mapper 'Mapper|TableKanji|Kanji' has no property 'OriginKanji'
Edit 2: tableclass.py (EDIT 3+4+5: updated)
import sqlalchemy as sqla
from sqlalchemy.orm import relationship
import sqlalchemy.ext.declarative as sqld
sqla_base = sqld.declarative_base()
class TableKanji(sqla_base):
__tablename__ = 'Kanji'
id = sqla.Column(sqla.Integer, primary_key=True)
character = sqla.Column(sqla.String, nullable=False)
radical = sqla.Column(sqla.Integer) # Can be defined as Boolean
heisig6 = sqla.Column(sqla.Integer, unique=True, nullable=True)
kanjiorigin = sqla.Column(sqla.Integer, unique=True, nullable=True)
cjk = sqla.Column(sqla.String, unique=True, nullable=True)
meaning_en = relationship("TableMeaningEN", back_populates="kanji") # backref="Kanji")
okanji_id = relationship("TableOriginKanji", foreign_keys=lambda: TableOriginKanji.kanji_id, back_populates="kanji")
class TableMeaningEN(sqla_base):
__tablename__ = 'MeaningEN'
kanji_id = sqla.Column(sqla.Integer, sqla.ForeignKey('Kanji.id'), primary_key=True)
meaning = sqla.Column(sqla.String, primary_key=True)
kanji = relationship("TableKanji", back_populates="meaning_en")
class TableOriginKanji(sqla_base):
__tablename__ = 'OriginKanji'
kanji_id = sqla.Column(sqla.Integer, sqla.ForeignKey('Kanji.id'), primary_key=True)
okanji_id = sqla.Column(sqla.Integer, sqla.ForeignKey('Kanji.id'), primary_key=True)
order = sqla.Column(sqla.Integer)
#okanji = relationship("TableKanji", foreign_keys=[kanji_id], backref="okanji")
kanji = relationship("TableKanji", foreign_keys=[kanji_id], back_populates="okanji_id")

We would really have to be able to see your database schema to give real critique, but assuming no foreign keys, what you said is basically the best you can do.
SQLAlchemy really begins to shine when you have complicated relations going on however. For example, if you properly had foreign keys set, you could do something like the following.
# Assuming kanji is a tc.tableMeaningEN.kanji_id object
kanji_meaning = kanji.meanings
And that would return the meanings for the kanji as an array, without any further queries.
You can go quite deep with relationships, so I'm linking the documentation here. http://docs.sqlalchemy.org/en/latest/orm/relationships.html
EDIT: Actually, you don't need to manually join at all, SQLAlchemy will do it for you.
The case is wrong on your classes, but I'm not sure if SQLAlchemy is case sensitive there or not. If it works, then just move on.
If you query the a table (self.session.query(User).filter(User.username == self.name).first()) you should have an object of the table type (User here).
So in your case, querying the TableKanji table alone will return an object of that type.
kanji_obj = session.query(TableKanji).filter(TableKanji.id == id).first()
# This will return an array of all meaning_ens that match the foreign key
meaning_arr = kanji_obj.meaning_en
# This will return a single meeting, just to show each member of the arr is of type TableMeaningEn
meaning_arr[0].meaning
I have a project made use of some of these features, hope it helps:
https://github.com/ApolloFortyNine/SongSense
Database declaration (with relationships): https://github.com/ApolloFortyNine/SongSense/blob/master/songsense/database.py
Automatic joins: https://github.com/ApolloFortyNine/SongSense/blob/master/songsense/getfriend.py#L134
I really like my database structure, but as for the rest it's pretty awful. Hope it still helps though.

Django - Reference data from another model using a foreign key

I'm new to Django so please tell me if I'm not on the right track.
I have a Django project that I'm building and just wanted to ask what is the correct Django way to retrieve data from one model and use it in another.
I have a for loop to assign the required fields to variables but I was looking for a cleaner solution and if one exists, one that uses the foreign key.
Here's an example of the code involved:
#MODELS
class Class1(models.Model):
tag_number = models.CharField(max_length = 300, null = True, blank = True)
example1 = models.CharField(max_length = 300, null = True, blank = True)
example2 = models.CharField(max_length = 300, null = True, blank = True)
class Class2(models.Model):
tag = models.ForeignKey(Class1, related_name = 'tag_foreignkey')
example3 = models.CharField(max_length = 300, null = True, blank = True)
example4 = models.CharField(max_length = 300, null = True, blank = True)
#VIEWS - This view is for Class2 but referencing fields from Class1
tag_number = str(instrumentform.cleaned_data['tag'])
query_results = Class1.objects.filter(tag_number = tag_number)
for query_result in query_results:
example5 = query_result.example1
example6 = query_result.example2
The above works but I assume it's not the Django way of doing things and is not taking advantage of the foreign key.
If someone could give me a nudge in the right direction that would be greatly appreciated.

Still, not hundred percent know what you want, since there are some missing info out there.
For you Class1, You should do what you have done to Class2, using foreign key to store tag.
For you code at bottom, there is a easy way to do it. (Assume you have used foreign key)
tag_number = int(instrumentform.cleaned_data['tag'])
for query_result in Class1.objects.filter(tag = tag_number).values_list('example1', 'exmaple2'):
example5, example6 = query_result

I'm not sure what you are trying to achieve here. I am imagining that PstCalc = Class1
Nevertheless, if you have
c1 = Class1()
c2 = Class2()
c3 = Class2()
and
c2.tag = c1
c3.tag = c1
using the foreignkey you would have:
c1.class2_set = (c2, c3) <- this would be a queryset, not a list
c2.tag = c1 => c2.tag.example1 = c1.example1
...hope I made my self understood :)