Develop Reference

How can i change this query to ORM?

Hi i have two models like this,
class Sample(models.Model):
name = models.CharField(max_length=256) ##
processid = models.IntegerField(default=0) #
class Process(models.Model):
sample = models.ForeignKey(Sample, blank=False, null=True, on_delete=models.SET_NULL, related_name="process_set")
end_at = models.DateTimeField(null=True, blank=True)
and I want to join Sample and Process model. Because Sample is related to process and I want to get process information with sample .
SELECT sample.id, sample.name, process.endstat
FROM sample
INNER JOIN process
ON sample.processid = process.id
AND process.endstat = 1;
(i'm using SQLite)
I used
sample_list = sample_list.filter(process_set__endstat=1))
but it returned
SELECT sample.id, sample.name
FROM sample
INNER JOIN process
ON (sample.id = process.sample_id)
AND process.endstat = 1)
This is NOT what I want.
How can i solve the problem?

This should work for you
Process.objects.filter(end_at=1).values('sample__id','sample__name','end_at')
.values() method returns selective table fields.

I'm assuming sample_list = Sample.objects.
When you are filtering a model, only the fields defined in the model are selected. In your example, id and processid. If you want to retrieve values from related models as a single record you need to use values or values_list. To get the desired query you have to do this
sample_list = sample_list.filter(process_set__endstat=1).values('id', 'name', 'process__endstat')
Btw, Django does JOIN on the foreign key field. So, you can't get ON sample.processid = process.id since processid is not a ForeignKey field.
Reference:
https://docs.djangoproject.com/en/4.0/ref/models/querysets/#values

I found JOIN not on foreign key field in django.
sample_list = sample_list.filter(processid__in=Process.objects.filter(endstat=1)
I used the medthod of
Django-queryset join without foreignkey

how to use a queryset expression in a then clause in conditional annotation in Django

i have two models, one of organizations and one with the membership and rol of an user in the organization
class Organization(models.Model):
name = models.CharField(blank=False,null=False,max_length=100, unique=True)
class Member(models.Model):
user_request = models.ForeignKey('accounts.User',on_delete=models.CASCADE,related_name="member_user_request")
user_second = models.ForeignKey('accounts.User',on_delete=models.CASCADE,blank=True,null=True, related_name="member_user_second")
role = models.ForeignKey(RoleOrganization,on_delete=models.CASCADE, verbose_name=_('Rol'))
status = models.ForeignKey(Status,on_delete=models.CASCADE, verbose_name=_('Status'))
organization = models.ForeignKey(Organization,on_delete=models.CASCADE, verbose_name=_('Organization'))
and im trying to use a annotate with case clause where i want to get the role of an user in the organization with this expression:
my_organizations = Member.objects.filter(
Q(user_request_id=self.request.user.id, status__name="accepted", type_request__name="request") |
Q(user_second_id=self.request.user.id, status__name="accepted", type_request__name="invitation")
)
Organization.objects.annotate(
rol=Case(
When(id__in=list(my_organizations.values_list('organization_id', flat=True)),
then=Value(my_organizations.get(organization_id=F('id')).role.name)),
default=None, output_field=CharField()
)
)
the problem here is that the then expression doesn't get the id of the object in the main queryset, if i return in the then just the F('id') the expression gets the value of the id in the main queryset, but i can use a filter or any queryset expression with some values of the main object.
its there a way to accomplish this.
PS: im just putting part of the code here for cleanliness, but if you need to know more please let me know

I think you can do it like this using Subquery:
from django.db.models import OuterRef, Subquery
members = Member.objects.filter(
Q(user_request_id=self.request.user.id, status__name="accepted", type_request__name="request") |
Q(user_second_id=self.request.user.id, status__name="accepted", type_request__name="invitation")
)
member_subquery = members.filter(organization=OuterRef('pk'))
organizations = Organization.objects.annotate(member_role=Subquery(member_subquery.values('role')[:1]))
print(organizations.values('member_role'))

django - prefetch only the newest record?

I am trying to prefetch only the latest record against the parent record.
my models are as such
class LinkTargets(models.Model):
device_circuit_subnet = models.ForeignKey(DeviceCircuitSubnets, verbose_name="Device", on_delete=models.PROTECT)
interface_index = models.CharField(max_length=100, verbose_name='Interface index (SNMP)', blank=True, null=True)
get_bgp = models.BooleanField(default=False, verbose_name="get BGP Data?")
dashboard = models.BooleanField(default=False, verbose_name="Display on monitoring dashboard?")
class LinkData(models.Model):
link_target = models.ForeignKey(LinkTargets, verbose_name="Link Target", on_delete=models.PROTECT)
interface_description = models.CharField(max_length=200, verbose_name='Interface Description', blank=True, null=True)
...
The below query fails with the error
AttributeError: 'LinkData' object has no attribute '_iterable_class'
Query:
link_data = LinkTargets.objects.filter(dashboard=True) \
.prefetch_related(
Prefetch(
'linkdata_set',
queryset=LinkData.objects.all().order_by('-id')[0]
)
)
I thought about getting LinkData instead and doing a select related but ive no idea how to get only 1 record for each link_target_id
link_data = LinkData.objects.filter(link_target__dashboard=True) \
.select_related('link_target')..?
EDIT:
using rtindru's solution, the pre fetched seems to be empty. there is 6 records in there currently, atest 1 record for each of the 3 LinkTargets
>>> link_data[0]
<LinkTargets: LinkTargets object>
>>> link_data[0].linkdata_set.all()
<QuerySet []>
>>>

The reason is that Prefetch expects a Django Queryset as the queryset parameter and you are giving an instance of an object.
Change your query as follows:
link_data = LinkTargets.objects.filter(dashboard=True) \
.prefetch_related(
Prefetch(
'linkdata_set',
queryset=LinkData.objects.filter(pk=LinkData.objects.latest('id').pk)
)
)
This does have the unfortunate effect of undoing the purpose of Prefetch to a large degree.
Update
This prefetches exactly one record globally; not the latest LinkData record per LinkTarget.
To prefetch the max LinkData for each LinkTarget you should start at LinkData: you can achieve this as follows:
LinkData.objects.filter(link_target__dashboard=True).values('link_target').annotate(max_id=Max('id'))
This will return a dictionary of {link_target: 12, max_id: 3223}
You can then use this to return the right set of objects; perhaps filter LinkData based on the values of max_id.
That will look something like this:
latest_link_data_pks = LinkData.objects.filter(link_target__dashboard=True).values('link_target').annotate(max_id=Max('id')).values_list('max_id', flat=True)
link_data = LinkTargets.objects.filter(dashboard=True) \
.prefetch_related(
Prefetch(
'linkdata_set',
queryset=LinkData.objects.filter(pk__in=latest_link_data_pks)
)
)

The following works on PostgreSQL. I understand it won't help OP, but it might be useful to somebody else.
from django.db.models import Count, Prefetch
from .models import LinkTargets, LinkData
link_data_qs = LinkData.objects.order_by(
'link_target__id',
'-id',
).distinct(
'link_target__id',
)
qs = LinkTargets.objects.prefetch_related(
Prefetch(
'linkdata_set',
queryset=link_data_qs,
)
).all()

LinkData.objects.all().order_by('-id')[0] is not a queryset, it is an model object, hence your error.
You could try LinkData.objects.all().order_by('-id')[0:1] which is indeed a QuerySet, but it's not going to work. Given how prefetch_related works, the queryset argument must return a queryset that contains all the LinkData records you need (this is then further filtered, and the items in it joined up with the LinkTarget objects). This queryset only contains one item, so that's no good. (And Django will complain "Cannot filter a query once a slice has been taken" and raise an exception, as it should).
Let's back up. Essentially you are asking an aggregation/annotation question - for each LinkTarget, you want to know the most recent LinkData object, or the 'max' of an 'id' column. The easiest way is to just annotate with the id, and then do a separate query to get all the objects.
So, it would look like this (I've checked with a similar model in my project, so it should work, but the code below may have some typos):
linktargets = (LinkTargets.objects
.filter(dashboard=True)
.annotate(most_recent_linkdata_id=Max('linkdata_set__id'))
# Now, if we need them, lets collect and get the actual objects
linkdata_ids = [t.most_recent_linkdata_id for t in linktargets]
linkdata_objects = LinkData.objects.filter(id__in=linkdata_ids)
# And we can decorate the LinkTarget objects as well if we want:
linkdata_d = {l.id: l for l in linkdata_objects}
for t in linktargets:
if t.most_recent_linkdata_id is not None:
t.most_recent_linkdata = linkdata_d[t.most_recent_linkdata_id]
I have deliberately not made this into a prefetch that masks linkdata_set, because the result is that you have objects that lie to you - the linkdata_set attribute is now missing results. Do you really want to be bitten by that somewhere down the line? Best to make a new attribute that has just the thing you want.

Tricky, but it seems to work:
class ForeignKeyAsOneToOneField(models.OneToOneField):
def __init__(self, to, on_delete, to_field=None, **kwargs):
super().__init__(to, on_delete, to_field=to_field, **kwargs)
self._unique = False
class LinkData(models.Model):
# link_target = models.ForeignKey(LinkTargets, verbose_name="Link Target", on_delete=models.PROTECT)
link_target = ForeignKeyAsOneToOneField(LinkTargets, verbose_name="Link Target", on_delete=models.PROTECT, related_name='linkdata_helper')
interface_description = models.CharField(max_length=200, verbose_name='Interface Description', blank=True, null=True)
link_data = LinkTargets.objects.filter(dashboard=True) \
.prefetch_related(
Prefetch(
'linkdata_helper',
queryset=LinkData.objects.all().order_by('-id'),
'linkdata'
)
)
# Now you can access linkdata:
link_data[0].linkdata
Ofcourse with this approach you can't use linkdata_helper to get related objects.

This is not a direct answer to you question, but solves the same problem. It is possible annotate newest object with a subquery, which I think is more clear. You also don't have to do stuff like Max("id") to limit the prefetch query.
It makes use of django.db.models.functions.JSONObject (added in Django 3.2) to combine multiple fields:
MainModel.objects.annotate(
last_object=RelatedModel.objects.filter(mainmodel=OuterRef("pk"))
.order_by("-date_created")
.values(
data=JSONObject(
id="id", body="body", date_created="date_created"
)
)[:1]
)

How to query Non Unique Together values in Django or SQL?

Let's say I have this model:
class Contact(BaseModel):
order = models.ForeignKey(Order, related_name='contacts', blank=True, null=True)
type = models.CharField(max_length=15, choices=TYPES, blank=True))
I want to find all orders where order and type are not unique together.
For example, there is order A and there are related contacts:
Contact(order=orderA, type='broker')
Contact(order=orderA, type='broker')
Contact(order=orderA, type='delivery')
I want to find this orderA because this order and type='broker' are not unique together in Contact model.
And then there is orderB and these related contacts:
Contact(order=orderB, type='broker')
Contact(order=orderB, type='delivery')
I don't want this orderB because it and the field type are unique in Contact model.
I tried using annonate() of Django but failed to relate these two fields.
Is it possible to do this with Django queries?
If not, a slight hint of how I could do it in SQL would be greatly appreciated.
Thank you very much.

You could use a SQL query like:
select distinct order_id
from (
select order_id, type
from Contact
group by order_id, type
having count(*) > 1);
The "order" column is shown as "order_id" because that's the way Django names columns.

This should do the trick
qs = (Contact.objects.values('order','type')
.annotate(cnt=models.Count('pk'))
.filter(cnt__gt=1))

You could write a couple methods to solve this problem. I might have gone over board writing these methods for you but regardless heres an explaination.
def equal_to takes self and some other contact, returns true if that contact is the same order and type else false. def all_not_unique returns a list of all not unique contact objects with no duplicates. And should be called like, not_unique = Contact.all_not_unique().
def equal_to(self, other):
assert(self.id != other.id)
if self.order.id == other.order.id:
if self.type == other.type:
return True
return false
#classmethod
def all_not_unique(cls):
query_set = cls.objects.all()
not_unique_query_set = []
for contact_one in query_set:
found = False
for contact_two in query_set:
if not found:
if contact_one.id != contact_two.id:
if contact_one.equal_to(contact_two):
not_unique_query_set.append(contact_one)
found = True

Database error: Lookup type 'in' can't be negated

I am using Django framework, appengine database.
My code for model is:
class Group(models.Model):
name = models.CharField(max_length=200)
ispublic = models.BooleanField()
logo = models.CharField(max_length=200)
description = models.CharField(max_length=200)
groupwebsite = models.CharField(max_length=200)
owner = models.ForeignKey('profile')
class Group_members(models.Model):
profile = models.CharField(max_length=200)
group = models.ForeignKey('group')
I am querying on Group_members to remove group. My query is as follows:
groups = Group_members.objects.filter(Q(profile=profile.id),~Q(group__in=group_id)
INFO:
group_id = ['128','52']
group is a foreign key to group model
My problem is when I run this query, it throws Database error: Lookup type 'in' can't be negated.
I have also performed query using __in it worked fine but does not work for foreign key.
Thanks in advance

I think you trying to filter profile id and remove groups in group_id in single filter
groups = Group_members.objects.filter(Q(profile=profile.id),~Q(group__in=group_id)
instead try this:
1)first filter the profiles form group_member :
groups = Group_members.objects.filter(profile=profile.id)
2)remove the groups form Queryset by:
groupId = [x.group.id for x in groups if x.group.id not in group_id]
Hope this will give you perfect result

2 suggestions.
Use ~Q(group__ pk__in=group_id)
Instead of using filter and not in, use exclude and in