I have tables called 'has_location' and 'locations'. 'has_location' has user_has and location_id and its own id which is given by django itself.
'locations' have more columns.
Now I want to get all locations of some certain user. What I did is..(user.id is known):
users_locations_id = has_location.objects.filter(user_has__exact=user.id)
locations = Location.objects.filter(id__in=users_locations_id)
print len(locations)
but I am getting 0 by this print. I have data in db. but I have the feeling that __in does not accept the models id, does it ?
thanks
Using __in for this kind of query is a common anti-pattern in Django: it's tempting because of its simplicity, but it scales poorly in most databases. See slides 66ff in this presentation by Christophe Pettus.
You have a many-to-many relationship between users and locations, represented by the has_location table. You would normally describe this to Django using a ManyToManyField with a through table, something like this:
class Location(models.Model):
# ...
class User(models.Model):
locations = models.ManyToManyField(Location, through = 'LocationUser')
# ...
class LocationUser(models.Model):
location = models.ForeignKey(Location)
user = models.ForeignKey(User)
class Meta:
db_table = 'has_location'
Then you can fetch the locations for a user like this:
user.locations.all()
You can query the locations in your filter operations:
User.objects.filter(locations__name = 'Barcelona')
And you can request that users' related locations be fetched efficiently using the prefetch_related() method on a query set.
You are using has_location's own id to filter locations. You have to use location_ids to filter locations:
user_haslocations = has_location.objects.filter(user_has=user)
locations = Location.objects.filter(id__in=user_haslocations.values('location_id'))
You can also filter the locations directly through the reverse relation:
location = Location.objects.filter(has_location__user_has=user.id)
What do your models look like?
For your doubt, __in does accept filtered ids.
For your current code, the solution:
locations = Location.objects.filter(id__in=has_location.objects.filter(user=user).values('location_id'))
# if you just want the length of the locations, evaluate locations.count()
locations.count()
# if you want to iterate locations to access items afterwards
len(locations)
Related
I have two Django models Profile and Device with a ManyToMany relationship with one another like so:
class Profile(models.Model):
devices = models.ManyToManyField(Device, related_name='profiles')
I am trying to use annotate() and Count() to query on all profiles that have 1 or more devices like this:
profiles = Profile.objects.annotate(dev_count=Count('devices')).filter(dev_count__gt=1)
This is great, it gives me a QuerySet with all the profiles (4500+) with one or more devices, as expected.
Next, because of the M2M relationship, I would like to get a list of all the distinct devices among all the profiles from the previous queryset.
All of my failed attempts below return an empty queryset. I have read the documentation on values, values_list, and annotate but I still can't figure out how to make the correct query here.
devices = profiles.values('devices').distinct()
devices = profiles.values_list('devices', flat=True).distinct()
I have also tried to do it in one go:
devices = (
Profile.objects.values_list('devices', flat=True)
.annotate(dev_count=Count('devices'))
.filter(dev_count__gt=1)
.distinct()
)
You can not work with .values() since that that item appears both in the SELECT clause and the GROUP BY clause, so then you start mentioning the field, and hence the COUNT(devices) will return 1 for each group.
You can filter on the Devices that are linked to at least one of these Profiles with:
profiles = Profile.objects.annotate(
dev_count=Count('devices')
).filter(dev_count__gt=1)
devices = Device.objects.filter(profile__in=profiles).distinct()
For some SQL dialects, usually MySQL it is better to first materialize the list of profiles and not work with a subquery, so:
profiles = Profile.objects.annotate(
dev_count=Count('devices')
).filter(dev_count__gt=1)
profiles_list = list(profiles)
devices = Device.objects.filter(profile__in=profiles_list).distinct()
I have a Queryset with a couple of records, and I wan't to remove duplicates using the related model field. For example:
class User(models.Model):
group = models.ForeignKey('Group')
...
class Address(models.Model):
...
models.ForeignKey('User')
addresses = Address.objects.filter(user__group__id=1).order_by('-id')
This returns a QuerySet of Address records, and I want to group by the User ID.
I can't use .annotate because I need all fields from Address, and the relationship between Address and User
I can't use .distinct() because it doesn't work, since all addresses are distinct, and I want distinct user addresses.
I could:
addresses = Address.objects.filter(user__group__id=1).order_by('-id')
unique_users_ids = []
unique_addresses = []
for address in addresses:
if address.user.id not in unique_users_ids:
unique_addresses.append(address)
unique_users_ids.append(address.user.id)
print unique_addresses # TA-DA!
But it seems too much for a simple thing like a group by (damn you Django).
Is there a easy way to achieve this?
By using .distinct() with a field name
Django has also a .distinct(..) function that takes as input column the column names that should be unique. Alas most database systems do not support this (only PostgreSQL to the best of my knowledge). But in PostgreSQL we can thus perform:
# Limited number of database systems support this
addresses = (Address.objects
.filter(user__group__id=1)
.order_by('-id')
.distinct('user_id'))
By using two queries
Another way to handle this is by first having a query that works over the users, and for each user obtains the largest address_id:
from django.db.models import Max
address_ids = (User.objects
.annotate(address_id=Max('address_set__id'))
.filter(address_id__isnull=False)
.values_list('address_id'))
So now for every user, we have calculated the largest corresponding address_id, and we eliminate Users that have no address. We then obtain the list of ids.
In a second step, we then fetch the addresses:
addresses = Address.objects.filter(pk__in=address_ids)
Let's say we have a two models like these:
Artist(models.Model):
name = models.CharField(max_length=50)
Track(models.Model):
title = models.CharField(max_length=50)
artist = models.ForeignKey(Artist, related_name='tracks')
How can I filter this relationship to get the first foreign record?
So I've tried something like this, but it didn't work (as expected)
artists = Artist.objects.filter(tracks__first__title=<some-title>)
artists = Artist.objects.filter(tracks[0]__title=<some-title>)
Is there any way to make this work?
Here's a solution not taking performance into consideration.
Artist.objects.filter(tracks__in=[a.tracks.first() for a in Artist.objects.all()], tracks__title=<some_title>)
No list approach, as requested.
Artist.objects.filter(tracks__in=Track.objects.all().distinct('artist').order_by('artist', 'id'), tracks__title=<some_title>)
The order_by 'id' is important to make sure distinct gets the first track based on insertion. The order_by 'artist' is a requirement for sorting distinct queries. Read about it here: https://www.postgresql.org/docs/9.0/static/sql-select.html#SQL-DISTINCT
I have two tables 'Contact' and other is "Subscriber".. I want to Compare Contact_id of both and want to show only those Contact_id which is present in Contact but not in Subscriber.These two tables are in two different Models.
Something like this should work:
Contact.objects.exclude(
id__in=Subscriber.objects.all()
).values_list('id', flat=True)
Note that these are actually two SQL queries. I'm sure there are ways to optimize it, but this will usually work fine.
Also, the values_list has nothing to do with selecting the objects, it just modifies "format" of what is returned (list of IDs instead of queryset of objects - but same database records in both cases).
If you are excluding by some field other then Subscriber.id (e.g: Subscriber.quasy_id):
Contact.objects.exclude(
id__in=Subscriber.objects.all().values_list('quasy_id', flat=True)
).values_list('id', flat=True)
Edit:
This answer assumes you don't have a relationship between your Contact and Subscriber models. If you do, then see #navit's answer, it is a better choice.
Edit 2:
That flat=True inside exclude is actually not needed.
I assume you have your model like this:
class Subscriber(models.Model):
contact = models.ForeignKey(Contact)
You can do what you want like this:
my_list = Subscriber.objects.filter(contact=None)
This retrieves Subscribers which don't have a Contact. Retrieveing a list of Contacts is straightforward.
If you want to compare value of fields in two different tables(which have connection with ForeignKey) you can use something like this:
I assume model is like below:
class Contact(models.Model):
name = models.TextField()
family = models.TextField()
class Subscriber(models.Model):
subscriber_name = models.ForeignKey(Contact, on_delete=models.CASCADE)
subscriber_family = models.TextField()
this would be the query:
query = Subscriber.objects.filter(subscriber_name =F(Contact__name))
return query
I have a python function that scrapes some data from a few different websites and I want to save that data into my database only if a certain condition is met. Namely, the scraped data should only be saved if the combination of the location and date field is unique
So in my view I have a new location variable and and date variable and essentially I just need to test this combination of values against what's already in the database. If this combination is unique, then save it. If it's not, then do nothing.
class Speech(models.Model):
location = models.ForeignKey(Location)
speaker = models.CharField(max_lenth=100)
date = models.DateField
I'm pretty new to django so I'm just not sure how to go about executing this sort of database query.
You want a combination of two things. First, you want a inner Meta class to enforce the uniqueness in the database:
class Speech(models.Model):
location = models.ForeignKey(Location)
speaker = models.CharField(max_length=100)
date = models.DateField()
class Meta:
unique_together = ('location', 'date')
Then, when you're doing your data manipulation in your view, you want the get_or_create method of the default model manager:
speech, new = Speech.objects.get_or_create(
location=my_location_string,
date=my_datetime_variable,
)
if new:
speech.speaker = my_speaker_string
speech.save()
I hope that gets you started. As always, you know your needs better than I do, so don't blindly copy this example, but adapt it to your needs.
Documentation:
unique_together
get_or_create