Develop Reference

Fill a ChoiceField in Django template with external data

I'm new to Django and I'm having a hard time understanding forms when the data to choose from are not taken from the database nor user input that they're generated on the go.
I currently have a template with a single ChoiceField. The data inside this field aren't fixed and they're calculated on the go once the page is requested. To calculate it I need the username of the User who is logged in. Basically, the calculation returns a list of lists in the form of ((title, id),(title,id),(title,id)), etc. that I need to put into the ChoiceField to make the User choose from one of the options.
Now, I'm not understanding how to pass the calculated list of lists to the form. I've tried to add the calculations inside the form as below but it is clearly the wrong way.
The main issue is that, to calculate my list of lists, I need the request value, and I don't know how to access it from the form.
Another idea was to add the generate_selection function inside the init but then I don't know how to pass main_playlist to being able to add it to ChoiceField
Below my not working forms.py
forms.py
class ChoosePlaylistForm(forms.Form):
playlists = forms.ChoiceField(choices=HERE_SHOULD_GO_main_playlist)
def generate_selection(self):
sp_auth, cache_handler = spotify_oauth2(self.request)
spotify = spotipy.Spotify(oauth_manager=sp_auth)
user_playlists = spotify.current_user_playlists(limit=10)
main_playlist = []
for playlists in user_playlists["items"]:
playlists_list = []
playlists_list.append(playlists['name'])
playlists_list.append(playlists['id'])
main_playlist.append(playlists_list)
return main_playlist
def __init__(self, *args, **kwargs):
self.request = kwargs.pop('request', None)
super(ChoosePlaylistForm, self).__init__(*args, **kwargs)
class Meta:
model = User
fields = ('playlists',)
The views should be something like below so I'm able to pass the request
views.py
form = ChoosePlaylistForm(request=request)

Maybe overriding the field choices in the form constructor would work:
class ChoosePlaylistForm(forms.Form):
playlists = forms.ChoiceField(choices=())
class Meta:
model = User
fields = ('playlists',)
def __init__(self, *args, request=None, **kwargs):
super(ChoosePlaylistForm, self).__init__(*args, **kwargs)
self.request = request
self.fields['playlists'].choices = self.generate_selection()
def generate_selection(self):
sp_auth, cache_handler = spotify_oauth2(self.request)
spotify = spotipy.Spotify(oauth_manager=sp_auth)
user_playlists = spotify.current_user_playlists(limit=10)
choices = []
for playlist in user_playlists["items"]:
playlist_choice = (playlist["name"], playlist["id"])
choices.append(playlist_choice)
return choices

Finding objects without relationship in django

I am learning Django, and want to retrieve all objects that DON'T have a relationship to the current object I am looking at.
The idea is a simple Twitter copycat.
I am trying to figure out how to implement get_non_followers.
from django.db import models
RELATIONSHIP_FOLLOWING = 1
RELATIONSHIP_BLOCKED = 2
RELATIONSHIP_STATUSES = (
(RELATIONSHIP_FOLLOWING, 'Following'),
(RELATIONSHIP_BLOCKED, 'Blocked'),
)
class UserProfile(models.Model):
name = models.CharField(max_length=200)
website = models.CharField(max_length=200)
email = models.EmailField()
relationships = models.ManyToManyField('self', through='Relationship',
symmetrical=False,
related_name='related_to')
def __unicode__ (self):
return self.name
def add_relationship(self, person, status):
relationship, created = Relationship.objects.get_or_create(
from_person=self,
to_person=person,
status=status)
return relationship
def remove_relationship(self, person, status):
Relationship.objects.filter(
from_person=self,
to_person=person,
status=status).delete()
return
def get_relationships(self, status):
return self.relationships.filter(
to_people__status=status,
to_people__from_person=self)
def get_related_to(self, status):
return self.related_to.filter(
from_people__status=status,
from_people__to_person=self)
def get_following(self):
return self.get_relationships(RELATIONSHIP_FOLLOWING)
def get_followers(self):
return self.get_related_to(RELATIONSHIP_FOLLOWING)
def get_non_followers(self):
# How to do this?
return
class Relationship(models.Model):
from_person = models.ForeignKey(UserProfile, related_name='from_people')
to_person = models.ForeignKey(UserProfile, related_name='to_people')
status = models.IntegerField(choices=RELATIONSHIP_STATUSES)

This isn't particularly glamorous, but it gives correct results (just tested):
def get_non_followers(self):
UserProfile.objects.exclude(to_people=self,
to_people__status=RELATIONSHIP_FOLLOWING).exclude(id=self.id)
In short, use exclude() to filter out all UserProfiles following the current user, which will leave the user themselves (who probably shouldn't be included) and all users not following them.

i'v been searching for a method or some way to do that for like an hour, but i found nothing.
but there is a way to do that.
you can simply use a for loop to iterate through all objects and just remove all objects that they have a special attribute value.
there is a sample code here:
all_objects = className.objects.all()
for obj in all_objects:
if obj.some_attribute == "some_value":
all_objects.remove(obj)

Solution to the implementation of get_non_followers:
def get_non_following(self):
return UserProfile.objects.exclude(to_person__from_person=self, to_person__status=RELATIONSHIP_FOLLOWING).exclude(id=self.id)
This answer was posted as an edit to the question Finding objects without relationship in django by the OP Avi Meir under CC BY-SA 3.0.

current_userprofile = current_user.get_profile()
rest_of_users = Set(UserProfile.objects.filter(user != current_userprofile))
follow_relationships = current_userprofile.relationships.filter(from_person=current_user)
followers = Set();
for follow in follow_relationships:
followers.add(follow.to_person)
non_followeres = rest_of_users.difference(followers)
Here non_followers is the list of userprofiles you desire. current_user is the user whose non_followers you are trying to find.

I haven't tested this out, but it think it should do what you want.
def get_non_followers(self):
return self.related_to.exclude(
from_people__to_person=self)

django heterogeneous queryset proxy models

I am trying to figure out how to use proxy classes in Django. I want to receive a queryset where each object belongs to a proxy class of a common super class so that I can run custom sub-classed methods with the same name and my controller logic doesn't need to know or care about which kind of Proxy model it is working with. One thing I don't want to do is to store the information in multiple tables because I want to have unified identifiers for easier reference/management.
I am pretty new to django/python so I would be happy to hear alternative ways to accomplish what I am trying to do.
Here is what I have:
TYPES = (
('aol','AOL'),
('yhoo','Yahoo'),
)
class SuperConnect(models.Model):
name = models.CharField(max_length=90)
type = models.CharField(max_length=45, choices = TYPES)
connection_string = models.TextField(null=True)
class ConnectAOL(SuperConnect):
class Meta:
proxy = True
def connect(self):
conn_options = self.deconstruct_constring()
# do special stuff to connect to AOL
def deconstruct_constring(self):
return pickle.loads(self.connection_string)
class ConnectYahoo(SuperConnect):
class Meta:
proxy = True
def connect(self):
conn_options = self.deconstruct_constring()
# do special stuff to connect to Yahoo
def deconstruct_constring(self):
return pickle.loads(self.connection_string)
Now what I want to do is this:
connections = SuperConnect.objects.all()
for connection in connections:
connection.connect()
connection.dostuff
I've looked around and found some hacks but they look questionable and may require me to go to the database for each item in order to retrieve data I probably already have...
Somebody please rescue me :) or I am going to go with this hack:
class MixedQuerySet(QuerySet):
def __getitem__(self, k):
item = super(MixedQuerySet, self).__getitem__(k)
if item.atype == 'aol':
yield(ConnectAOL.objects.get(id=item.id))
elif item.atype == 'yhoo':
yield(ConnectYahoo.objects.get(id=item.id))
else:
raise NotImplementedError
def __iter__(self):
for item in super(MixedQuerySet, self).__iter__():
if item.atype == 'aol':
yield(ConnectAOL.objects.get(id=item.id))
elif item.atype == 'yhoo':
yield(ConnectYahoo.objects.get(id=item.id))
else:
raise NotImplementedError
class MixManager(models.Manager):
def get_query_set(self):
return MixedQuerySet(self.model)
TYPES = (
('aol','AOL'),
('yhoo','Yahoo'),
)
class SuperConnect(models.Model):
name = models.CharField(max_length=90)
atype = models.CharField(max_length=45, choices = TYPES)
connection_string = models.TextField(null=True)
objects = MixManager()
class ConnectAOL(SuperConnect):
class Meta:
proxy = True
def connect(self):
conn_options = self.deconstruct_constring()
# do special stuff to connect to AOL
def deconstruct_constring(self):
return pickle.loads(self.connection_string)
class ConnectYahoo(SuperConnect):
class Meta:
proxy = True
def connect(self):
conn_options = self.deconstruct_constring()
# do special stuff to connect to Yahoo
def deconstruct_constring(self):
return pickle.loads(self.connection_string)

As you mentioned in your question, the problem with your solution is that it generates a SQL query for every object instead of using one SQL in = (id1, id2) query. Proxy models cannot contain additional database fields, so there is no need for extra SQL queries.
Instead, you can convert a SuperConnect object to the appropriate type in SuperConnect.__init__, using the __class__ attribute:
class SuperConnect(models.Model):
name = models.CharField(max_length=90)
type = models.CharField(max_length=45, choices = TYPES)
connection_string = models.TextField(null=True)
def __init__(self, *args, **kwargs):
super(SuperConnect, self).__init__(*args, **kwargs)
if self.type == 'aol':
self.__class__ = ConnectAOL
elif self.type == 'yahoo':
self.__class__ = ConnectYahoo
There is no need for custom managers or querysets, the correct type is set when the SuperConnect object is initialized.

How about putting all the logic in one class. Something like this:
def connect(self):
return getattr(self, "connect_%s" % self.type)()
def connect_aol(self):
pass # AOL stuff
def connect_yahoo(self):
pass # Yahoo! stuff
In the end you have your type field and you should be able to do most (if not all) things that you can do with seperate proxy classes.
If this approach doesn't solve your specific use cases, please clarify.

email whitelist/blacklist in python/django

I am writing a django app that keeps track of which email addresses are allowed to post content to a user's account. The user can whitelist and blacklist addresses as they like.
Any addresses that aren't specified can either be handled per message or just default to whitelist or blacklist (again user specified).
Here are the django models I wrote... do you think is a good way to do it? or should I add a whitelist and blacklist field to each user's profile model?
class knownEmail(models.Model):
# The user who set this address' permission, NOT
# the user who the address belongs to...
relatedUser = models.ManyToManyField(User)
email = models.EmailField()
class whiteList(knownEmail):
pass
class blackList(knownEmail):
pass
Then I could do something like:
def checkPermission(user, emailAddress):
"Check if 'emailAddress' is allowed to post content to 'user's profile"
if whiteList.objects.filter(relatedUser=user, email=emailAddress):
return True
elif blackList.objects.filter(relatedUser=user, email=emailAddress):
return False
else:
return None
Is there a better way?

I would restructure it so both lists were contained in one model.
class PermissionList(models.Model):
setter = models.ManyToManyField(User)
email = models.EmailField(unique=True) #don't want conflicting results
permission = models.BooleanField()
Then, your lists would just be:
# whitelist
PermissionList.objects.filter(permission=True)
# blacklist
PermissionList.objects.filter(permission=False)
To check a particular user, you just add a couple functions to the model:
class PermissionList(...):
...
#classmethod
def is_on_whitelist(email):
return PermissionList.objects.filter(email=email, permission=True).count() > 0
#classmethod
def is_on_blacklist(email):
return PermissionList.objects.filter(email=email, permission=False).count() > 0
#classmethod
def has_permission(email):
if PermissionList.is_on_whitelist(email):
return True
if PermissionList.is_on_blacklist(email):
return False
return None
Having everything in one place is a lot simpler, and you can make more interesting queries with less work.

[Please start All Class Names With Upper Case Letters.]
Your code doesn't make use of your class distinction very well.
Specifically, your classes don't have any different behavior. Since both classes have all the same methods, it isn't clear why these are two different classes in the first place. If they have different methods, then your solution is good.
If, however, they don't have different methods, you might want to look at providing a customized manager for each of the two subsets of KnownEmail
class WhiteList( models.Manager ):
def get_query_set( self ):
return super( WhiteList, self ).get_query_set().filter( status='W' )
class BlackList( models.Manager )
def get_query_set( self ):
return super( BlackList, self ).get_query_set().filter( status='B' )
class KnownEmail( models.Model ):
relatedUser = models.ForeignKey(User)
email = models.EmailField()
status = models.CharField( max_length=1, choices=LIST_CHOICES )
objects = models.Manager() # default manager shows all lists
whiteList= WhiteList() # KnownEmail.whiteList.all() is whitelist subset
blackList= BlackList() # KnownEmail.blackList.all() is blackList subset

This class compares an email address with a blacklist of email domains. If you preffer you can download this module using pip install django-email-blacklist.
from django.conf import settings
import re
class DisposableEmailChecker():
"""
Check if an email is from a disposable
email service
"""
def __init__(self):
self.emails = [line.strip() for line in open(settings.DISPOSABLE_EMAIL_DOMAINS)]
def chunk(self, l, n):
return (l[i:i + n] for i in range(0, len(l), n))
def is_disposable(self, email):
for email_group in self.chunk(self.emails, 20):
regex = "(.*" + ")|(.*".join(email_group) + ")"
if re.match(regex, email):
return True
return False

Django: Access primary key in models.filefield(upload_to) location

I'd like to save my files using the primary key of the entry.
Here is my code:
def get_nzb_filename(instance, filename):
if not instance.pk:
instance.save() # Does not work.
name_slug = re.sub('[^a-zA-Z0-9]', '-', instance.name).strip('-').lower()
name_slug = re.sub('[-]+', '-', name_slug)
return u'files/%s_%s.nzb' % (instance.pk, name_slug)
class File(models.Model):
nzb = models.FileField(upload_to=get_nzb_filename)
name = models.CharField(max_length=256)
I know the first time an object is saved the primary key isn't available, so I'm willing to take the extra hit to save the object just to get the primary key, and then continue on.
The above code doesn't work. It throws the following error:
maximum recursion depth exceeded while calling a Python object
I'm assuming this is an infinite loop. Calling the save method would call the get_nzb_filename method, which would again call the save method, and so on.
I'm using the latest version of the Django trunk.
How can I get the primary key so I can use it to save my uploaded files?
Update #muhuk:
I like your solution. Can you help me implement it? I've updated my code to the following and the error is 'File' object has no attribute 'create'. Perhaps I'm using what you've written out of context?
def create_with_pk(self):
instance = self.create()
instance.save()
return instance
def get_nzb_filename(instance, filename):
if not instance.pk:
create_with_pk(instance)
name_slug = re.sub('[^a-zA-Z0-9]', '-', instance.name).strip('-').lower()
name_slug = re.sub('[-]+', '-', name_slug)
return u'files/%s_%s.nzb' % (instance.pk, name_slug)
class File(models.Model):
nzb = models.FileField(upload_to=get_nzb_filename, blank=True, null=True)
name = models.CharField(max_length=256)
Instead of enforcing the required field in my model I'll do it in my Form class. No problem.

It seems you'll need to pre-generate your File models with empty file fields first. Then pick up one and save it with the given file object.
You can have a custom manager method like this;
def create_with_pk(self):
instance = self.create()
instance.save() # probably this line is unneeded
return instance
But this will be troublesome if either of your fields is required. Because you are initially creating a null object, you can't enforce required fields on the model level.
EDIT
create_with_pk is supposed to be a custom manager method, in your code it is just a regular method. Hence self is meaningless. It is all properly documented with examples.

You can do this by setting upload_to to a temporary location and by creating a custom save method.
The save method should call super first, to generate the primary key (this will save the file to the temporary location). Then you can rename the file using the primary key and move it to it's proper location. Call super one more time to save the changes and you are good to go! This worked well for me when I came across this exact issue.
For example:
class File( models.Model ):
nzb = models.FileField( upload_to='temp' )
def save( self, *args, **kwargs ):
# Call save first, to create a primary key
super( File, self ).save( *args, **kwargs )
nzb = self.nzb
if nzb:
# Create new filename, using primary key and file extension
oldfile = self.nzb.name
dot = oldfile.rfind( '.' )
newfile = str( self.pk ) + oldfile[dot:]
# Create new file and remove old one
if newfile != oldfile:
self.nzb.storage.delete( newfile )
self.nzb.storage.save( newfile, nzb )
self.nzb.name = newfile
self.nzb.close()
self.nzb.storage.delete( oldfile )
# Save again to keep changes
super( File, self ).save( *args, **kwargs )

Context
Had the same issue.
Solved it attributing an id to the current object by saving the object first.
Method
create a custom upload_to function
detect if object has pk
if not, save instance first, retrieve the pk and assign it to the object
generate your path with that
Sample working code :
class Image(models.Model):
def upload_path(self, filename):
if not self.pk:
i = Image.objects.create()
self.id = self.pk = i.id
return "my/path/%s" % str(self.id)
file = models.ImageField(upload_to=upload_path)

You can create pre_save and post_save signals. Actual file saving will be in post_save, when pk is already created.
Do not forget to include signals in app.py so they work.
Here is an example:
_UNSAVED_FILE_FIELD = 'unsaved_file'
#receiver(pre_save, sender=File)
def skip_saving_file_field(sender, instance: File, **kwargs):
if not instance.pk and not hasattr(instance, _UNSAVED_FILE_FIELD):
setattr(instance, _UNSAVED_FILE_FIELD, instance.image)
instance.nzb = None
#receiver(post_save, sender=File)
def save_file_field(sender, instance: Icon, created, **kwargs):
if created and hasattr(instance, _UNSAVED_FILE_FIELD):
instance.nzb = getattr(instance, _UNSAVED_FILE_FIELD)
instance.save()

Here are 2 possible solutions:
Retrieve id before inserting a row
For simplicity I use postgresql db, although it is possible to adjust implementation for your db backend.
By default django creates id as bigserial (or serial depending on DEFAULT_AUTO_FIELD). For example, this model:
class File(models.Model):
nzb = models.FileField(upload_to=get_nzb_filename)
name = models.CharField(max_length=256)
Produces the following DDL:
CREATE TABLE "example_file" ("id" bigserial NOT NULL PRIMARY KEY, "nzb" varchar(100) NOT NULL, "name" varchar(256) NOT NULL);
There is no explicit sequence specification. By default bigserial creates sequence name in the form of tablename_colname_seq (example_file_id_seq in our case)
The solution is to retrieve this id using nextval :
def get_nextval(model, using=None):
seq_name = f"{model._meta.db_table}_id_seq"
if using is None:
using = "default"
with connections[using].cursor() as cursor:
cursor.execute("select nextval(%s)", [seq_name])
return cursor.fetchone()[0]
And set it before saving the model:
class File(models.Model):
# fields definition
def save(
self, force_insert=False, force_update=False, using=None, update_fields=None
):
if not self.pk:
self.pk = get_nextval(self, using=using)
force_insert = True
super().save(
force_insert=force_insert,
force_update=force_update,
using=using,
update_fields=update_fields,
)
Note that we rely on force_insert behavior, so make sure to read documentation and cover your code with tests:
from django.core.files.uploadedfile import SimpleUploadedFile
from django.forms import ModelForm
from django.test import TestCase
from example import models
class FileForm(ModelForm):
class Meta:
model = models.File
fields = (
"nzb",
"name",
)
class FileTest(TestCase):
def test(self):
form = FileForm(
{
"name": "picture",
},
{
"nzb": SimpleUploadedFile("filename", b"content"),
},
)
self.assertTrue(form.is_valid())
form.save()
self.assertEqual(models.File.objects.count(), 1)
f = models.File.objects.first()
self.assertRegexpMatches(f.nzb.name, rf"files/{f.pk}_picture(.*)\.nzb")
Insert without nzt then update with actual nzt value
The idea is self-explanatory - we basically pop nzt on the object creation and save object again after we know id:
def save(
self, force_insert=False, force_update=False, using=None, update_fields=None
):
nzb = None
if not self.pk:
nzb = self.nzb
self.nzb = None
super().save(
force_insert=force_insert,
force_update=force_update,
using=using,
update_fields=update_fields,
)
if nzb:
self.nzb = nzb
super().save(
force_insert=False,
force_update=True,
using=using,
update_fields=["nzb"],
)
Test is updated to check actual queries:
def test(self):
form = FileForm(
{
"name": "picture",
},
{
"nzb": SimpleUploadedFile("filename", b"content"),
},
)
self.assertTrue(form.is_valid())
with CaptureQueriesContext(connection) as ctx:
form.save()
self.assertEqual(models.File.objects.count(), 1)
f = models.File.objects.first()
self.assertRegexpMatches(f.nzb.name, rf"files/{f.pk}_picture(.*)\.nzb")
self.assertEqual(len(ctx.captured_queries), 2)
insert, update = ctx.captured_queries
self.assertEqual(
insert["sql"],
'''INSERT INTO "example_file" ("nzb", "name") VALUES ('', 'picture') RETURNING "example_file"."id"''',
)
self.assertRegexpMatches(
update["sql"],
rf"""UPDATE "example_file" SET "nzb" = 'files/{f.pk}_picture(.*)\.nzb' WHERE "example_file"."id" = {f.pk}""",
)

Ty, is there a reason you rolled your own slugify filter?
Django ships with a built-in slugify filter, you can use it like so:
from django.template.defaultfilters import slugify
slug = slugify(some_string)
Not sure if you were aware it was available to use...

You can use the next available primary key ID:
class Document(models.Model):
def upload_path(self, filename):
if not self.pk:
document_next_id = Document.objects.order_by('-id').first().id + 1
self.id = self.pk = document_next_id
return "my/path/document-%s" % str(self.pk)
document = models.FileField(upload_to=upload_path)
Details
My example is a modification of #vinyll's answer, however, the problem Giles mentioned in his comment (two objects being created) is resolved here.
I am aware that my answer is not perfect, and there can be issues with the "next available ID", e.g., when more users will attempt to submit many forms at once. Giles's answer is more robust, mine is simpler (no need to generate temp files, then moving files, and deleting them). For simpler applications, this will be enough.
Also credits to Tjorriemorrie for the clear example on how to get the next available ID of an object.

Well I'm not sure of my answer but -
use nested models, if you can -
class File(models.Model):
name = models.CharField(max_length=256)
class FileName(models.Model):
def get_nzb_filename(instance, filename):
return instance.name
name = models.ForeignKey(File)
nzb = models.FileField(upload_to=get_nzb_filename)
And in create method -
File_name = validated_data.pop(file_name_data)
file = File.objects.create(validated_data)
F = FileName.objects.create(name=file, **File_name)