I have basically a username is unique (case insensitive), but the case matters when displaying as provided by the user.
I have the following requirements:
field is CharField compatible
field is unique, but case insensitive
field needs to be searchable ignoring case (avoid using iexact, easily forgotten)
field is stored with case intact
preferably enforced on database level
preferably avoid storing an extra field
Is this possible in Django?
The only solution I came up with is "somehow" override the Model manager, use an extra field, or always use 'iexact' in searches.
I'm on Django 1.3 and PostgreSQL 8.4.2.
As of Django 1.11, you can use CITextField, a Postgres-specific Field for case-insensitive text backed by the citext type.
from django.db import models
from django.contrib.postgres.fields import CITextField
class Something(models.Model):
foo = CITextField()
Django also provides CIEmailField and CICharField, which are case-insensitive versions of EmailField and CharField.
Store the original mixed-case string in a plain text column. Use the data type text or varchar without length modifier rather than varchar(n). They are essentially the same, but with varchar(n) you have to set an arbitrary length limit, that can be a pain if you want to change later. Read more about that in the manual or in this related answer by Peter Eisentraut #serverfault.SE.
Create a functional unique index on lower(string). That's the major point here:
CREATE UNIQUE INDEX my_idx ON mytbl(lower(name));
If you try to INSERT a mixed case name that's already there in lower case you get a unique key violation error.
For fast equality searches use a query like this:
SELECT * FROM mytbl WHERE lower(name) = 'foo' --'foo' is lower case, of course.
Use the same expression you have in the index (so the query planner recognizes the compatibility) and this will be very fast.
As an aside: you may want to upgrade to a more recent version of PostgreSQL. There have been lots of important fixes since 8.4.2. More on the official Postgres versioning site.
With overriding the model manager, you have two options. First is to just create a new lookup method:
class MyModelManager(models.Manager):
def get_by_username(self, username):
return self.get(username__iexact=username)
class MyModel(models.Model):
...
objects = MyModelManager()
Then, you use get_by_username('blah') instead of get(username='blah'), and you don't have to worry about forgetting iexact. Of course that then requires that you remember to use get_by_username.
The second option is much hackier and convoluted. I'm hesitant to even suggest it, but for completeness sake, I will: override filter and get such that if you forget iexact when querying by username, it will add it for you.
class MyModelManager(models.Manager):
def filter(self, **kwargs):
if 'username' in kwargs:
kwargs['username__iexact'] = kwargs['username']
del kwargs['username']
return super(MyModelManager, self).filter(**kwargs)
def get(self, **kwargs):
if 'username' in kwargs:
kwargs['username__iexact'] = kwargs['username']
del kwargs['username']
return super(MyModelManager, self).get(**kwargs)
class MyModel(models.Model):
...
objects = MyModelManager()
As of December 2021, with the help of Django 4.0 UniqueConstraint expressions you can add a Meta class to your model like this:
class Meta:
constraints = [
models.UniqueConstraint(
Lower('<field name>'),
name='<constraint name>'
),
]
I'm by no mean a Django professional developer and I don't know technical considerations like performance issues about this solution. Hope others comment on that.
Since a username is always lowercase, it's recommended to use a custom lowercase model field in Django. For the ease of access and code-tidiness, create a new file fields.py in your app folder.
from django.db import models
from django.utils.six import with_metaclass
# Custom lowecase CharField
class LowerCharField(with_metaclass(models.SubfieldBase, models.CharField)):
def __init__(self, *args, **kwargs):
self.is_lowercase = kwargs.pop('lowercase', False)
super(LowerCharField, self).__init__(*args, **kwargs)
def get_prep_value(self, value):
value = super(LowerCharField, self).get_prep_value(value)
if self.is_lowercase:
return value.lower()
return value
Usage in models.py
from django.db import models
from your_app_name.fields import LowerCharField
class TheUser(models.Model):
username = LowerCharField(max_length=128, lowercase=True, null=False, unique=True)
End Note : You can use this method to store lowercase values in the database, and not worry about __iexact.
You can use citext postgres type instead and not bother anymore with any sort of iexact. Just make a note in model that underlying field is case insensitive.
Much easier solution.
You can use lookup='iexact' in UniqueValidator on serializer, like this:
Unique model field in Django and case sensitivity (postgres)
I liked Chris Pratt's Answer but it didn't worked for me, because the models.Manager-class doesn't have the get(...) or filter(...) Methods.
I had to take an extra step via a custom QuerySet:
from django.contrib.auth.base_user import BaseUserManager
from django.db.models import QuerySet
class CustomUserManager(BaseUserManager):
# Use the custom QuerySet where get and filter will change 'email'
def get_queryset(self):
return UserQuerySet(self.model, using=self._db)
def create_user(self, email, password, **extra_fields):
...
def create_superuser(self, email, password, **extra_fields):
...
class UserQuerySet(QuerySet):
def filter(self, *args, **kwargs):
if 'email' in kwargs:
# Probably also have to replace...
# email_contains -> email_icontains,
# email_exact -> email_iexact,
# etc.
kwargs['email__iexact'] = kwargs['email']
del kwargs['email']
return super().filter(*args, **kwargs)
def get(self, *args, **kwargs):
if 'email' in kwargs:
kwargs['email__iexact'] = kwargs['email']
del kwargs['email']
return super().get(*args, **kwargs)
This worked for me in a very simple case but is working pretty good so far.
You can also override get_prep_value() and reuse it through inheritance.
class LowerCaseField:
def get_prep_value(self, value):
value = super().get_prep_value(value)
if value:
value = value.strip().lower()
return value
class LowerSlugField(LowerCaseField, models.SlugField):
pass
class LowerEmailField(LowerCaseField, models.EmailField):
pass
class MyModel(models.Model):
email = LowerEmailField(max_length=255, unique=True)
This way, if you ever want to reuse this field in another model, you can use the same consistent strategy.
From Django Docs:
get_prep_value(value)
value is the current value of the model’s
attribute, and the method should return data in a format that has been
prepared for use as a parameter in a query.
See Converting Python objects to query values for usage.
Related
Django has various numeric fields available for use in models, e.g. DecimalField and PositiveIntegerField. Although the former can be restricted to the number of decimal places stored and the overall number of characters stored, is there any way to restrict it to storing only numbers within a certain range, e.g. 0.0-5.0 ?
Failing that, is there any way to restrict a PositiveIntegerField to only store, for instance, numbers up to 50?
Update: now that Bug 6845 has been closed, this StackOverflow question may be moot. - sampablokuper
You can use Django's built-in validators—
from django.db.models import IntegerField, Model
from django.core.validators import MaxValueValidator, MinValueValidator
class CoolModelBro(Model):
limited_integer_field = IntegerField(
default=1,
validators=[
MaxValueValidator(100),
MinValueValidator(1)
]
)
Edit: When working directly with the model, make sure to call the model full_clean method before saving the model in order to trigger the validators. This is not required when using ModelForm since the forms will do that automatically.
You could also create a custom model field type - see http://docs.djangoproject.com/en/dev/howto/custom-model-fields/#howto-custom-model-fields
In this case, you could 'inherit' from the built-in IntegerField and override its validation logic.
The more I think about this, I realize how useful this would be for many Django apps. Perhaps a IntegerRangeField type could be submitted as a patch for the Django devs to consider adding to trunk.
This is working for me:
from django.db import models
class IntegerRangeField(models.IntegerField):
def __init__(self, verbose_name=None, name=None, min_value=None, max_value=None, **kwargs):
self.min_value, self.max_value = min_value, max_value
models.IntegerField.__init__(self, verbose_name, name, **kwargs)
def formfield(self, **kwargs):
defaults = {'min_value': self.min_value, 'max_value':self.max_value}
defaults.update(kwargs)
return super(IntegerRangeField, self).formfield(**defaults)
Then in your model class, you would use it like this (field being the module where you put the above code):
size = fields.IntegerRangeField(min_value=1, max_value=50)
OR for a range of negative and positive (like an oscillator range):
size = fields.IntegerRangeField(min_value=-100, max_value=100)
What would be really cool is if it could be called with the range operator like this:
size = fields.IntegerRangeField(range(1, 50))
But, that would require a lot more code since since you can specify a 'skip' parameter - range(1, 50, 2) - Interesting idea though...
from django.db import models
from django.core.validators import MinValueValidator, MaxValueValidator
size = models.IntegerField(validators=[MinValueValidator(0),
MaxValueValidator(5)])
I had this very same problem; here was my solution:
SCORE_CHOICES = zip( range(1,n), range(1,n) )
score = models.IntegerField(choices=SCORE_CHOICES, blank=True)
There are two ways to do this. One is to use form validation to never let any number over 50 be entered by a user. Form validation docs.
If there is no user involved in the process, or you're not using a form to enter data, then you'll have to override the model's save method to throw an exception or limit the data going into the field.
Here is the best solution if you want some extra flexibility and don't want to change your model field. Just add this custom validator:
#Imports
from django.core.exceptions import ValidationError
class validate_range_or_null(object):
compare = lambda self, a, b, c: a > c or a < b
clean = lambda self, x: x
message = ('Ensure this value is between %(limit_min)s and %(limit_max)s (it is %(show_value)s).')
code = 'limit_value'
def __init__(self, limit_min, limit_max):
self.limit_min = limit_min
self.limit_max = limit_max
def __call__(self, value):
cleaned = self.clean(value)
params = {'limit_min': self.limit_min, 'limit_max': self.limit_max, 'show_value': cleaned}
if value: # make it optional, remove it to make required, or make required on the model
if self.compare(cleaned, self.limit_min, self.limit_max):
raise ValidationError(self.message, code=self.code, params=params)
And it can be used as such:
class YourModel(models.Model):
....
no_dependents = models.PositiveSmallIntegerField("How many dependants?", blank=True, null=True, default=0, validators=[validate_range_or_null(1,100)])
The two parameters are max and min, and it allows nulls. You can customize the validator if you like by getting rid of the marked if statement or change your field to be blank=False, null=False in the model. That will of course require a migration.
Note: I had to add the validator because Django does not validate the range on PositiveSmallIntegerField, instead it creates a smallint (in postgres) for this field and you get a DB error if the numeric specified is out of range.
Hope this helps :) More on Validators in Django.
PS. I based my answer on BaseValidator in django.core.validators, but everything is different except for the code.
In the forms.py
Class FloatForm(forms.ModelForm):
class Meta:
model = Float
fields = ('name','country', 'city', 'point', 'year')
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.fields['point'] = forms.FloatField(max_value=100, min_value=1)
It is worth mentioning that sometimes Django validation doesn't work as Django validation is mostly an application-level validation, not validation at the database level. Also, Model validation is not run automatically on the save/create/update of the model. If you want to validate your values instantly in your code then you need to do it manually — using the override save() method:
class UserRating():
SCORE_CHOICES = (
(1, _("Terrible")),
(2, _("Poor")),
(3, _("Average")),
(4, _("Very Good")),
(5, _("Excellent")),
)
score = models.PositiveSmallIntegerField(
choices=SCORE_CHOICES, default=1,
validators=[
MaxValueValidator(5),
MinValueValidator(1)
]
)
def save(self, *args, **kwargs):
if int(self.score) < 1 or int(self.score) > 5:
raise ValidationError('Score must be located between 0 to 5')
super(UserRating, self).save(*args, **kwargs)
...
Add validator like this your model column in models.py
class Planogram(models.Model):
camera = models.ForeignKey(Camera, on_delete=models.CASCADE)
xtl = models.DecimalField(decimal_places=10, max_digits=11,validators=[MaxValueValidator(1),MinValueValidator(0)])
if you are using create function to create objects change it to constructor like below....
and call fullclean() on that object and then save..
everything will work perfectly.
planogram = Planogram(camera_id = camera,xtl=xtl,ytl=ytl,xbr=xbr,ybr=ybr,product_id=product_id)
planogram.full_clean()
planogram.save()
I would like a model to generate automatically a random alphanumeric string as its primary key when I create a new instance of it.
example:
from django.db import models
class MyTemporaryObject(models.Model):
id = AutoGenStringField(lenght=16, primary_key=True)
some_filed = ...
some_other_field = ...
in my mind the key should look something like this "Ay3kJaBdGfcadZdao03293". It's for very temporary use. In case of collision I would like it Django to try a new key.
I was wondering if there was already something out there, or a very simple solution I am not seeing (I am fairly new to python and Django). Otherwise I was thinking to do my own version of models.AutoField, would that be the right approach?
I have already found how to generate the key here, so it's not about the string generation. I would just like to have it work seamlessly with a simple Django service without adding too much complexity to the code.
EDIT:
Possible solution? What do you think?
id = models.CharField(unique=True, primary_key=True, default=StringKeyGenerator(), editable=False)
with
class StringKeyGenerator(object):
def __init__(self, len=16):
self.lenght = len
def __call__(self):
return ''.join(random.choice(string.letters + string.digits) for x in range(self.lenght))
I came up with it after going through the Django documentation one more time.
One of the simplest way to generate unique strings in python is to use uuid module. If you want to get alphanumeric output, you can simply use base64 encoding as well:
import uuid
import base64
uuid = base64.b64encode(uuid.uuid4().bytes).replace('=', '')
# sample value: 1Ctu77qhTaSSh5soJBJifg
You can then put this code in the model's save method or define a custom model field using it.
Here's how I would do it without making the field a primary key:
from django.db import IntegrityError
class MyTemporaryObject(models.Model):
auto_pseudoid = models.CharField(max_length=16, blank=True, editable=False, unique=True)
# add index=True if you plan to look objects up by it
# blank=True is so you can validate objects before saving - the save method will ensure that it gets a value
# other fields as desired
def save(self, *args, **kwargs):
if not self.auto_pseudoid:
self.auto_pseudoid = generate_random_alphanumeric(16)
# using your function as above or anything else
success = False
failures = 0
while not success:
try:
super(MyTemporaryObject, self).save(*args, **kwargs)
except IntegrityError:
failures += 1
if failures > 5: # or some other arbitrary cutoff point at which things are clearly wrong
raise
else:
# looks like a collision, try another random value
self.auto_pseudoid = generate_random_alphanumeric(16)
else:
success = True
Two problems that this avoids, compared to using the field as the primary key are that:
1) Django's built in relationship fields require integer keys
2) Django uses the presence of the primary key in the database as a sign that save should update an existing record rather than insert a new one. This means if you do get a collision in your primary key field, it'll silently overwrite whatever else used to be in the row.
Try this:
The if statement below is to make sure that the model is update able.
Without the if statement you'll update the id field everytime you resave the model, hence creating a new model everytime
from uuid import uuid4
from django.db import IntegrityError
class Book(models.Model):
id = models.CharField(primary_key=True, max_length=32)
def save(self, *args, **kwargs):
if self.id:
super(Book, self).save(*args, **kwargs)
return
unique = False
while not unique:
try:
self.id = uuid4().hex
super(Book, self).save(*args, **kwargs)
except IntegrityError:
self.id = uuid4().hex
else:
unique = True
The code snippet below uses the secrets library that comes with Python, handles id collisions, and continues to pass integrity errors when there isn't an id collision.
example of the ids 0TCKybG1qgAhRuEN , yJariA4QN42E9aLf , AZOMrzlkJ-RKh4dp
import secrets
from django.db import models, IntegrityError
class Test(models.Model):
pk = models.CharField(primary_key=True, max_length=32)
def save(self, *args, **kwargs):
unique = False
while not unique:
try:
self.pk = secrets.token_urlsafe(12)
super(Test, self).save(*args, **kwargs)
except IntegrityError as e :
# keep raising the exception if it's not id collision error
if not 'pk' in e.args[0]:
unique = True
raise e
else:
unique = True
Is there any plugin or 3rd party backend to manage redis connections in Django, so the methods in view.py don't have to explicitly connect to redis for every request?
If not, how would you start implementing one? A new plugin? a new backend? a new django middleware?
Thank you.
I think the emerging standard for non-rel databases is django-nonrel . I don't know if django-nonrel is production ready or if support redis, but they have a guide on writing a custom no-sql backend.
Unfortunately, i don't think that writing support for a redis on standard django is easy as writing a DatabaseBackend. There's a lot in django models mechanics and workflow that simply assumes an ACID database. What about syncdb ? And about Querysets?
However, you may try to write a poor-mans approach using models.Manager and a lot of tweaking on your model. For example:
# helper
def fill_model_instance(instance, values):
""" Fills an model instance with the values from dict values """
attributes = filter(lambda x: not x.startswith('_'), instance.__dict__.keys())
for a in attributes:
try:
setattr(instance, a, values[a.upper()])
del values[a.upper()]
except:
pass
for v in values.keys():
setattr(instance, v, values[v])
return instance
class AuthorManager( models.Manager ):
# You may try to use the default methods.
# But should be freaking hard...
def get_query_set(self):
raise NotImplementedError("Maybe you can write a Non relational Queryset()! ")
def latest(self, *args, **kwargs):
# redis Latest query
pass
def filter(self, *args, **kwargs):
# redis filter query
pass
# Custom methods that you may use, instead of rewriting
# the defaults ones.
def open_connection(self):
# Open a redis connection
pass
def search_author( self, *args, **kwargs ):
self.open_connection()
# Write your query. I don't know how this shiny non-sql works.
# Assumes it returns a dict for every matched author.
authors_list = [{'name': 'Leibniz', 'email': 'iinventedcalculus#gmail.com'},
'name': 'Kurt Godel','email': 'self.consistent.error#gmail.com'}]
return [fill_instance(Author(), author) for author in authors_list]
class Author( models.Model ):
name = models.CharField( max_length = 255 )
email = models.EmailField( max_length = 255 )
def save(self):
raise NotImplementedError("TODO: write a redis save")
def delete(self):
raise NotImplementedError(""TODO: write a delete save")
class Meta:
managed = False
Please not that i've only made an sketch of how you can tweak the django models. I have not
tested and run this code. I first suggest you to investigate django-nonrel.
I do not care about concurrency issues.
It is relatively easy to build unique form field:
from django import forms
class UniqueUserEmailField(forms.CharField):
def clean(self, value):
self.check_uniqueness(super(UniqueUserEmailField, self).clean(value))
def check_uniqueness(self, value):
same_user = users.User.all().filter('email', value).get()
if same_user:
raise forms.ValidationError('%s already_registered' % value)
so one could add users on-the-fly. Editing existing user is tricky. This field would not allow to save user having other user email. At the same time it would not allow to save a user with the same email. What code do you use to put a field with uniqueness check into ModelForm?
quick and dirty way would be:
make check_uniqueness classmethod
use custom field check in ModelForm, like this:
class User(forms.ModelForm):
email = forms.EmailField()
def clean_email(self):
data = self.cleaned_data['email']
original = self.instance.email
if original == data:
return data
UniqueUserEmailField.check_uniqueness(data)
return data
better options?
Django has various numeric fields available for use in models, e.g. DecimalField and PositiveIntegerField. Although the former can be restricted to the number of decimal places stored and the overall number of characters stored, is there any way to restrict it to storing only numbers within a certain range, e.g. 0.0-5.0 ?
Failing that, is there any way to restrict a PositiveIntegerField to only store, for instance, numbers up to 50?
Update: now that Bug 6845 has been closed, this StackOverflow question may be moot. - sampablokuper
You can use Django's built-in validators—
from django.db.models import IntegerField, Model
from django.core.validators import MaxValueValidator, MinValueValidator
class CoolModelBro(Model):
limited_integer_field = IntegerField(
default=1,
validators=[
MaxValueValidator(100),
MinValueValidator(1)
]
)
Edit: When working directly with the model, make sure to call the model full_clean method before saving the model in order to trigger the validators. This is not required when using ModelForm since the forms will do that automatically.
You could also create a custom model field type - see http://docs.djangoproject.com/en/dev/howto/custom-model-fields/#howto-custom-model-fields
In this case, you could 'inherit' from the built-in IntegerField and override its validation logic.
The more I think about this, I realize how useful this would be for many Django apps. Perhaps a IntegerRangeField type could be submitted as a patch for the Django devs to consider adding to trunk.
This is working for me:
from django.db import models
class IntegerRangeField(models.IntegerField):
def __init__(self, verbose_name=None, name=None, min_value=None, max_value=None, **kwargs):
self.min_value, self.max_value = min_value, max_value
models.IntegerField.__init__(self, verbose_name, name, **kwargs)
def formfield(self, **kwargs):
defaults = {'min_value': self.min_value, 'max_value':self.max_value}
defaults.update(kwargs)
return super(IntegerRangeField, self).formfield(**defaults)
Then in your model class, you would use it like this (field being the module where you put the above code):
size = fields.IntegerRangeField(min_value=1, max_value=50)
OR for a range of negative and positive (like an oscillator range):
size = fields.IntegerRangeField(min_value=-100, max_value=100)
What would be really cool is if it could be called with the range operator like this:
size = fields.IntegerRangeField(range(1, 50))
But, that would require a lot more code since since you can specify a 'skip' parameter - range(1, 50, 2) - Interesting idea though...
from django.db import models
from django.core.validators import MinValueValidator, MaxValueValidator
size = models.IntegerField(validators=[MinValueValidator(0),
MaxValueValidator(5)])
I had this very same problem; here was my solution:
SCORE_CHOICES = zip( range(1,n), range(1,n) )
score = models.IntegerField(choices=SCORE_CHOICES, blank=True)
There are two ways to do this. One is to use form validation to never let any number over 50 be entered by a user. Form validation docs.
If there is no user involved in the process, or you're not using a form to enter data, then you'll have to override the model's save method to throw an exception or limit the data going into the field.
Here is the best solution if you want some extra flexibility and don't want to change your model field. Just add this custom validator:
#Imports
from django.core.exceptions import ValidationError
class validate_range_or_null(object):
compare = lambda self, a, b, c: a > c or a < b
clean = lambda self, x: x
message = ('Ensure this value is between %(limit_min)s and %(limit_max)s (it is %(show_value)s).')
code = 'limit_value'
def __init__(self, limit_min, limit_max):
self.limit_min = limit_min
self.limit_max = limit_max
def __call__(self, value):
cleaned = self.clean(value)
params = {'limit_min': self.limit_min, 'limit_max': self.limit_max, 'show_value': cleaned}
if value: # make it optional, remove it to make required, or make required on the model
if self.compare(cleaned, self.limit_min, self.limit_max):
raise ValidationError(self.message, code=self.code, params=params)
And it can be used as such:
class YourModel(models.Model):
....
no_dependents = models.PositiveSmallIntegerField("How many dependants?", blank=True, null=True, default=0, validators=[validate_range_or_null(1,100)])
The two parameters are max and min, and it allows nulls. You can customize the validator if you like by getting rid of the marked if statement or change your field to be blank=False, null=False in the model. That will of course require a migration.
Note: I had to add the validator because Django does not validate the range on PositiveSmallIntegerField, instead it creates a smallint (in postgres) for this field and you get a DB error if the numeric specified is out of range.
Hope this helps :) More on Validators in Django.
PS. I based my answer on BaseValidator in django.core.validators, but everything is different except for the code.
In the forms.py
Class FloatForm(forms.ModelForm):
class Meta:
model = Float
fields = ('name','country', 'city', 'point', 'year')
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.fields['point'] = forms.FloatField(max_value=100, min_value=1)
It is worth mentioning that sometimes Django validation doesn't work as Django validation is mostly an application-level validation, not validation at the database level. Also, Model validation is not run automatically on the save/create/update of the model. If you want to validate your values instantly in your code then you need to do it manually — using the override save() method:
class UserRating():
SCORE_CHOICES = (
(1, _("Terrible")),
(2, _("Poor")),
(3, _("Average")),
(4, _("Very Good")),
(5, _("Excellent")),
)
score = models.PositiveSmallIntegerField(
choices=SCORE_CHOICES, default=1,
validators=[
MaxValueValidator(5),
MinValueValidator(1)
]
)
def save(self, *args, **kwargs):
if int(self.score) < 1 or int(self.score) > 5:
raise ValidationError('Score must be located between 0 to 5')
super(UserRating, self).save(*args, **kwargs)
...
Add validator like this your model column in models.py
class Planogram(models.Model):
camera = models.ForeignKey(Camera, on_delete=models.CASCADE)
xtl = models.DecimalField(decimal_places=10, max_digits=11,validators=[MaxValueValidator(1),MinValueValidator(0)])
if you are using create function to create objects change it to constructor like below....
and call fullclean() on that object and then save..
everything will work perfectly.
planogram = Planogram(camera_id = camera,xtl=xtl,ytl=ytl,xbr=xbr,ybr=ybr,product_id=product_id)
planogram.full_clean()
planogram.save()