I wanted to write some code like this:
class SomeModel(models.Model):
field = models.ForeignKey(SomeOtherModel)
def __init__(self, *args, **kwargs):
super(SomeModel, self).__init__(*args, **kwargs)
if self.field is None:
self.field = SomeOtherModel()
...
However this raises self.field.rel.to.DoesNotExist. The Django code is very clear on that:
class ReverseSingleRelatedObjectDescriptor(object):
def __get__(self, instance, instance_type=None):
...
if val is None:
# If NULL is an allowed value, return it.
if self.field.null:
return None
raise self.field.rel.to.DoesNotExist
An obvious workaround would be of course to make the field nullable however as far as I understand that would actually have an effect on the database schema, also I like the integrity checks Django offers. Another one would be to catch the exception and handle it appropriately. However this adds a lot of boilerplate code. Especially when there are multiple fields like that (a separate try...except block for each one of them - now that's ugly). What could I do?
I could use initial however this is quite limited when it comes to foreign keys. I do not always know the kind of default that I would like to have at the moment of creation. I will however know it at the initialization phase. Moreover it could then be dependent on the values of the other fields.
Check if it has the attribute set -
if hasattr(self, 'field')
proper way to refer a field in form is like this:
self.fields['myfield']
so, in your case, the null check should go like this
self.fields['myfield'].value is None
on the other note, don't use reserved/near to reserved words like 'field' for naming your fields.
Related
I need to store a complex number in a Django model. For those who forget, that simply means Z=R+jX where R and X are real numbers representing the real and imaginary components of the complex. There will be individual numbers, as well as lists that need to be stored. My searches so far haven't provided a good solution for lists, so I intend to let the database handle the list as individual records.
I see two options for storing a complex number:
1) create a custom field: class Complex(models.CharField)
This would allow me to customize all aspects of the field, but that is a lot of extra work for validation if it is to be done properly. The major upside is that a single number is represented by a single field in the table.
2) let each complex number be represented by a row, with a float field for the real part, R, and another float field for the imaginary part, X. The downside to this approach is that I would need to write some converters that will create a complex number from the components, and vice versa. The upside is that the database will just see it as another record.
Surely this issue has been resolved in the past, but I can't find any good references, never mind one particular to Django.
This is my first crack at the field, it is based on another example I found that involved a few string manipulations. What isn't clear to me is how and where various validations should be performed (such as coercing a simple float into a complex number by adding +0j). I intend to add form functionality as well, so that the field behaves like a float field, but with additional restrictions or requirements.
I have not tested this code yet, so there may be issues with it. It is based on the code from an answer in this SO question. It appears after running the code that some changes took place in method names.
What is the most efficient way to store a list in the Django models?
class ComplexField(models.CharField):
description = 'A complex number represented as a string'
def __init__(self, *args, **kwargs):
kwargs['verbose_name'] = 'Complex Number'
kwargs['max_length'] = 64
kwargs['default'] = '0+0j'
super().__init__(*args, **kwargs)
def to_python(self, value):
if not value: return
if isinstance(value, complex):
return value
return complex(value)
def get_db_prep_value(self, value):
if not value: return
assert(isinstance(value, complex))
return str(item)[1:-1]
def value_to_string(self, obj):
value = self._get_val_from_obj(obj)
return self.get_db_prep_value(value)
Regarding custom fields, you've probably found the relevant part in the Django documentation already.
Whether a custom field (or a custom database type, see below) is worth the trouble really depends on what you need to do with the stored numbers. For storage and some occasional pushing around, you can go with the easiest sane solution (your number two as enhanced by Tobit).
With PostgreSQL, you have to possibility to implement custom types directly in the database, including operators. Here's the relevant part in the Postgres docs, complete with a complex numbers example, no less.
Of course you then need to expose the new type and the operators to Django. Quite a bit of work, but then you could do arithmetics with individual fields right in the database using Django ORM.
If your expression every time like R + jX you can make the following class
class ComplexNumber(models.Model):
real_number = models.FloatField('Real number part')
img_number = models.FloatFoeld('Img number part')
def __str__(self):
return complex(self.real_number, self.img_number)
and handle the outcome string with python see here
If you have multiple real and img part you can handle this with foreign keys or ManyToMany Fields. This maybe depend on your need.
To be honest, I'd just split the complex number into two float/decimal fields and add a property for reading and writing as a single complex number.
I came up with this custom field that ends up as a split field on the actual model and injects the aforementioned property too.
contribute_to_class is called deep in the Django model machinery for all the fields that are declared on the model. Generally, they might just add the field itself to the model, and maybe additional methods like get_latest_by_..., but here we're hijacking that mechanism to instead add two fields we construct within, and not the actual "self" field itself at all, as it does not need to exist as a database column. (This might break something, who knows...) Some of this mechanism is explained here in the Django wiki.
The ComplexProperty class is a property descriptor, which allows customization of what happens when the property it's "attached as" into an instance is accessed (read or written). (How descriptors work is a little bit beyond the scope of this answer, but there's a how-to guide in the Python docs.)
NB: I did not test this beyond running migrations, so things may be broken in unexpected ways, but at least the theory is sound. :)
from django.db import models
class ComplexField(models.Field):
def __init__(self, **kwargs):
self.field_class = kwargs.pop('field_class', models.FloatField)
self.field_kwargs = kwargs.pop('field_kwargs', {})
super().__init__(**kwargs)
def contribute_to_class(self, cls, name, private_only=False):
for field in (
self.field_class(name=name + '_real', **self.field_kwargs),
self.field_class(name=name + '_imag', **self.field_kwargs),
):
field.contribute_to_class(cls, field.name)
setattr(cls, name, ComplexProperty(name))
class ComplexProperty:
def __init__(self, name):
self.name = name
def __get__(self, instance, owner):
if not instance:
return self
real = getattr(instance, self.name + '_real')
imag = getattr(instance, self.name + '_imag')
return complex(real, imag)
def __set__(self, instance, value: complex):
setattr(instance, self.name + '_real', value.real)
setattr(instance, self.name + '_imag', value.imag)
class Test(models.Model):
num1 = ComplexField()
num2 = ComplexField()
num3 = ComplexField()
The migration for this looks like
migrations.CreateModel(
name="Test",
fields=[
(
"id",
models.AutoField(
auto_created=True, primary_key=True, serialize=False, verbose_name="ID"
),
),
("num1_real", models.FloatField()),
("num1_imag", models.FloatField()),
("num2_real", models.FloatField()),
("num2_imag", models.FloatField()),
("num3_real", models.FloatField()),
("num3_imag", models.FloatField()),
],
)
so as you can see, the three ComplexFields are broken down into six FloatFields.
This is an example Python 2 code:
from django.db import models
def my_validator(value):
assert isinstance(value, (int, long))
class Foo(models.Model):
name = models.CharField(...) # irrelevant here
class Bar(models.Model):
name = models.CharField(...) # irrelevant here
foo = models.ForeignKey(Foo, validators=[my_validator])
If I create a Foo instance, then a Bar instance (assigning the foo instance), and then validate, this code passes: the FK value to validate is not a model instance but an ID (which is an integer, by default):
foo = Foo.objects.create(name='foo')
bar = Bar.objects.create(name='bar', foo=foo)
Edit: I forgot to include the full_clean() call. But yes: the troublesome code calls full_clean(). In fact, the first time I noticed this behavior was when trying to treat the value in the validator callable, as a model instance instead of a raw value, which triggered a int value has no attribute xxx when trying to invoke an instance method inside the validator.
bar.full_clean()
This happens in Django 1.9. Is this documented and expected?
Yes - this is implicitly referred to in the documentation for ForeignKey.to_field:
The field on the related object that the relation is to. By default, Django uses the primary key of the related object.
Also:
For fields like ForeignKey that map to model instances, defaults should be the value of the field they reference (pk unless to_field is set) instead of model instances.
That is, by default, the value of the ForeignKey is the primary key of the related object - i.e., an integer.
You can however specify a different to_field, in which case the value would take the type of that field.
In terms of what value is passed to the validators, it seems that the assumption is implicit that this is the to_field (what else would you validate other than the value that is going to be stored in the database? It does not make much sense to pass a model object when validating a foreign key, because the key itself is only a pointer to the object and does not say anything about what that object should be.).
But to answer your question - there doesn't appear to be any explicit documentation stating this.
I'm not sure that #solarissmoke answer is relevant to the question.
IMO, validation is not invoked at objects.create, if you want to validate your model before creating it you should either use a ModelForm, or call it manually.
foo = Foo.objects.create(name='foo')
bar = Bar(name='bar', foo=foo)
try:
bar.full_clean()
bar.save()
except ValidationError as e:
# Do something based on the errors contained in e.message_dict.
# Display them to a user, or handle them programmatically.
pass
UPDATE:
OK, so what exactly is happening is that when you call .full_clean() we get .clean_fields() called.
Inside the clean_fields we have something like:
raw_value = getattr(self, f.attname)
if f.blank and raw_value in f.empty_values:
continue
try:
setattr(self, f.attname, f.clean(raw_value, self))
except ValidationError as e:
errors[f.name] = e.error_list
Where two thing happens:
We get raw_value for the field
We call field.clean
In the field.clean() we have .to_python(), validate() and .run_validators() called in this order, its something like:
value = self.to_python(value)
self.validate(value)
self.run_validators(value)
return value
Which Django explains here: Form and field validation
BUT, that's not the reason why you get int/long in your custom validator.
The reason is because ForeignKey fields store their values in an attribute with _id at the end, which equals to f.attname. So during the whole process of validating FKs Django works with int/long values, not with objects.
If you see the ForeignKey.validate method, you will find out that it just checks if a row with that id exists.
I have a Django model where a lot of fields are choices. So I had to write a lot of "is_something" properties of the class to check whether the instance value is equal to some choice value. Something along the lines of:
class MyModel(models.Model):
some_choicefield = models.IntegerField(choices=SOME_CHOICES)
#property
def is_some_value(self):
return self.some_choicefield == SOME_CHOICES.SOME_CHOICE_VALUE
# a lot of these...
In order to automate this and spare me a lot of redundant code, I thought about patching the instance at creation, with a function that adds a bunch of methods that do the checks.
The code became as follows (I'm assuming there's a "normalize" function that makes the label of the choice a usable function name):
def dynamic_add_checks(instance, field):
if hasattr(field, 'choices'):
choices = getattr(field, 'choices')
for (value,label) in choices:
def fun(instance):
return getattr(instance, field.name) == value
normalized_func_name = "is_%s_%s" % (field.name, normalize(label))
setattr(instance, normalized_func_name, fun(instance))
class MyModel(models.Model):
def __init__(self, *args, **kwargs):
super(MyModel).__init__(*args, **kwargs)
dynamic_add_checks(self, self._meta.get_field('some_choicefield')
some_choicefield = models.IntegerField(choices=SOME_CHOICES)
Now, this works but I have the feeling there is a better way to do it. Perhaps at class creation time (with metaclasses or in the new method)? Do you have any thoughts/suggestions about that?
Well I am not sure how to do this in your way, but in such cases I think the way to go is to simply create a new model, where you keep your choices, and change the field to ForeignKey. This is simpler to code and manage.
You can find a lot of information at a basic level in Django docs: Models: Relationships. In there, there are many links to follow expanding on various topics. Beyong that, I believe it just needs a bit of imagination, and maybe trial and error in the beginning.
I came across a similar problem where I needed to write large number of properties at runtime to provide backward compatibility while changing model fields. There are 2 standard ways to handle this -
First is to use a custom metaclass in your models, which inherits from models default metaclass.
Second, is to use class decorators. Class decorators sometimes provides an easy alternative to metaclasses, unless you have to do something before the creation of class, in which case you have to go with metaclasses.
I bet you know Django fields with choices provided will automatically have a display function.
Say you have a field defined like this:
category = models.SmallIntegerField(choices=CHOICES)
You can simply call a function called get_category_display() to access the display value. Here is the Django source code of this feature:
https://github.com/django/django/blob/baff4dd37dabfef1ff939513fa45124382b57bf8/django/db/models/base.py#L962
https://github.com/django/django/blob/baff4dd37dabfef1ff939513fa45124382b57bf8/django/db/models/fields/init.py#L704
So we can follow this approach to achieve our dynamically set property goal.
Here is my scenario, a little bit different from yours but down to the end it's the same:
I have two classes, Course and Lesson, class Lesson has a ForeignKey field of Course, and I want to add a property name cached_course to class Lesson which will try to get Course from cache first, and fallback to database if cache misses:
Here is a typical solution:
from django.db import models
class Course(models.Model):
# some fields
class Lesson(models.Model):
course = models.ForeignKey(Course)
#property
def cached_course(self):
key = key_func()
course = cache.get(key)
if not course:
course = get_model_from_db()
cache.set(key, course)
return course
Turns out I have so many ForeignKey fields to cache, so here is the code following the similar approach of Django get_FIELD_display feature:
from django.db import models
from django.utils.functional import curry
class CachedForeignKeyField(models.ForeignKey):
def contribute_to_class(self, cls, name, **kwargs):
super(models.ForeignKey, self).contribute_to_class(cls, name, **kwargs)
setattr(cls, "cached_%s" % self.name,
property(curry(cls._cached_FIELD, field=self)))
class BaseModel(models.Model):
def _cached_FIELD(self, field):
value = getattr(self, field.attname)
Model = field.related_model
return cache.get_model(Model, pk=value)
class Meta:
abstract = True
class Course(BaseModel):
# some fields
class Lesson(BaseModel):
course = CachedForeignKeyField(Course)
By customizing CachedForeignKeyField, and overwrite the contribute_to_class method, along with BaseModel class with a _cached_FIELD method, every CachedForeignKeyField will automatically have a cached_FIELD property accordingly.
Too good to be true, bravo!
I'm looking at a state isolation / read-only situation in Django (1.6) and i'm looking for a method to make a query return objects that are immutable.
I'm looking to fit something like the following wrapping the usual db atomicity api
MyModel.objects.filter(foo="bar").all(read_only=True)
My current thinking is this will be a custom Manager, but i'd potentially like something that can be added at runtime like:
read_only(MyModel.objects.filter(foo="bar").all())
Without too much voodoo or making them unmanaged (the option to throw an Exception on state change would be good).
The key thing is that the Model supports both read-only and the default read-write query type ideally with changes limited to code that is required to be read-only.
My other option is something like:
with isolation(raise_exception=True):
m = MyModel.objects.get(id=foo)
m.do_unknown_thing_that_may_mutate()
Are there existing solutions I'm missing at a higher level than the database?
One possibility might be to define a proxy class which overrides save to be a no-op:
class MyReadOnlyModel(MyModel):
def save(self, *args, **kwargs):
pass
class Meta:
proxy = True
Then just query MyReadOnlyModel instead of MyModel.
Say I have an object, "Order," a field of which, "items," holds a list of order items. The list of items will never be searched or individually selected in the database so I just want to store it in a DB field as a JSON string.
I'm trying to figure out the best way to embed this functionality so it's fairly transparent to anyone using the model. I think saving the model is pretty easy - just override the save method and serialize the "items" list into an internal "_items" field, and then write that to the db. I'm confused about how to deserialize, though. Having looked into possibly some kind of classmethod for creation, or creating a custom manger, or something to do with signals, I've thoroughly confused myself. I'm sure this has been solved a hundred times over and I'm curious what people consider to be best practice.
Example classes:
class OrderItem():
def __init__(self, desc="", qty=0):
self.desc = desc
self.qty = qty
class Order(Model):
user = ForeignKey(User)
_items = TextField()
def save(self, *args, **kwargs):
self._items = jsonpickle.encode(self.items)
super(Order, self).save(*args, **kwargs)
Example usage:
order = Order()
order.items = [OrderItem("widget", 5)]
order.save()
This would create a record in the DB in which
_items = [{"desc":"widget", "qty":5}]
Now I want to be able to later select the object
order = Order.objects.get(id=whatever)
and have order.items be the unpacked array of items, not the stored JSON string.
EDIT:
The solution turned out to be quite simple, and I'm posting here in case it helps any other newbies. Based on Daniel's suggestion, I went with this custom model field:
class JSONField(with_metaclass(SubfieldBase, TextField)):
def db_type(self, connection):
return 'JSONField'
def to_python(self, value):
if isinstance(value, basestring):
return jsonpickle.decode(value)
else:
return value
def get_prep_value(self, value):
return jsonpickle.encode(value)
A much better approach is to subclass TextField and override the relevant methods to do the serialization/deserialization transparently as required. In fact there are a number of implementations of this already: here's one, for example.