I need to store a complex number in a Django model. For those who forget, that simply means Z=R+jX where R and X are real numbers representing the real and imaginary components of the complex. There will be individual numbers, as well as lists that need to be stored. My searches so far haven't provided a good solution for lists, so I intend to let the database handle the list as individual records.
I see two options for storing a complex number:
1) create a custom field: class Complex(models.CharField)
This would allow me to customize all aspects of the field, but that is a lot of extra work for validation if it is to be done properly. The major upside is that a single number is represented by a single field in the table.
2) let each complex number be represented by a row, with a float field for the real part, R, and another float field for the imaginary part, X. The downside to this approach is that I would need to write some converters that will create a complex number from the components, and vice versa. The upside is that the database will just see it as another record.
Surely this issue has been resolved in the past, but I can't find any good references, never mind one particular to Django.
This is my first crack at the field, it is based on another example I found that involved a few string manipulations. What isn't clear to me is how and where various validations should be performed (such as coercing a simple float into a complex number by adding +0j). I intend to add form functionality as well, so that the field behaves like a float field, but with additional restrictions or requirements.
I have not tested this code yet, so there may be issues with it. It is based on the code from an answer in this SO question. It appears after running the code that some changes took place in method names.
What is the most efficient way to store a list in the Django models?
class ComplexField(models.CharField):
description = 'A complex number represented as a string'
def __init__(self, *args, **kwargs):
kwargs['verbose_name'] = 'Complex Number'
kwargs['max_length'] = 64
kwargs['default'] = '0+0j'
super().__init__(*args, **kwargs)
def to_python(self, value):
if not value: return
if isinstance(value, complex):
return value
return complex(value)
def get_db_prep_value(self, value):
if not value: return
assert(isinstance(value, complex))
return str(item)[1:-1]
def value_to_string(self, obj):
value = self._get_val_from_obj(obj)
return self.get_db_prep_value(value)
Regarding custom fields, you've probably found the relevant part in the Django documentation already.
Whether a custom field (or a custom database type, see below) is worth the trouble really depends on what you need to do with the stored numbers. For storage and some occasional pushing around, you can go with the easiest sane solution (your number two as enhanced by Tobit).
With PostgreSQL, you have to possibility to implement custom types directly in the database, including operators. Here's the relevant part in the Postgres docs, complete with a complex numbers example, no less.
Of course you then need to expose the new type and the operators to Django. Quite a bit of work, but then you could do arithmetics with individual fields right in the database using Django ORM.
If your expression every time like R + jX you can make the following class
class ComplexNumber(models.Model):
real_number = models.FloatField('Real number part')
img_number = models.FloatFoeld('Img number part')
def __str__(self):
return complex(self.real_number, self.img_number)
and handle the outcome string with python see here
If you have multiple real and img part you can handle this with foreign keys or ManyToMany Fields. This maybe depend on your need.
To be honest, I'd just split the complex number into two float/decimal fields and add a property for reading and writing as a single complex number.
I came up with this custom field that ends up as a split field on the actual model and injects the aforementioned property too.
contribute_to_class is called deep in the Django model machinery for all the fields that are declared on the model. Generally, they might just add the field itself to the model, and maybe additional methods like get_latest_by_..., but here we're hijacking that mechanism to instead add two fields we construct within, and not the actual "self" field itself at all, as it does not need to exist as a database column. (This might break something, who knows...) Some of this mechanism is explained here in the Django wiki.
The ComplexProperty class is a property descriptor, which allows customization of what happens when the property it's "attached as" into an instance is accessed (read or written). (How descriptors work is a little bit beyond the scope of this answer, but there's a how-to guide in the Python docs.)
NB: I did not test this beyond running migrations, so things may be broken in unexpected ways, but at least the theory is sound. :)
from django.db import models
class ComplexField(models.Field):
def __init__(self, **kwargs):
self.field_class = kwargs.pop('field_class', models.FloatField)
self.field_kwargs = kwargs.pop('field_kwargs', {})
super().__init__(**kwargs)
def contribute_to_class(self, cls, name, private_only=False):
for field in (
self.field_class(name=name + '_real', **self.field_kwargs),
self.field_class(name=name + '_imag', **self.field_kwargs),
):
field.contribute_to_class(cls, field.name)
setattr(cls, name, ComplexProperty(name))
class ComplexProperty:
def __init__(self, name):
self.name = name
def __get__(self, instance, owner):
if not instance:
return self
real = getattr(instance, self.name + '_real')
imag = getattr(instance, self.name + '_imag')
return complex(real, imag)
def __set__(self, instance, value: complex):
setattr(instance, self.name + '_real', value.real)
setattr(instance, self.name + '_imag', value.imag)
class Test(models.Model):
num1 = ComplexField()
num2 = ComplexField()
num3 = ComplexField()
The migration for this looks like
migrations.CreateModel(
name="Test",
fields=[
(
"id",
models.AutoField(
auto_created=True, primary_key=True, serialize=False, verbose_name="ID"
),
),
("num1_real", models.FloatField()),
("num1_imag", models.FloatField()),
("num2_real", models.FloatField()),
("num2_imag", models.FloatField()),
("num3_real", models.FloatField()),
("num3_imag", models.FloatField()),
],
)
so as you can see, the three ComplexFields are broken down into six FloatFields.
Related
When we define a model in django we write something like..
class Student(models.Model):
name = models.CharField(max_length=64)
age = models.IntegerField()
...
where, name = models.CharField() implies that name would be an object of models.CharField. When we have to make an object of student we simple do..
my_name = "John Doe"
my_age = 18
s = Student.objects.create(name=my_name, age=my_age)
where my_name and my_age are string and integer data types respectively, and not an object of models.CharField/models.IntegerField. Although while assigning the values the respective validations are performed (like checking on the max_length for CharField)
I'm trying to build similar models for an abstraction of Neo4j over Django but not able to get this workflow. How can I implement this ?
Found a similar question but didn't find it helpful enough.
How things work
First thing I we need to understand that each field on your models has own validation, this one refer to the CharField(_check_max_length_attribute) and it also calling the super on method check from the Field class to validate some basic common things.
That in mind, we now move to the create method which is much more complicated and total different thing, the basics operations for specific object:
Create a python object
Call save()
Using a lot of getattrs the save does tons of validation
Commit to the DB, if anything wrong goes from the DB, raise it to the user
A third thing you need to understand that when you query an object it first get the data from the db, and then(after long process) it set the data to the object.
Simple Example
class BasicCharField:
def __init__(self, max_len):
self.max_len = max_len
def validate(self, value):
if value > self.max_len:
raise ValueError('the value must be lower than {}'.format(self.max_len))
class BasicModel:
score = BasicCharField(max_len=4)
#staticmethod
def create(**kwargs):
obj = BasicModel()
obj.score = kwargs['score']
obj.save()
return obj
def save(self):
# Lots of validations here
BasicModel.score.validate(self.score)
# DB commit here
BasicModel.create(score=5)
And like we was expecting:
>>> ValueError: the value must be lower than 4
Obviously I had to simplify things to make it into few lines of code, you can improve this by a lot (like iterate over the attribute and not hardcode it like obj.score = ...)
I have a Django model where a lot of fields are choices. So I had to write a lot of "is_something" properties of the class to check whether the instance value is equal to some choice value. Something along the lines of:
class MyModel(models.Model):
some_choicefield = models.IntegerField(choices=SOME_CHOICES)
#property
def is_some_value(self):
return self.some_choicefield == SOME_CHOICES.SOME_CHOICE_VALUE
# a lot of these...
In order to automate this and spare me a lot of redundant code, I thought about patching the instance at creation, with a function that adds a bunch of methods that do the checks.
The code became as follows (I'm assuming there's a "normalize" function that makes the label of the choice a usable function name):
def dynamic_add_checks(instance, field):
if hasattr(field, 'choices'):
choices = getattr(field, 'choices')
for (value,label) in choices:
def fun(instance):
return getattr(instance, field.name) == value
normalized_func_name = "is_%s_%s" % (field.name, normalize(label))
setattr(instance, normalized_func_name, fun(instance))
class MyModel(models.Model):
def __init__(self, *args, **kwargs):
super(MyModel).__init__(*args, **kwargs)
dynamic_add_checks(self, self._meta.get_field('some_choicefield')
some_choicefield = models.IntegerField(choices=SOME_CHOICES)
Now, this works but I have the feeling there is a better way to do it. Perhaps at class creation time (with metaclasses or in the new method)? Do you have any thoughts/suggestions about that?
Well I am not sure how to do this in your way, but in such cases I think the way to go is to simply create a new model, where you keep your choices, and change the field to ForeignKey. This is simpler to code and manage.
You can find a lot of information at a basic level in Django docs: Models: Relationships. In there, there are many links to follow expanding on various topics. Beyong that, I believe it just needs a bit of imagination, and maybe trial and error in the beginning.
I came across a similar problem where I needed to write large number of properties at runtime to provide backward compatibility while changing model fields. There are 2 standard ways to handle this -
First is to use a custom metaclass in your models, which inherits from models default metaclass.
Second, is to use class decorators. Class decorators sometimes provides an easy alternative to metaclasses, unless you have to do something before the creation of class, in which case you have to go with metaclasses.
I bet you know Django fields with choices provided will automatically have a display function.
Say you have a field defined like this:
category = models.SmallIntegerField(choices=CHOICES)
You can simply call a function called get_category_display() to access the display value. Here is the Django source code of this feature:
https://github.com/django/django/blob/baff4dd37dabfef1ff939513fa45124382b57bf8/django/db/models/base.py#L962
https://github.com/django/django/blob/baff4dd37dabfef1ff939513fa45124382b57bf8/django/db/models/fields/init.py#L704
So we can follow this approach to achieve our dynamically set property goal.
Here is my scenario, a little bit different from yours but down to the end it's the same:
I have two classes, Course and Lesson, class Lesson has a ForeignKey field of Course, and I want to add a property name cached_course to class Lesson which will try to get Course from cache first, and fallback to database if cache misses:
Here is a typical solution:
from django.db import models
class Course(models.Model):
# some fields
class Lesson(models.Model):
course = models.ForeignKey(Course)
#property
def cached_course(self):
key = key_func()
course = cache.get(key)
if not course:
course = get_model_from_db()
cache.set(key, course)
return course
Turns out I have so many ForeignKey fields to cache, so here is the code following the similar approach of Django get_FIELD_display feature:
from django.db import models
from django.utils.functional import curry
class CachedForeignKeyField(models.ForeignKey):
def contribute_to_class(self, cls, name, **kwargs):
super(models.ForeignKey, self).contribute_to_class(cls, name, **kwargs)
setattr(cls, "cached_%s" % self.name,
property(curry(cls._cached_FIELD, field=self)))
class BaseModel(models.Model):
def _cached_FIELD(self, field):
value = getattr(self, field.attname)
Model = field.related_model
return cache.get_model(Model, pk=value)
class Meta:
abstract = True
class Course(BaseModel):
# some fields
class Lesson(BaseModel):
course = CachedForeignKeyField(Course)
By customizing CachedForeignKeyField, and overwrite the contribute_to_class method, along with BaseModel class with a _cached_FIELD method, every CachedForeignKeyField will automatically have a cached_FIELD property accordingly.
Too good to be true, bravo!
Say I have an object, "Order," a field of which, "items," holds a list of order items. The list of items will never be searched or individually selected in the database so I just want to store it in a DB field as a JSON string.
I'm trying to figure out the best way to embed this functionality so it's fairly transparent to anyone using the model. I think saving the model is pretty easy - just override the save method and serialize the "items" list into an internal "_items" field, and then write that to the db. I'm confused about how to deserialize, though. Having looked into possibly some kind of classmethod for creation, or creating a custom manger, or something to do with signals, I've thoroughly confused myself. I'm sure this has been solved a hundred times over and I'm curious what people consider to be best practice.
Example classes:
class OrderItem():
def __init__(self, desc="", qty=0):
self.desc = desc
self.qty = qty
class Order(Model):
user = ForeignKey(User)
_items = TextField()
def save(self, *args, **kwargs):
self._items = jsonpickle.encode(self.items)
super(Order, self).save(*args, **kwargs)
Example usage:
order = Order()
order.items = [OrderItem("widget", 5)]
order.save()
This would create a record in the DB in which
_items = [{"desc":"widget", "qty":5}]
Now I want to be able to later select the object
order = Order.objects.get(id=whatever)
and have order.items be the unpacked array of items, not the stored JSON string.
EDIT:
The solution turned out to be quite simple, and I'm posting here in case it helps any other newbies. Based on Daniel's suggestion, I went with this custom model field:
class JSONField(with_metaclass(SubfieldBase, TextField)):
def db_type(self, connection):
return 'JSONField'
def to_python(self, value):
if isinstance(value, basestring):
return jsonpickle.decode(value)
else:
return value
def get_prep_value(self, value):
return jsonpickle.encode(value)
A much better approach is to subclass TextField and override the relevant methods to do the serialization/deserialization transparently as required. In fact there are a number of implementations of this already: here's one, for example.
I'm trying to sort my QuerySet based on how the objects in the QuerySet are evaluated as Strings.
So my model looks something like this:
class System(models.Model):
operating_system = models.CharField(...)
language = models.CharField(...)
locale = models.CharField(...)
def __unicode__(self):
def __clean(orig, new):
if orig is None or orig == "":
if new is None or new == "":
return ""
else:
return str(new)
else:
if new is None or new == "":
return str(orig)
else:
return str(orig) + " " + str(new)
name = None
for attr in System._meta.fields:
if attr.name != "id":
name = __clean(name, getattr(self, attr.name))
for m2mfield in System._meta.many_to_many:
for object in getattr(self, m2mfield.name).all():
name = __clean(name, object)
if name == "":
return "Undefined"
return name
And, I'd like to be able to make a query something like:
System.objects.filter(...).order_by('__unicode__')
I'm wondering if there's a way to do this without a custom manager.
Thanks!
In __unicode__ you eventually end up with a single string that represents a System object. Instead of calculating it every time you need it, calculate it once and save it onto the model.
class System(models.Model):
operating_system = models.CharField(...)
language = models.CharField(...)
locale= models.CharField(...)
name = models.CharField(editable=False, ...)
def save(self, *args, **kwargs):
self.name = self._calculate_name()
super(System, self).save(*args, **kwargs)
def __unicode__(self):
return self.name
def _calculate_name(self):
# all that string manipulation and relationship stuff
Now you can order by this name easily
System.objects.filter(...).order_by('name')
There are some caveats to this approach, it really depends on the usage of System. Also, do NOT worry about the space, that's my opinion!
Expanding on caveats
Since this field is 'denormalized', it suffers from the same problems other relational data that isn't normalized face. Denormalization can introduce update anomalies (a field or relation that name depends on can change without a change to name if the change happens through some other route than the System model's save() method. It can also slow down writes (in this case probably by a very small amount), it can increase space requirements (again not a problem here in my opinion), and a whole wack of other stuff that Google would love to tell you about I'm sure.
I think all you have to be careful about is updating .name whenever it should be, so consider carefully under what conditions your 'cleaning' code will produce different results. If, for example, you had an OS table where you can change the description of an OS without touching the System table, then you have to realize your .name will not be updated by a save to OS, it will require recalculation. There are mechanisms to help with this like signals and overriding more save() methods. You could also batch update them as necessary.
It really depends heavily on your use case, which is not fully illustrated here. SO is full of people that could help you narrow down the best solution if you present your use case more completely.
I wanted to write some code like this:
class SomeModel(models.Model):
field = models.ForeignKey(SomeOtherModel)
def __init__(self, *args, **kwargs):
super(SomeModel, self).__init__(*args, **kwargs)
if self.field is None:
self.field = SomeOtherModel()
...
However this raises self.field.rel.to.DoesNotExist. The Django code is very clear on that:
class ReverseSingleRelatedObjectDescriptor(object):
def __get__(self, instance, instance_type=None):
...
if val is None:
# If NULL is an allowed value, return it.
if self.field.null:
return None
raise self.field.rel.to.DoesNotExist
An obvious workaround would be of course to make the field nullable however as far as I understand that would actually have an effect on the database schema, also I like the integrity checks Django offers. Another one would be to catch the exception and handle it appropriately. However this adds a lot of boilerplate code. Especially when there are multiple fields like that (a separate try...except block for each one of them - now that's ugly). What could I do?
I could use initial however this is quite limited when it comes to foreign keys. I do not always know the kind of default that I would like to have at the moment of creation. I will however know it at the initialization phase. Moreover it could then be dependent on the values of the other fields.
Check if it has the attribute set -
if hasattr(self, 'field')
proper way to refer a field in form is like this:
self.fields['myfield']
so, in your case, the null check should go like this
self.fields['myfield'].value is None
on the other note, don't use reserved/near to reserved words like 'field' for naming your fields.