This is still in dev, so I am relatively flexible in how to approach this.
So I'm importing data (via fixtures, the import works fine) from an existing database into a django application. The source data is not uniformed in how it manages ID & primary keys. Some tables use what seems to be an auto-increment (similar to what django would produce by default). Others use some sort of integers. The relationships in the data dump are established based on those fields. Seems I can keep on using auto-increments in all cases.
They are conveniently not named uniformly: id, pk, pk_sometablename, etc.
The fixtures I use to import look like this (I generated them using a script based on the datadump, so this can be changed if needs be):
{
"model": "admin_account.client",
"pk": "168",
"fields":
{
"pk_client": "168",
My django model:
class Client(models.Model):
pk_client = models.IntegerField(verbose_name='Pk_client', blank=True, null=True)
I need to be able to import the data in such a way that this field, the pk_client field is used as the primary key (it can still remain as an auto-increment). So I tried to change to this:
class Client(models.Model):
pk_client = models.AutoField(primary_key=True, verbose_name="pk_client", default=-9999)
However if I try this migration with my currently populated dev DB, I get an error:
django.db.utils.OperationalError: foreign key mismatch - "purchase_orders_apent" referencing "admin_client"
I assume django complains because the apent table used to try to lookup client.id, and since I know tell django to use pk_client as the primary key, perhaps those tables referencing it are now unable to find their match). But there's a lot of tables involved.
What the easiest way to manage this?
Should I completely define those models with an empty DB (e.g. define the AutoFields for each model, assign the old db auto-increment value to that same field, and only THEN import the data)?
Or is there something I should change in my fixture definition/the way I import the data?
Related
I'm developing a Django application that uses mongodb in a part of it. My database schema is changing (some fields are deleted/added continuously), so how to integrate it with Django in a way that accepts the changes without changing the old data, and without affecting the search queries?
I have searched about the available libraries, and I found the below:
mongoengine and django-mongodb-engine: mongoengine is not supported for a while now and not updated, also django-mongodb-engine required a forked old django-nonrel package. This matches with the question: Use pymongo in django directly
djongo: Initially it is working fine with the most updated versions of python and Django, and it accepts the changes in my database models without migrations and Django admin panel is working fine. Later on, after applying some changes, I have faced an issue when it comes to querying the database or listing the data in the admin panel. The old data fails to fit with the new model if the change includes deleted fields.
pymongo: The disadvantage is that I cannot use django models or panel and I have to build my own database abstract layer, but the advantage is about the higher control that I will have over the database. It will be like the first solution in Use pymongo in django directly , then I can build some layers for the different database structures that I will have
Using djongo
Let's say that I have a model called Test:
models.py
from djongo import models
class Test(models.Model):
x = models.CharField(max_length=100)
y = models.CharField(max_length=100)
I have created a new object as below:
{
_id: ObjectId("..."),
x: "x1",
y: "y1"
}
Then, I have removed the y field and added a new field called z, then I have created a new object, so it is created as below:
{
_id: ObjectId("..."),
x: "x2",
z: "z2"
}
Now, I want to extract all the collection data as below:
python manage.py shell
>>Test.objects.all()
Error as field "y" is not exist in the model
>>Test.objects.filter(z="z2")
Error as field "y" is not exist in the model
I can understand that it cannot be mapped to the new changed model, but I want the old fields to be ignored at least without errors exactly like the queries in mongodb directly.
According to my request, it is the wrong approach to use djongo? Or is there any workaround for to handle that issue? If no, how to apply that properly using pymongo? I expect to change my collection fields with addition or deletion anytime, and extracting all the data anytime without errors.
I have a PostgreSQL database that is being used by a front-end application built with Django, but being populated by a scraping tool in Node.js. I have made a sequence that I want to use across two different tables/entities, which can be accessed by a function (nexval(serial)) and is called on every insert. This is not the primary key for these tables, but simply a way to maintain order through some metadata. Using it in Node.js during the insertion of the data into the tables is trivial, as I am using raw SQL queries. However, I am struggling with how to represent this using Django models. There does not seem to be any way to associate this Postgres function with a model's field.
Question:
Is there a way to use a Postgres function as the default value of a Django model field?
you can also set your own function for the default
from django.db import connection, models
def sequence_id():
with connection.cursor() as cursor:
cursor.execute("""SELECT nextval('model_someid_seq')""")
return cursor.fetchone()[0]
class MyModel(models.Model):
field_id = models.IntegerField(default=sequence_id)
My eventual solution:
Override the save method of the model, using a raw query to SELECT nextval('serial') inside the override, setting that as the value of the necessary field, then call save on the parent (super(PARENT, self).save()).
I have to add dynamic fields at run time in my django application,but I don't know the proper way how to add new fields at run time.
I want to add the code which will generate the dynamic field and will update database too. I am using postgresql database. please help if anyone can.
My "model.py" is simply like this:
class Student(models.Model):
name=models.CharField(max_length=100)
school=models.CharField(max_length=100)
created_at=models.DateField(auto_now_add=True)
is_active=models.BooleanField(default=False)
def __str__(self):
return self.name
Django is not made for dynamic models, as relational databases are not. A model change at runtime will create a ton of problems.
You have to simulate it, by...
clever use of related models
storing values in a large field, e.g. JSON as text
having a generic model that stores the data as key, value; e.g. a table with PK, a FK, key, value as columns.
You should try the first option and only if that does not work out try the other two.
Well, I do my first steps with Django and Django REST framework. The problem I face is that all examples throughout the whole Internet are based on hard-coded models. But the whole concept of models frustrates me a little bit, because I'm used to deal with different data which comes from numerous sources (various relational databases and nosql - all that stuff). So, I do not want to stick to a particular model with a fixed number of predefined fields, but I want to specify them just at the moment when a user goes to a particular page of my app.
Let's say I have a table or a collection in one of my databases, which stores information about users - it has any kinds of fields (not just email, name and likewise - all those fields as in all those examples throughout the web). So when a user goes to /users/ I connect to my datebase, get my table, set my cursor and populate my resultant dictionary with all rows and all fields I need. And REST API does all the rest.
So, I need a "first-step" example wich starts from data, not from a model: you have a table "items" in your favorite database, when a user goes to /items/, he or she gets all data from that table. To make such simplistic api, you should do this and this... I need this kind of example.
I think the key is to use the models differently. If you use onetomany or foreignkey references in your model construction you can more dynamically link different types of data together, then access that from the parent object.
For example, for your user, you could create a basic user model and reference that in many other models such as interests, occupation, and have those models store very dynamic data.
When you have the root user model object, you can access it's foreign key objects by either iterating through the dictionary of fields returned by the object or accessing the foreign key references directly with model.reference_set.all()
I am using 0.97-pre-SVN-unknown release of Django.
I have a model for which I have not given any primary_key. Django, consequently, automatically provides an AutoField that is called "id". Everything's fine with that. But now, I have to change the "verbose_name" of that AutoField to something other than "id". I cannot override the "id" field the usual way, because that would require dropping/resetting the entire model and its data (which is strictly not an option). I cannot find another way around it. Does what I want even possible to achieve? If you may suggest any alternatives that would get me away with what I want without having to drop the model/table, I'd be happy.
Hmm... and what about explicitly write id field in the model definition? Like this for example:
class Entry(models.Model):
id = models.AutoField(verbose_name="custom name")
# and other fields...
It doesn't require any underlying database changes.
Look into the command-line options for manage.py; there's a command to dump all of the model data to JSON, and another command to load it back in from JSON. You can export all of your model data, add your new field to the model, then import your data back in. Just make sure that you set the db_column option to 'id' so you don't break your existing data.
Edit: Specifically, you want the commands dumpdata and loaddata.