If I change a field in a Django model, how can I synchronize it with the database tables? Do I need to do it manually on the database or is there a tool that does helps with the process?
Alas, Django does not support any easy solution to this.
The only thing django will do for you, is restart your database with new tables that match your new models:
$ #DON'T DO THIS UNLESS YOU CAN AFFORD TO LOSE ALL YOUR DATA!
$ python PROJECT_DIR/manage.py syncdb
the next option is to use the various sql* options to manage.py to see what django would do to match the current models to the database, then issue your own ALTER TABLE commands to make everything work right. Of course this is error prone and difficult.
The real solution is to use a database migration tool, such as south to generate migration code.
Here is a similar question with discussion about various database migration options for django.
Can't seem to be able to add a comment to the marked answer, probably because I haven't got enough rep (be nice if SO told me so though).
Anyway, just wanted to add that in the answered post, I believe it is wrong about syncdb - syncdb does not touch tables once they have been created and have data in them. You should not lose data by calling it (otherwise, how could you add new tables for new apps?)
I believe the poster was referring to the reset command instead, which does result in data loss - it will drop the table and recreate it and hence it'll have all the latest model changes.
Django Evolution can help, but the best option really is to plan out your schema in advance, or to make simple modifications manually. Or, to be willing to toast your test data by dropping tables and re-syncing.
Django does not provide for this out of the box.
Here's some information from the Django Book on doing it by hand (see Making Changes to a Database Schema). This works for straightforward, simple changes.
Longer-term, you'll probably want to use a migration tool. There are three major options:
django-evolution
Dmigrations (written by Simon Willison, one of the creators of Django) (works only with MySQL)
South
EDIT: Looking through the question linked by TokenMacGuy, I'll add two more to the list for the sake of completeness:
Migratory
simplemigrations
Just to throw in an extra opinion - dmigrations is pretty nice and clear to use, but I'd say South is your best bet. Again, it's easy to get into, but it's more powerful and also has support for more database backends than just MySQL. It even handles MSSQL, if that's your thing
Related
I'm working on a project that I inherited, and I want to add a table to my database that is very similar to one that already exists. Basically, we have a table to log users for our website, and I want to create a second table to specifically log users that our site fails to do a task for.
Since I didn't write the site myself, and am pretty new to both SQL and Django, I'm a little paranoid about running a migration (we have a lot of really sensitive data that I'm paranoid about wiping).
Instead of having a django migration create the table itself, can I create the second table in MySQL, and the corresponding model in Django, and then have this model "recognize" the SQL table? without explicitly using a migration?
SHORT ANSWER: Yes.
MEDIUM ANSWER: Yes. But you will have to figure out how Django would have created the table, and do it by hand. That's not terribly hard.
Django may also spit out some warnings on startup about migrations being needed...but those are warnings, and if the app works, then you're OK.
LONG ANSWER: Yes. But for the sake of your sanity and sleep quality, get a completely separate development environment and test your backups. (But you knew that already.)
I have been playing arround with django for a couple of days and it seems great, but I find it a pain if I want to change the structure of my database, I then am stuck with a few rather awkward options.
Is there a way to completely bypass djangos database abstraction so if I change the structure of the database I dont have to guess what model would have generated it or use a tool (south or ...) to change things?
I essentially want this: https://docs.djangoproject.com/en/dev/topics/db/sql/ (Raw SQL Queries) but instead of refering to a model, refering to an external database.
Could I just create an empty model and then only perform raw queries on it? (and set up the DB externally)
Thanks
P.S. I dont really mind if I have separate databases for the admin stuff and the app data
It's in your question already, just read the docs article from here: Executing custom SQL directly
I put my first Django site online eight months ago. It was both a proof of concept as well as my first experience with Django. Fast forward eight months, I have validated my idea, but since it was a proof of concept and my first Django project, the code is pretty messy. Essentially, I am going to be re-writing the majority of the site, including re-engineering the models.
This is all fine and good. I have all my new models planned out. Essentially, I am going to create a new database to develop off of and let South manage any new database schema changes I make.
It is important to note two things:
I will not be creating a new project, just a new database.
This will be the first time I am incorporating South into the project and I would prefer to start with fresh models and a fresh database.
My question is, when I create the new database, will importing the contents of the old auth_* and django_* tables into the new auth_* and django_* tables create any problems? I have had some users register using the original proof of concept and I don't want to lose their information. I've never had to do this before so I'm not sure if there will be any repercussions.
If you use sql dump, such as
mysqldump -uusername -ppassword db_name table_name > xxxx.sql
mysql -uusername -ppassword new_db_name < xxxx.sql
The database's side is fine，if your backend is some other db，you can still find the similar commands.
For a new db, i think you need to export/import auth_user, i'm not quite sure if you need other contents in django_* tables. You can do this step by step, and see whether the new project works.
i'm working on a project (written in Django) which has only a few entities, but many rows for each entity.
In my application i have several static "reports", directly written in plain SQL. The users can also search the database via a generic filter form. Since the target audience is really tech-savvy and at some point the filter doesn't fit their needs, i think about creating a query language for my database like YQL or Jira's advanced search.
I found http://sourceforge.net/projects/littletable/ and http://www.quicksort.co.uk/DeeDoc.html, but it seems that they only operate on in-memory objects. Since the database can be too large for holding it in-memory, i would prefer that the query is translated in SQL (or better a Django query) before doing the actual work.
Are there any library or best practices on how to do this?
Writing such a DSL is actually surprisingly easy with PLY, and what ho—there's already an example available for doing just what you want, in Django. You see, Django has this fancy thing called a Q object which make the Django querying side of things fairly easy.
At DjangoCon EU 2012, Matthieu Amiguet gave a session entitled Implementing Domain-specific Languages in Django Applications in which he went through the process, right down to implementing such a DSL as you desire. His slides, which include all you need, are available on his website. The final code (linked to from the last slide, anyway) is available at http://www.matthieuamiguet.ch/media/misc/djangocon2012/resources/compiler.html.
Reinout van Rees also produced some good comments on that session. (He normally does!) These cover a little of the missing context.
You see in there something very similar to YQL and JQL in the examples given:
groups__name="XXX" AND NOT groups__name="YYY"
(modified > 1/4/2011 OR NOT state__name="OK") AND groups__name="XXX"
It can also be tweaked very easily; for example, you might want to use groups.name rather than groups__name (I would). This modification could be made fairly trivially (allow . in the FIELD token, by modifying t_FIELD, and then replacing . with __ before constructing the Q object in p_expression_ID).
So, that satisfies simple querying; it also gives you a good starting point should you wish to make a more complex DSL.
I've faced exactly this problem - a large database which needs searching. I made some static reports and several fancy filters using django (very easy with django) just like you have.
However the power users were clamouring for more. I decided that there already was a DSL that they all knew - SQL. The question was how to make it secure enough.
So I used django permissions to give the power users permission to make SQL queries in a new table. I then made a view for the not-quite-so-power users to use these queries. I made them take optional parameters. The queries were run using Python's lower level DB-API which django is using under the hood for its ORM anyway.
The real trick was opening a read only database connection to run these queries just to make sure that no updates were ever run. I made a read only connection by creating a different user in the database with lower permissions and opening a specific connection for that in the view.
TL;DR - SQL is the way to go!
Depending on the form of your data, the types of queries your users need to use, and the frequency that your data is updated, an alternative to the pure SQL solution suggested by Nick Craig-Wood is to index your data in Solr and then run queries against it.
Solr is an added layer of complexity (configuration, data synchronization) but it is super-fast, can handle large datasets, and provides a (relatively) intuitive query language.
You could write your own SQL-ish language using pyparsing, actually. There is even pretty verbose example you could extend.
So, I'm looking into the Django and Neo4j integration, but there's not much about it out there ... What I want to know is if I have a model like the one here:
If I want to add a new property to a model, it would be as simple as doing:
node.setProperty( "newProperty", "something" );
??
Moreover, all the queries in django would work ? How is the traversing made ?
I would appreciate any response
:D
Thanks.
We've been working on updating the Neo4j/Django integration to work with neo4j-rest-client - the fruits of our labor are on GitHub, with some quick comments on my blog.
There are some pros and cons to our integration. The most obviously impacting is our use of a REST client- you get to use a remote database, while losing quite a bit, performance-wise. OTOH, the integration works alongside a relational database, so you can still use django.contrib stuff that relies on the original ORM, and it handles indexing and query sets pretty nicely.
To do what you want above using neo4django, you'd simply get the neo4j-rest-client based node from the model instance, and have at it.
model_instance.node['newProperty'] = 'something'
We're still cranking on making the integration more dynamic, supporting traversals/etc in a Pythonic way, and (currently most important) improving the performance. If you're interested, I'd love feedback.
Have you seen Tobias's blog post about Django integration? It's kinda old now, but still relevant. Plus there are examples you can check out too.