I stating read Storm ORM docs, and try some examples with sqlite. I have one question, can Storm automaticaly create tables from models or no? I don't want to do this:
store.execute("CREATE TABLE person "
"(id INTEGER PRIMARY KEY, name VARCHAR)")
every time when I want to create new table, also this is not good when table alredy exists.
Storm ORM haven't feature for autocreating tables. I start use peewee ORM it looks very nice.
If you're still starting the project and haven't put too much work into it yet, let me kindly suggest, that you try an object oriented database directly, instead of emulating an object oriented database with some relational backend. ZODB is a very good match for that, but you should also have a look at MongoDB and colleaques. I had a try with Storm a while ago, and dropt it quite soon again, throwing a lot of code away, because of the horribly slow performance, especially with insert-or-update statements. You don't have to make the same mistake.
More on-topic: As far as I know, there is no such feature. I was also looking for it, and was somewhat disappointed, after setting up a detailed data model, that it couldn't generate the tables automatically. Beat me, if I missed it.
Related
Here is my situation. I used Python, Django and MySQL for a web development.
I have several tables for form posting, whose fields may change dynamically. Here is an example.
Like a table called Article, it has three fields now, called id INT, title VARCHAR(50), author VARCHAR(20), and it should be able store some other values dynamically in the future, like source VARCHAR(100) or something else.
How can I implement this gracefully? Is MySQL be able to handle it? Anyway, I don't want to give up MySQL totally, for that I'm not really familiar with NoSQL databases, and it may be risky to change technique plan in the process of development.
Any ideas welcome. Thanks in advance!
You might be interested in this post about FriendFeed's schemaless SQL approach.
Loosely:
Store documents in JSON, extracting the ID as a column but no other columns
Create new tables for any indexes you require
Populate the indexes via code
There are several drawbacks to this approach, such as indexes not necessarily reflecting the actual data. You'll also need to hack up django's ORM pretty heavily. Depending on your requirements you might be able to keep some of your fields as pure DB columns and store others as JSON?
I've never actually used it, but django-not-eav looks like the tool for the job.
"This app attempts the impossible: implement a bad idea the right way." I already love it :)
That said, this question sounds like a "rethink your approach" situation, for sure. But yes, sometimes that is simply not an option...
I am looking for a generic way to store python objects in a database. Of course I could just pickle the objects, but that way I would have binary blobs in my database. That way I can not search my objects. Also it seems to be easier to put it together with other applications.
So in my fantasy, I have on object like
class myClass
data1=1
data2='foobar'
data3=some_html_object
...
and could do something like
mydata=myClass()
mydata.add_data(various_things)
mydata.save_to_database()
and would end up with a database which has colums called data1,data2, data3, where I have the values of the of the objects attributes in the rows stored as text which would be searchable. Of course some inital setup would have to be done.
And of course it would be nice if I could plug any database I want (well, at least not just one database) and would not be bothered with the details.
Now of course I could program my own framework to let me do this, but I was hoping that this has been done bevore by someone else :)
Any suggestions?
Your fantasy in fact exists!
You describe something called the Active Record Pattern. It is usually implemented by using Object-Relational Mapping. One common solution for Python is SQLAlchemy, but Storm is somehow popular too:
See What are some good Python ORM solutions?
If your are developing for the Web, Django possess its own ORM.
It sounds like what you want is an Object Relational Mapper (ORM) to map SQL tables to objects.
The most popular ORMs that support different dialects by community are the following:
Python -- SQLAlchemy, Storm, Django (built into web framework)
Ruby -- ActiveRecord, Sequel
Node -- Sequelize
For a specific example of implementing what you described in Python using SQLAlchemy, check out this blog post that walks through a simple example
Rather than use an ORM, I am considering the following approach in Python and MySQL with no ORM (SQLObject/SQLAlchemy). I would like to get some feedback on whether this seems likely to have any negative long-term consequences since in the short-term view it seems fine from what I can tell.
Rather than translate a row from the database into an object:
each table is represented by a class
a row is retrieved as a dict
an object representing a cursor provides access to a table like so:
cursor.mytable.get_by_ids(low, high)
removing means setting the time_of_removal to the current time
So essentially this does away with the need for an ORM since each table has a class to represent it and within that class, a separate dict represents each row.
Type mapping is trivial because each dict (row) being a first class object in python/blub allows you to know the class of the object and, besides, the low-level database library in Python handles the conversion of types at the field level into their appropriate application-level types.
If you see any potential problems with going down this road, please let me know. Thanks.
That doesn't do away with the need for an ORM. That is an ORM. In which case, why reinvent the wheel?
Is there a compelling reason you're trying to avoid using an established ORM?
You will still be using SQLAlchemy. ResultProxy is actually a dictionary once you go for .fetchmany() or similar.
Use SQLAlchemy as a tool that makes managing connections easier, as well as executing statements. Documentation is pretty much separated in sections, so you will be reading just the part that you need.
web.py has in a decent db abstraction too (not an ORM).
Queries are written in SQL (not specific to any rdbms), but your code remains compatible with any of the supported dbs (sqlite, mysql, postresql, and others).
from http://webpy.org/cookbook/select:
myvar = dict(name="Bob")
results = db.select('mytable', myvar, where="name = $name")
If I change a field in a Django model, how can I synchronize it with the database tables? Do I need to do it manually on the database or is there a tool that does helps with the process?
Alas, Django does not support any easy solution to this.
The only thing django will do for you, is restart your database with new tables that match your new models:
$ #DON'T DO THIS UNLESS YOU CAN AFFORD TO LOSE ALL YOUR DATA!
$ python PROJECT_DIR/manage.py syncdb
the next option is to use the various sql* options to manage.py to see what django would do to match the current models to the database, then issue your own ALTER TABLE commands to make everything work right. Of course this is error prone and difficult.
The real solution is to use a database migration tool, such as south to generate migration code.
Here is a similar question with discussion about various database migration options for django.
Can't seem to be able to add a comment to the marked answer, probably because I haven't got enough rep (be nice if SO told me so though).
Anyway, just wanted to add that in the answered post, I believe it is wrong about syncdb - syncdb does not touch tables once they have been created and have data in them. You should not lose data by calling it (otherwise, how could you add new tables for new apps?)
I believe the poster was referring to the reset command instead, which does result in data loss - it will drop the table and recreate it and hence it'll have all the latest model changes.
Django Evolution can help, but the best option really is to plan out your schema in advance, or to make simple modifications manually. Or, to be willing to toast your test data by dropping tables and re-syncing.
Django does not provide for this out of the box.
Here's some information from the Django Book on doing it by hand (see Making Changes to a Database Schema). This works for straightforward, simple changes.
Longer-term, you'll probably want to use a migration tool. There are three major options:
django-evolution
Dmigrations (written by Simon Willison, one of the creators of Django) (works only with MySQL)
South
EDIT: Looking through the question linked by TokenMacGuy, I'll add two more to the list for the sake of completeness:
Migratory
simplemigrations
Just to throw in an extra opinion - dmigrations is pretty nice and clear to use, but I'd say South is your best bet. Again, it's easy to get into, but it's more powerful and also has support for more database backends than just MySQL. It even handles MSSQL, if that's your thing
I'm starting a web project that likely should be fine with SQLite. I have SQLObject on top of it, but thinking long term here -- if this project should require a more robust (e.g. able to handle high traffic), I will need to have a transition plan ready. My questions:
How easy is it to transition from one DB (SQLite) to another (MySQL or Firebird or PostGre) under SQLObject?
Does SQLObject provide any tools to make such a transition easier? Is it simply take the objects I've defined and call createTable?
What about having multiple SQLite databases instead? E.g. one per visitor group? Does SQLObject provide a mechanism for handling this scenario and if so, what is the mechanism to use?
Thanks,
Sean
3) Is quite an interesting question. In general, SQLite is pretty useless for web-based stuff. It scales fairly well for size, but scales terribly for concurrency, and so if you are planning to hit it with a few requests at the same time, you will be in trouble.
Now your idea in part 3) of the question is to use multiple SQLite databases (eg one per user group, or even one per user). Unfortunately, SQLite will give you no help in this department. But it is possible. The one project I know that has done this before is Divmod's Axiom. So I would certainly check that out.
Of course, it would probably be much easier to just use a good concurrent DB like the ones you mention (Firebird, PG, etc).
For completeness:
1 and 2) It should be straightforward without you actually writing much code. I find SQLObject a bit restrictive in this department, and would strongly recommend SQLAlchemy instead. This is far more flexible, and if I was starting a new project today, I would certainly use it over SQLObject. It won't be moving "Objects" anywhere. There is no magic involved here, it will be transferring rows in tables in a database. Which as mentioned you could do by hand, but this might save you some time.
Your success with createTable() will depend on your existing underlying table schema / data types. In other words, how well SQLite maps to the database you choose and how SQLObject decides to use your data types.
The safest option may be to create the new database by hand. Then you'll have to deal with data migration, which may be as easy as instantiating two SQLObject database connections over the same table definitions.
Why not just start with the more full-featured database?
I'm not sure I understand the question.
The SQLObject documentation lists six kinds of connections available. Further, the database connection (or scheme) is specified in a connection string. Changing database connections from SQLite to MySQL is trivial. Just change the connection string.
The documentation lists the different kinds of schemes that are supported.