I have an existing python application that runs in a console. This application contain features that is currently using sqlite3 as its data storage.
I am now trying to make use of Django as the presentation and it seems that I would need some help from you guys on how to speed up my transition from console to a web enabled user interface.
I understand that web is stateless and the app will be mostly consumed several people.
Most of my data is in dict which I can be able to convert to a JSON. I am seeking for kind advice on what are the things that I need to understand or if any of you guys have encountered this kind of app that you will need to migrate to a web centric presentation.
It has mostly query based functions which have few argumenta/parameters passed but it seems that I will have problem with the database access in python due to that most of my db queries are raw sql.
Thank you guys in advance.
(if there is any existing projects that I can use as a pattern by all means please do help to share with me)
==========
Resolution
Added this into manage.py:
sys.path.insert(0, os.path.join(BASE_DIR, 'app', 'engine'))
Now I can call my 'engine' package classes/scripts.
This is a pretty open-ended question, but I'll address moving your database to Django's models and custom queries.
Data Layer
Before building your app, you can try and use the manage.py command inspectdb to describe your current schema as Django models. It won't write your model layer for you, but can help you get started.
As #joran-beasley mentioned, sqlite will not be suitable as a production database for a multi-client application. Postgres is widely-used in the Django community, but it is hardly your option that supports raw SQL.
It is possible to migrate some data in XML, YAML, or JSON and import them with django's de/serializers or via the loaddata manage.py command. Consider adding custom managers to your models that are the targets of 1-to-1, Foreign Key, or Many-to-Many fields. You can add a custom get_by_natural_key method that can simplify de/serializing data that contains foreign keys, etc.
Queries
Django's QuerySet api is a wonderful wrapper for wrap SQL, but it will have some limitations. Take some time to review it's capabilities and shortcomings to decide if it can help or hinder your needs.
If you want to continue making raw queries, Django can help you with that too as it offers helper queryset methods like raw or extra. You can avoid the model layer entirely using its django.db.connection.cursor class.
I can't speak to much else that will help you in your transition, but you may find that Django's class-based generic views like DetailView, ListView, UpdateView, or DeleteView may be helpful in setting up some of your more basic interfaces on the web.
Related
Every Django tutorial/book I've seen approaches Django projects from what I am going to characterize as an ad-hoc database design method. A project ends-up being a bunch of little apps, each with it's own models, view, etc.
I am trying to locate a resource that covers how to structure a Django project when you do, in fact, start with a traditional DB design process.
For example, let's say that this is the starting point:
And a document describing it:
LEAD Database Guide
How does one approach this so that the various Django apps access a single "global" (bad word) db model that covers the entire schema rather than a bunch of models spread across a bunch of apps?
Are there any resources (books, pdf's, tutorials) that cover this approach rather than the piece-meal approach most commonly seen?
A corollary to this question might be: Is there a, perhaps automated, way to go from an SQL (or MySQL Workbench) schema definition to the equivalent Django ORM?
You can use the inspectdb command to generate models.py from database:
http://docs.djangoproject.com/en/dev/ref/django-admin/#inspectdb
Then, you can treat the models as a single app, and add more features/tables into new apps in the future.
I have a few databases in MongoDB that I want to create models for dynamically, since there are many databases and I cannot do it manually. Questions:
What should my models.py look like? (Does inspectdb work with mongodb databases or only SQL based dbs?)
Since the database models are created dynamically, how do I code the serializer class to return the dynamic fields?
Thanks in advance.
Django supports an object-relational mapper, that is aimed at traditional relational databases. While there are a number of mongodb packages for Django, none of them support inspectdb to construct your models. Either way, inspectdb is a kludge designed as a one of process to help a one-of migratation away from a legacy system, i.e. you'd build your models.py file once and never run inspectdb again. This is not what you want to do, as you seem to want dynamic models that can be added or altered at runtime.
On the bright side, Django MongoDB Engine has some support for arbitrary embedded models within pre-defined models. But even then they don't seem too supportive of it:
As you can see, generic embedded models add a lot of overhead that bloats up your data records. If you want to use them anyway, here’s how you’d do it...
In summary, try to build your models as best you can to actually match your requirements. If you know nothing about your models ahead of production, then perhaps Django isn't the right solution for you.
Is it fine not to define table stucture in models.py...I mean we are using
cursor.execute('some query')
so its okay not to make something like this :
def tblNew(models.Model)
col1 = models.CharField(max_length=2)
Thanks for guidance, Im a newbie in django.
You CAN create database tables directly in postgresql But you'd have to create models.py files later because otherwise you will be missing out on several things. If you have pre-existing database, then you have django-admin.py command to pick them up for you (https://docs.djangoproject.com/en/dev/ref/django-admin/#inspectdb)
ORM. It makes alot of things easyer. Writing sql in views? Why would that be good? Because that is what you get if you do not use ORM at all. And while i do have sql in some of my views, it only exists there because django lacks some features like language collation specific sorting. But otherwise writing sql in views hampers readability.
You can't use database migrations. South (http://south.aeracode.org/) is one awesome project that most big django projects use. In fact i think it was most used django project in some apps poll. Django 1.7 will provide the migrations too, but you will need to declare models im models.py for that. What you loose if you do not use database migrations - SANITY. I do not exaggerate here. Keeping track of database changes in development and prelive/live servers is nightmare otherwise.
Also is not using django ORM in some places (not declaring models) kind of strange, while you will definately use django models and ORM elsewhere - or will you stop using django authentication, sessions and site logic also? All those use django ORM...
I've written some python code to accomplish a task. Currently, there are 4-5 classes that I'm storing in separate files. I'd now like to change this whole thing into a database-backed web app. I've been reading tutorials on Django, and so far I get the impression that I'll need to manually specify the fields and their types for every "model" that I use. This is a little surprising to me, since I was expecting some kind of ORM capability that would just take the existing classes I've already defined, and map them onto a database somehow, in a manner abstracted away from me.
Is this not the case? Am I missing something? It looks like I need to specify all the fields and types in the file 'models.py'.
Okay, now beyond those specifics, does anyone have any general tips on the best way to migrate an object-oriented desktop application to a web application?
Thanks!
That is Django's ORM: it maps classes to tables. What else did you expect? There needs to be some way of specifying what the fields are, though, before you can use them, and that's managed through the models.Model class and the various models.Field subclasses. You can certainly use your classes as mixins in order to use the existing business logic on top of the field definitions.
If you are thinking about a database backend based web app, you have to specify what fields of the data you want to store and what type of the value you want stored.
There is an abstraction that introspects the db to convert it into the django models.py format. But I know not of any that introspects a python class and stores arbitrary data into db. How would that even work? Are the objects, now, stored as a pickle?
You're going to have to check the output, but you can have Django automatically create models from existing databases through one-time introspection.
Taken from the link below, you would set up your database in settings.py, and then call
python manage.py inspectdb
This will dump the sample models.py file to standard out for your inspection. In order to create the file, simply redirect the output
python manage.py inspectdb > models.py
See for more:
http://docs.djangoproject.com/en/dev/howto/legacy-databases/?from=olddocs#auto-generate-the-models
Pretty new to this scene and trying to find some documentation to adopt best practices. We're building a fairly large content site which will consist of various media catalogs and I'm trying to find some comparable data / architectural models so that we can get a better idea of the approach we should use using a framework we've never made use of before. Any insight / help would be greatly appreciated!
"data / architectural models so that we can get a better idea of the approach we should use using a framework we've never made use of before"
Django imposes best practices on you. You don't have a lot of choices and can't make a lot of mistakes.
MVC (while a noble aspiration) is implemented as follows:
Data is defined in "models.py" files using the Django ORM models.
urls.py file maps URL to view function. Pick your URL's wisely.
View function does all processing, making use of models and methods in models
Presentation (via HTML templates) invoked by View function. Essentially no processing can be done in presentation, just lightweight iteration and decision-making
The model is defined for you. Just stick to what Django does naturally and you'll be happy.
Architecturally, you usually have a stack like this.
Apache does two things.
serves static content directly and immediately
hands dynamic URL to Django (via mod_python, mod_wsgi or mod_fastcgi). Django apps map URL to view functions (which access to database (via ORM/model) and display via templates.
Database used by Django view functions.
The architecture is well-defined for you. Just stick to what Django does naturally and you'll be happy.
Feel free to read the Django documentation. It's excellent; perhaps the best there is.
first, forget all MVC mantra. it's important to have a good layered structure, but MVC (as defined originally) isn't one, it was a modular structure, where each GUI module is split in these tree submodules. nothing to use on the web here.
in web development, it really pays to have a layered structure, where the most important layer is the storage/modelling one, which came to be called model layer. on top of that, you need a few other layers but they're really not anything like views and controllers in the GUI world.
the Django layers are roughly:
storage/modelling: models.py, obviously. try to put most of the 'working' concepts there. all the relationships, all the operations should be implemented here.
dispatching: mostly in urls.py. here you turn your URL scheme into code paths. think of it like a big switch() statement. try hard to have readable URLs, which map into user intentions. it will help a lot to add new functionality, or new ways to do the same things (like an AJAX UI later).
gathering: mostly the view functions, both yours and the prebuilt generic views. here you simply gather all the from the models to satisfy a user request. in surprisingly many cases, it just have to pick a single model instance, and everything else can be retrieved from relationships. for these URLs, a generic view is enough.
presentation: the templates. if the view gives you the data you need, it's simple enough to turn it into a webpage. it's here where you'll thank that the model classes have good accessors to get any kind of relevant data from any given instance.
To understand django fundementals and the django take on MVC, consult the following:
http://www.djangobook.com/
As a starting point to getting your hands dirty with ...
"...trying to find some comparable data / architectural models"
Here is a quick and dirty way to reverse engineer a database to get a models.py file,
which you can then inspect to see how django would handle it.
1.) get an er diagram that closely matches your target. For example something like this
http://www.databaseanswers.org/data_models/product_catalogs/index.htm
2.) create an sql script from the er diagram and create the database,
I suggest Postgre, as some MySQL
table type will not have forgien key constraints, but in a pinch MySQL or SQLITE
will do
3.) create and configure a django app to use that database. Then run:
python manage.py inspectdb
This will at least give you a models.py file which you can read to see how django attempts
to model it.
Note that the inspect command is intended to be a shortcut for dealing with legacy
database when developing in django, and as such is not perfect. Be sure to read the
following before attempting this:
http://docs.djangoproject.com/en/dev/ref/django-admin/#ref-django-admin