Develop Reference

From python console app to django as UI best practice

I have an existing python application that runs in a console. This application contain features that is currently using sqlite3 as its data storage.
I am now trying to make use of Django as the presentation and it seems that I would need some help from you guys on how to speed up my transition from console to a web enabled user interface.
I understand that web is stateless and the app will be mostly consumed several people.
Most of my data is in dict which I can be able to convert to a JSON. I am seeking for kind advice on what are the things that I need to understand or if any of you guys have encountered this kind of app that you will need to migrate to a web centric presentation.
It has mostly query based functions which have few argumenta/parameters passed but it seems that I will have problem with the database access in python due to that most of my db queries are raw sql.
Thank you guys in advance.
(if there is any existing projects that I can use as a pattern by all means please do help to share with me)
==========
Resolution
Added this into manage.py:
sys.path.insert(0, os.path.join(BASE_DIR, 'app', 'engine'))
Now I can call my 'engine' package classes/scripts.

This is a pretty open-ended question, but I'll address moving your database to Django's models and custom queries.
Data Layer
Before building your app, you can try and use the manage.py command inspectdb to describe your current schema as Django models. It won't write your model layer for you, but can help you get started.
As #joran-beasley mentioned, sqlite will not be suitable as a production database for a multi-client application. Postgres is widely-used in the Django community, but it is hardly your option that supports raw SQL.
It is possible to migrate some data in XML, YAML, or JSON and import them with django's de/serializers or via the loaddata manage.py command. Consider adding custom managers to your models that are the targets of 1-to-1, Foreign Key, or Many-to-Many fields. You can add a custom get_by_natural_key method that can simplify de/serializing data that contains foreign keys, etc.
Queries
Django's QuerySet api is a wonderful wrapper for wrap SQL, but it will have some limitations. Take some time to review it's capabilities and shortcomings to decide if it can help or hinder your needs.
If you want to continue making raw queries, Django can help you with that too as it offers helper queryset methods like raw or extra. You can avoid the model layer entirely using its django.db.connection.cursor class.
I can't speak to much else that will help you in your transition, but you may find that Django's class-based generic views like DetailView, ListView, UpdateView, or DeleteView may be helpful in setting up some of your more basic interfaces on the web.

Storing Configuration details

I have a bunch of string and integer constants that I use at various places in my app.
I am planning to put them up in a centralized location, so that it is easier to change them in the future. I could think of the following approaches:
1) Have them as individual variables, stored in the model db.py
settings_title = "My Amazing App"
settings_ver = 2.0
settings_desc = "Moar and Moar cats"
2) Have them as a dict, stored in db.py
settings = { "title": "My Amazing App",
"ver" = 2.0,
"desc" = "Moar and Moar cats"
}
Is it a good idea to use the model db.py? I've heard that it is evaluated for every request. Could putting settings there have a noticeable overhead?
Is there any difference, performance-wise between the two approaches?
Is there a better way of doing this?

You can directly put your config variable in .py file and use that file by import in your module as django used setting.py. If you want to combine the variable on some section bases then you can use ConfigParser which can ready the .cfg file.

Your db.py model file will be executed on every request in any case, so adding a few setting assignments to the code will add negligible overhead. Instead of putting the settings in db.py, for better organization you might consider creating a separate model file. Note, model files are executed in alphabetical order, so if the settings have to be available in subsequent model files, name the settings file something like 0_settings.py to ensure it is executed before any other model files.
If you prefer, you can instead put the settings in a module (e.g., settings.py) in the application's /modules folder and import the settings object in your application code wherever you need it (in that case, the module will only be loaded once by the interpreter). If any of the settings need to be set dynamically based on the incoming request, though, you are probably better off keeping the settings in a model file.
Finally, rather than a standard dictionary, you might consider using a web2py Storage object, which is like a dictionary but allows you to access values as attributes and returns None rather than a KeyError if you try to access a key/attribute that doesn't exist:
from gluon.storage import Storage
settings = Storage()
settings.title = 'My Amazing App'
or
settings = Storage({'title': 'My Amazing App'})
Note, the web2py request, response, and session objects are all instances of the Storage class.

Django, for instance, uses a file settings.py.
It's not a model, but just a collection of variables of all types, strings/ints/dicts/whatever, and you import settings or from settings import * in every module that needs access to them.
Since it is not a single model, there's no overhead on access.

Django : is it better to import variables from a settings.py file, or basic configuration file?

I was wondering about whether it is better to import variables into your view from the settings.py file? or better to create a configuration file with the variables that are needed?
I tend to like to write configuration files for my Django applications, read, and import the variables from there when necessary. For example:
.configrc
[awesomeVariables]
someMagicNumber = 7
views.py
from ConfigParser import SafeConfigParser
#Open and parse the file
config = SafeConfigParser()
config.read(pathToConfig)
#Get variable
magicNumber = config.get('awesomeVariables', 'someMagicNumber'))
However, I've noticed some programmers prefer the following :
settings.py
SOME_MAGIC_NUMBER=7
views.py
import settings
magicNumber = settings.SOME_MAGIC_NUMBER
I was curious as to the pros and cons of the different methods? Can importing variables directly from your settings compromise the integrity of the architecture?

It's a judgement call. Using the settings module is the "Django way", although you should do from django.conf import settings; settings.MY_SETTING, which will respect DJANGO_SETTINGS_MODULE. But Django is just Python; there's nothing that will stop you from using ConfigParser. In the interest of having only one place where such things are defined, I'd recommend putting it in the Django settings file - but if you have a reason not to, don't.

Using a config file is completely un-Django. Settings go in settings.py. Period. For app-specific settings, you simply set a default in your app and allow the user to override in their project's settings.py:
from django.conf import settings
SOME_MAGIC_NUMBER = settings.SOME_MAGIC_NUMBER if hasattr(settings, 'SOME_MAGIC_NUMBER') else 0
# Where `0` is the default value

There also an app for storing settings dynamically in db. Maybe you find it usefull .
django-constance

How to work with settings in Django

I want to keep some global settings for my project in Django. I need to have access to these settings from the code. For example, I need to set a current theme for my site that I can set using admin console or from the code. Or I need to set a tagline that will show in the header of all pages. I suppose I should use models for keeping the settings but I can't realize how I should better do it.

There are quite some packages that store settings in models, pick the one that works best for you:
http://pypi.python.org/pypi?:action=search&term=django+settings&submit=search

If you are okay with changing these setting programmatically via settings.py you should do that. However, if you want to change these settings via the Admin Console, you should use models.

Use only some parts of Django?

I like Django, but for a particular application I would like to use only parts of it, but I'm not familiar enough with how Django works on the inside, so maybe someone can point me into the right direction as to what I have to check out.
Specifically, I want to use:
The models and database abstraction
The caching API, although I want to avoid database lookups by caching, not HTML generation, and since the caching framework in Django is intended for the latter, I'm not sure yet whether that's really appropriate.
I would not use:
Templating
urlconfigs
Or, more exactly, I'm neither using HTTP nor HTML. So basically, I have a different input / output chain than usual.
Can this work?
My personal killer feature in Django is the Object / database mapping that I can do with the models, so if there's another technology (doesn't have to be Python, I'm in the design phase and I'm pretty agnostic about languages and platforms) that gives me the same abilities, that would be great, too.

I myself use Django for its object/db mapping without using its urlconfigs. Simply create a file called djangosettings.py and insert the necessary configuration, for example:
DATABASE_ENGINE = 'oracle'
DATABASE_HOST = 'localhost'
DATABASE_NAME = 'ORCL'
DATABASE_USER = 'scott'
DATABASE_PASSWORD = 'tiger'
Then in your regular Python code, do
import os
os.environ["DJANGO_SETTINGS_MODULE"] = "djangosettings"
before you import any Django modules. This will let you use Django's object/db mappings without actually having a Django project, so you can use it for standalone scripts or other web applications or whatever you want.
As for caching, if you don't want to use Django then you should probably decide what you are using and go from there. I recommend using CherryPy, which doesn't use Django-style regular expression URL mapping, but instead automatically maps URLs to functions based on the function names. There's an example right at the top of the CherryPy home page: http://cherrypy.org/
CherryPy has its own caching system, so you can accomplish exactly the same thing as what Django does but without needing to use Django's urlconfig system.

Django, being a web framework, is extremely efficient at creating websites. However, it's also equally well-suited to tackling problems off the web. This is the loose coupling that the project prides itself on. Nothing stops you from installing a complete version of Django, and just using what you need. As a rule, very few components of Django make broad assumptions about their usage.
Specifically:
Django models don't know anything
about HTML or HTTP.
Templates don't know anything about HTML or HTTP.
The cache
system can be used to store
anything that can be pickled.
One of the main things you'll face when trying to use Django without a web server is setting up the environment properly. The ORM and cache system still need to be configured in settings.py. There are docs on using django without a settings module that you may find useful.

Django ORM Standalone
I've created a template Django project that allows you to do just that.
https://github.com/dancaron/Django-ORM
Just follow the instructions and you can write standalone python files that utilize Django's database functionality, without having to use urlconf, views, etc.

I tend to prefer a mix-and-match approach to using Python for web programming. :-)
I don't have a lot of experience with Django, but I'd recommend giving sqlalchemy a look for the database stuff. It works well with others and gives you several potential layers of abstraction (so you can go with something basic or tweak the hell out of it if you want). Plus, you'll already be somewhat familiar with it if you've ever used hibernate/nhibernate.
My favorite part is that it has a lot of options for databases to connect to (most notably SQL Server, which django doesn't have built in last time I checked).
With that said, I'm told that with Django, it's pretty easy to decouple functionality (but never done so myself).

There are of course other projects out there that specifically implement single parts of django. TurboGears for example is a collection of several projects that can work by themselves and together form a complete web development framework.
For the db abstraction SQLAlchemy comes to mind.
Regarding the caching part: I'm not aware of any standalone project that implements a generic caching facility.
On the other hand, it should be fairly easy to implement your own caching, for example by using pickles. Have a look at this recipe for a decorator for ideas and google for "memoize".
Also keep in mind that your database has its own caching mechanism, so maybe you don't even need to concern yourself with the details.

I've shared an example of solution, which prevents Python Path manipulation inside code:
https://github.com/askalyuk/django-orm-standalone
It contains a standalone data access package, a separated simple Django site and a unit test.

I found KeyboardInterrupt's answer but it was answered in 2009 and I failed to run it in Django 1.8.For recent Django 1.8, You can have a look at this, in which some parts come from KeyboardInterrupt's answer.
The folder structure is:
.
├── myApp
│   ├── __init__.py
│   └── models.py
└── my_manage.py
myApp is a module, contains an empty __init__.py and models.py.
There is an example model class in models.py:
from django.db import models
class MyModel(models.Model):
field = models.CharField(max_length=255)
my_manage.py contains django database, installed_app settings and acts as django offical manage.py, so you can:
python my_manage.py sql myApp
python my_manage.py migrate
......
The codes in my_manage.py are:
from django.conf import settings
db_conf = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'your_database_name',
'USER': 'your_user_name',
'PASSWORD': 'your_password',
'HOST': 'your_mysql_server_host',
'PORT': 'your_mysql_server_port',
}
}
settings.configure(
DATABASES = db_conf,
INSTALLED_APPS = ( "myApp", )
)
# Calling django.setup() is required for “standalone” Django u usage
# https://docs.djangoproject.com/en/1.8/topics/settings/#calling-django-setup-is-required-for-standalone-django-usage
import django
django.setup()
if __name__ == '__main__':
import sys
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)