I like Django, but for a particular application I would like to use only parts of it, but I'm not familiar enough with how Django works on the inside, so maybe someone can point me into the right direction as to what I have to check out.
Specifically, I want to use:
The models and database abstraction
The caching API, although I want to avoid database lookups by caching, not HTML generation, and since the caching framework in Django is intended for the latter, I'm not sure yet whether that's really appropriate.
I would not use:
Templating
urlconfigs
Or, more exactly, I'm neither using HTTP nor HTML. So basically, I have a different input / output chain than usual.
Can this work?
My personal killer feature in Django is the Object / database mapping that I can do with the models, so if there's another technology (doesn't have to be Python, I'm in the design phase and I'm pretty agnostic about languages and platforms) that gives me the same abilities, that would be great, too.
I myself use Django for its object/db mapping without using its urlconfigs. Simply create a file called djangosettings.py and insert the necessary configuration, for example:
DATABASE_ENGINE = 'oracle'
DATABASE_HOST = 'localhost'
DATABASE_NAME = 'ORCL'
DATABASE_USER = 'scott'
DATABASE_PASSWORD = 'tiger'
Then in your regular Python code, do
import os
os.environ["DJANGO_SETTINGS_MODULE"] = "djangosettings"
before you import any Django modules. This will let you use Django's object/db mappings without actually having a Django project, so you can use it for standalone scripts or other web applications or whatever you want.
As for caching, if you don't want to use Django then you should probably decide what you are using and go from there. I recommend using CherryPy, which doesn't use Django-style regular expression URL mapping, but instead automatically maps URLs to functions based on the function names. There's an example right at the top of the CherryPy home page: http://cherrypy.org/
CherryPy has its own caching system, so you can accomplish exactly the same thing as what Django does but without needing to use Django's urlconfig system.
Django, being a web framework, is extremely efficient at creating websites. However, it's also equally well-suited to tackling problems off the web. This is the loose coupling that the project prides itself on. Nothing stops you from installing a complete version of Django, and just using what you need. As a rule, very few components of Django make broad assumptions about their usage.
Specifically:
Django models don't know anything
about HTML or HTTP.
Templates don't know anything about HTML or HTTP.
The cache
system can be used to store
anything that can be pickled.
One of the main things you'll face when trying to use Django without a web server is setting up the environment properly. The ORM and cache system still need to be configured in settings.py. There are docs on using django without a settings module that you may find useful.
Django ORM Standalone
I've created a template Django project that allows you to do just that.
https://github.com/dancaron/Django-ORM
Just follow the instructions and you can write standalone python files that utilize Django's database functionality, without having to use urlconf, views, etc.
I tend to prefer a mix-and-match approach to using Python for web programming. :-)
I don't have a lot of experience with Django, but I'd recommend giving sqlalchemy a look for the database stuff. It works well with others and gives you several potential layers of abstraction (so you can go with something basic or tweak the hell out of it if you want). Plus, you'll already be somewhat familiar with it if you've ever used hibernate/nhibernate.
My favorite part is that it has a lot of options for databases to connect to (most notably SQL Server, which django doesn't have built in last time I checked).
With that said, I'm told that with Django, it's pretty easy to decouple functionality (but never done so myself).
There are of course other projects out there that specifically implement single parts of django. TurboGears for example is a collection of several projects that can work by themselves and together form a complete web development framework.
For the db abstraction SQLAlchemy comes to mind.
Regarding the caching part: I'm not aware of any standalone project that implements a generic caching facility.
On the other hand, it should be fairly easy to implement your own caching, for example by using pickles. Have a look at this recipe for a decorator for ideas and google for "memoize".
Also keep in mind that your database has its own caching mechanism, so maybe you don't even need to concern yourself with the details.
I've shared an example of solution, which prevents Python Path manipulation inside code:
https://github.com/askalyuk/django-orm-standalone
It contains a standalone data access package, a separated simple Django site and a unit test.
I found KeyboardInterrupt's answer but it was answered in 2009 and I failed to run it in Django 1.8.For recent Django 1.8, You can have a look at this, in which some parts come from KeyboardInterrupt's answer.
The folder structure is:
.
├── myApp
│ ├── __init__.py
│ └── models.py
└── my_manage.py
myApp is a module, contains an empty __init__.py and models.py.
There is an example model class in models.py:
from django.db import models
class MyModel(models.Model):
field = models.CharField(max_length=255)
my_manage.py contains django database, installed_app settings and acts as django offical manage.py, so you can:
python my_manage.py sql myApp
python my_manage.py migrate
......
The codes in my_manage.py are:
from django.conf import settings
db_conf = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'your_database_name',
'USER': 'your_user_name',
'PASSWORD': 'your_password',
'HOST': 'your_mysql_server_host',
'PORT': 'your_mysql_server_port',
}
}
settings.configure(
DATABASES = db_conf,
INSTALLED_APPS = ( "myApp", )
)
# Calling django.setup() is required for “standalone” Django u usage
# https://docs.djangoproject.com/en/1.8/topics/settings/#calling-django-setup-is-required-for-standalone-django-usage
import django
django.setup()
if __name__ == '__main__':
import sys
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
Related
I'd like to expose some (app-specific) settings to the admin interface, so users can change them comfortably and also not have to restart Django.
How should I go about this?
I checked out the applications on http://djangopackages.com/grids/g/live-setting/ (btw django-constance was the most appealing) but really what all these apps are doing is storing values in a database, providing a web interface to change them, and caching. Aren't the first two features already built into Django?
The biggest drawbacks I see are that none of the apps are drop-in replacements for the old location of these settings (settings.py), and require me to migrate to their notation, and often add another context processor to access them in templates.
Couldn't I just do this?
Create a model for my settings (this gives me the various types and validation)
Instantiate one such object to hold my settings (this allows the users to edit them in the admin interface) - I could dump defaults as fixtures like for other models
Wrap settings.py so it makes a database query for my settings - http://www.loose-bits.com/2011/04/extending-django-settings-with-derived.html
From my current, naive point of view the only drawbacks I see would be:
Adding or changing the available settings requires a schema migration (south). - I can live with that.
I have a model with possibly multiple instances but really only need a singleton. - That could actually be a useful feature at some point.
Performance/Caching: Looking at http://code.djangoproject.com/svn/django/trunk/django/conf/ I'd have to put a little bit of cleverness into the settings wrapper and/or model, so that model changes clear or update cached values. - doesn't seem to be rocket science.
Doing the same in another project would require a similar effort again. - I think a single dictionary constant in settings.py, holding model name(s) and field names for the lookups is all that would differ.
Wouldn't this be the best of both worlds - runtime admin (with all its perks), database backend, caching, and none of my settings.USED_TO_BE_IN_SETTINGS_DOT_PY would need any changing. Am I missing something?
AFAIK, the Django settings are supposed to be immutable. There are multiple reasons for this, the most obvious being that Django is not aware of the server's execution model (prefork / multi-threaded).
Also, you can't load the settings themselves from a Django model because the settings need to be loaded before you can use anything in the ORM.
So, basically, you have two solutions:
you can bootstrap the settings from the database by using any lower-level database access mechanism to load them; or
you can just define your settings in some other model and fetch them directly when you need them.
The first is an incredible hack and I don't suggest it. The second is much more direct and cleaner, but requires you to change your usual habits (from django.conf import settings).
The second approach is probably what's implemented by the 3rd-party apps you linked to.
From Django 1.8 docs:
You shouldn’t alter settings in your applications at runtime.
DATABASES is a dict. So you can manipulate how a dictionary:
import django.conf as conf
conf.settings.DATABASES['default']['NAME'] = 'novo_banco'
Take a look: https://bitbucket.org/bkroeze/django-livesettings
*Django-Livesettings is a project split from the Satchmo Project_. It provides the ability to configure settings via an admin interface, rather than by editing "settings.py".*
Maybe it can be helpful for you.
Honestly I get more Django when I analyze his code. In version 1.4.5 did it (following the module below):
myproject\manage.py
django\core\management__init__.py ## method - execute_manager
django\conf__init__.py ## class - LazySettings; attr - _wrapped
django\utils\functional.py ## class LazyObject; important method -
new_method_proxy
Functional option, but it has its risks. In the python "_" considers the attribute as protected.
from django.conf import settings
settings._wrapped.INSTALLED_APPS = () ## *really work*
In the following project: https://github.com/alexsilva/DJPlugins
you can see this variable being modified at runtime. the idea of the project is already working.
You can use recomended .configure() method of settings module:
from django.conf import settings
settings.configure(DEBUG=True)
settings module has additional handy features. Check docs.
You cannot directly modify the settings.py file
For example:
If u want change the database at runtime, you should Separate the configuration of the database
# Projecr_name/user_database.py
user_database = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'name',
'USER': 'admin',
'PASSWORD': '111111',
'HOST': '127.0.0.1',
'PORT': '3306'
},
'user_db': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'user1',
'USER': 'admin',
'PASSWORD': '22222',
'HOST': '127.0.0.1',
'PORT': '3306'
}
}
# Projecr_name/settings.py
from .user_database import user_database
...
DATABASES = user_database
...
Call in your logical view
# view.py
from ../Projecr_name/user_database import user_database
class Some(...):
def Some(request):
user_database['user_db']['NAME']='user2'
then you can change any setting at runtime use this way
If you have a variable in the settings.py and you want it to change at any time there is 2 ways the
First one is to make a table in database then make for it a serializer then make a view set and any time you want to get it you could send a http request to this viewset then retrieve the data
Second one is to use caching it is so fast and it is familiar to localStorage in angular
Before I ask my question I need to give some context:
I wrote a simple python script that read linux's syslog file and search for certain strings. I have other similar scripts like these (scripts that do file system stuff, scripts that interact with other servers and so on). Most of these scripts write simple write stuff to stdout.
I would like to port these scripts to a web-server so I could simple browser to https://server/syslog and get the same output that I would get by running the script on the command line interface.
According with my research Django seems to be a great choice. I followed some Django tutorials and I was capable of developing some basic django web apps.
My question is: Since django does not have a "controller" where should I place the scripts code? My best bet in the view, but according with djangos documentation it does not make sence.
Extracted from django doc: In our interpretation of MVC, the “view” describes the data that gets presented to the user. It’s not necessarily how the data looks, but which data is presented. The view describes which data you see, not how you see it. It’s a subtle distinction.
The description of MVC is not so important. The typical use of django is for database backed web applications. And this describes a design pattern or paradigm for that. It's completely possible to use django in other ways as well.
If you want to build a django app that is a web interface for your existing scripts, you might not even need the django ORM at all. In any case, you can put as much or as little logic in your view as you want. Your use case might just not fit neatly into the MVC or MVT paradigm. Django views are just python functions (or classes, but Django class based views are more tightly coupled with the ORM).
I would recommend:
leaving your scripts largely as they are, but wrap the parts you want to reuse as
functions. You can keep them functional as standalone scripts with an
if __name__=='__main__':
block to call the functions.
Import the functions to views.py - it doesn't matter where they are as long as your server will always be able to find them. I put mine right in the app directory.
Call the function(s) in your view(s), and return the text to a HttpResponse object which you return from the view. (I think this is more direct than creating a template and a context and calling render, but its not what I usually do so there may be some issues?)
Thats bit old code - but you will get enough idea to start - check https://github.com/alex2/django_logtail (Django_LogTail)
I have an existing python application that runs in a console. This application contain features that is currently using sqlite3 as its data storage.
I am now trying to make use of Django as the presentation and it seems that I would need some help from you guys on how to speed up my transition from console to a web enabled user interface.
I understand that web is stateless and the app will be mostly consumed several people.
Most of my data is in dict which I can be able to convert to a JSON. I am seeking for kind advice on what are the things that I need to understand or if any of you guys have encountered this kind of app that you will need to migrate to a web centric presentation.
It has mostly query based functions which have few argumenta/parameters passed but it seems that I will have problem with the database access in python due to that most of my db queries are raw sql.
Thank you guys in advance.
(if there is any existing projects that I can use as a pattern by all means please do help to share with me)
==========
Resolution
Added this into manage.py:
sys.path.insert(0, os.path.join(BASE_DIR, 'app', 'engine'))
Now I can call my 'engine' package classes/scripts.
This is a pretty open-ended question, but I'll address moving your database to Django's models and custom queries.
Data Layer
Before building your app, you can try and use the manage.py command inspectdb to describe your current schema as Django models. It won't write your model layer for you, but can help you get started.
As #joran-beasley mentioned, sqlite will not be suitable as a production database for a multi-client application. Postgres is widely-used in the Django community, but it is hardly your option that supports raw SQL.
It is possible to migrate some data in XML, YAML, or JSON and import them with django's de/serializers or via the loaddata manage.py command. Consider adding custom managers to your models that are the targets of 1-to-1, Foreign Key, or Many-to-Many fields. You can add a custom get_by_natural_key method that can simplify de/serializing data that contains foreign keys, etc.
Queries
Django's QuerySet api is a wonderful wrapper for wrap SQL, but it will have some limitations. Take some time to review it's capabilities and shortcomings to decide if it can help or hinder your needs.
If you want to continue making raw queries, Django can help you with that too as it offers helper queryset methods like raw or extra. You can avoid the model layer entirely using its django.db.connection.cursor class.
I can't speak to much else that will help you in your transition, but you may find that Django's class-based generic views like DetailView, ListView, UpdateView, or DeleteView may be helpful in setting up some of your more basic interfaces on the web.
I come from Ruby on rails world. In rails, there is a file called schema.rb. It lists all the tables, columns and their types of the entire rails app.
Is there anyway in django to see the entire database schema at one place?
The models for one given app usually lives in the app's "models.py" module. Now Rails and Django might not have the same definition of what's an "app" is. In Django, you have a "project" which consists of one or more (usually more) "apps", and it's considered good practice to try to make apps as independant (hence potentially reusable) as possible, and there are indeed quite a few reusable apps available, so it's pretty uncommon to have all of the project's models in a single models.py module.
But anyway: if what you really want is "to see the entire database schema", then the best solution is to ask the database itself, whatever framework you use.
I have a fairly complex "product" I'm getting ready to build using Django. I'm going to avoid using the terms "project" and "application" in this context, because I'm not clear on their specific meaning in Django.
Projects can have many apps. Apps can be shared among many projects. Fine.
I'm not reinventing the blog or forum - I don't see any portion of my product being reusable in any context. Intuitively, I would call this one "application." Do I then do all my work in a single "app" folder?
If so... in terms of Django's project.app namespace, my inclination is to use myproduct.myproduct, but of course this isn't allowed (but the application I'm building is my project, and my project is an application!). I'm therefore lead to believe that perhaps I'm supposed to approach Django by building one app per "significant" model, but I don't know where to draw the boundaries in my schema to separate it into apps - I have a lot of models with relatively complex relationships.
I'm hoping there's a common solution to this...
Once you graduate from using startproject and startapp, there's nothing to stop you from combining a "project" and "app" in the same Python package. A project is really nothing more than a settings module, and an app is really nothing more than a models module—everything else is optional.
For small sites, it's entirely reasonable to have something like:
site/
models.py
settings.py
tests.py
urls.py
views.py
Try to answer question: "What does my
application do?". If you cannot answer
in a single sentence, then maybe you can
split it into several apps with cleaner
logic.
I read this thought somewhere soon after I've started to work with django and I find that I ask this question of myself quite often and it helps me.
Your apps don't have to be reusable, they can depend on each other, but they should do one thing.
What is to stop you using myproduct.myproduct? What you need to achieve that roughly consists of doing this:
django-admin.py startproject myproduct
cd myproduct
mkdir myproduct
touch myproduct/__init__.py
touch myproduct/models.py
touch myproduct/views.py
and so on. Would it help if I said views.py doesn't have to be called views.py? Provided you can name, on the python path, a function (usually package.package.views.function_name) it will get handled. Simple as that. All this "project"/"app" stuff is just python packages.
Now, how are you supposed to do it? Or rather, how might I do it? Well, if you create a significant piece of reusable functionality, like say a markup editor, that's when you create a "top level app" which might contain widgets.py, fields.py, context_processors.py etc - all things you might want to import.
Similarly, if you can create something like a blog in a format that is pretty generic across installs, you can wrap it up in an app, with its own template, static content folder etc, and configure an instance of a django project to use that app's content.
There are no hard and fast rules saying you must do this, but it is one of the goals of the framework. The fact that everything, templates included, allows you to include from some common base means your blog should fit snugly into any other setup, simply by looking after its own part.
However, to address your actual concern, yes, nothing says you can't work with the top level project folder. That's what apps do and you can do it if you really want to. I tend not to, however, for several reasons:
Django's default setup doesn't do it.
Often, I want to create a main app, so I create one, usually called website. However, at a later date I might want to develop original functionality just for this site. With a view to making it removable (whether or not I ever do) I tend to then create a separate directory. This also means I can drop said functionality just by unlinking that package from the config and removing the folder, rather than a complex delete the right urls from a global urls.py folder.
Very often, even when I want to make something independent, it needs somewhere to live whilst I look after it / make it independent. Basically the above case, but for stuff I do intend to make generic.
My top level folder often contains a few other things, including but not limited to wsgi scripts, sql scripts etc.
django's management extensions rely on subdirectories. So it makes sense to name packages appropriately.
In short, the reason there is a convention is the same as any other convention - it helps when it comes to others working with your project. If I see fields.py I immediately expect code in it to subclass django's field, whereas if I see inputtypes.py I might not be so clear on what that means without looking at it.
I've found the following blog posts very useful about django applications and projects:
http://www.b-list.org/weblog/2006/sep/10/django-tips-laying-out-application/
http://web.archive.org/web/20080302205555/www.pointy-stick.com/blog/2007/11/09/django-tip-developing-without-projects/
In principle, you have a lot of freedom with django for organizing the source code of your product.
If so... in terms of Django's project.app namespace, my inclination is to usemyproduct.myproduct, but of course this isn't allowed
There is nothing like not allowed. Its your project, no one is restricting you. It is advisable to keep a reasonable name.
I don't see any portion of my product being reusable in any context. Intuitively, I would call this one "application." Do I then do all my work in a single "app" folder?
In a general django project there are many apps (contrib apps) which are used really in every project.
Let us say that your project does only one task and has only a single app (I name it main as thethe project revolves around it and is hardly pluggable). This project too still uses some other apps generally.
Now if you say that your project is using just the one app (INSTALLED_APPS='myproduct') so what is use of project defining the project as project.app, I think you should consider some points:
There are many other things that the code other than the app in a project handles (base static files, base templates, settings....i.e. provides the base).
In the general project.app approach django automatically defines sql schema from models.
Your project would be much easier to be built with the conventional approach.
You may define some different names for urls, views and other files as you wish, but I don't see the need.
You might need to add some applications in future which would be real easy with the conventional django projects which otherwise it may become equally or more difficult and tedious to do.
As far as most of the work being done in the app is concerned, I think that is the case with most of django projects.
Here Django creators points out that difference themselves.
I think that thinking about Apps as they have to be reusable in other projects is good. Also a good way of thinking about Apps in Django provide modern web applications.
Imagine that you are creating big dynamic web app basing on JavaScript.
You can create then in django App named e.g "FrontEnd" <-- in thins app you will display content.
Then you create some backend Apps. E.g App named "Comments" that will store user comments. And "Comments" App will not display anything itself. It will be just API for AJAX requests of your dynamic JS website.
In this way you can always reuse your "Comments" app. You can make it open source without opening source of whole project. And you keep clean logic of your project.