Develop Reference

Why django urls end with a slash?

Django official documentation and other tutorials on the web always use a trailing slash at the end of url. ex:
url(r'^accounts/login/', views.login) # login view in turn calls login.html
# instead of
url(r'^accounts/login', views.login)
Since accounts is the directory and login (login.html) is the file, shouldn't we be using second url? This will also make GET parameters look more structured:
accounts/login?name='abc' # login is a file that is accepting parameters
vs.
accounts/login/?name='abc' # login directory (maybe index) is accepting parameters??

One of Django’s core design philosophies is URLs should be beautiful.
So some url like accounts/detail?name='abc' should be mapped as accounts/detail/abc/. You can capture it using regex at your url configurations. Here the URL is pretty neat and user friendly. This will help the search engines to index your pages correctly (now you can forget about rel=canonical) and will help in seo.
Now the reason for a trailing slash, consider a view (in any framework) which relatively resolves about.html for a user at path, users/awesomeUser
since users/awesomeUser and users/awesomeUser/ are different,
If the user is at users/awesomeUser, the browser will resolve it as users/about.html because there ain't a trailing slash which we don't want
If the user is at users/awesomeUser/, the browser will resolve it as users/awesomeUser/about.html because there is a trailing slash
child relative to family/parent/ is family/parent/child.
child relative to family/parent is family/child.
Django Design philosophy on Definitive URLs reads,
Technically, foo.com/bar and foo.com/bar/ are two different URLs, and search-engine robots (and some Web traffic-analyzing tools) would treat them as separate pages. Django should make an effort to “normalize” URLs so that search-engine robots don’t get confused.
This is the reasoning behind the APPEND_SLASH setting. (APPEND_SLASH lets you force append slashes to a URL)
Still not convinced?
Since django observes both the urls as different, if you are caching your app, Django will keep two copies for same page at user/awesomeUser and user/awesomeUser/.
You gotta have issues with HTTP methods other than GET if you don't append slash to a URL (If you ever plan to build a REST API).
Update
You can't make POST/PUT/PATCH/DELETE methods to work with rest_framework unless you explicitly define APPEND_SLASH=False in settings and trailing_slash=False for each and every router you gotta use(if you use Routers). It is like you basically gonna skip this most times and you gotta waste a hell lot of time debugging this. Django recommends append slashes and doesn't force it.
Its up to the developer to append slashes or not.

From the docs for the middleware that uses APPEND_SLASH
a search-engine indexer would treat them as separate URLs – so it’s best practice to normalize URLs.
Its not required by django, its just trying to help your SEO by suggesting a standard way of doing urls.
Yes, I know that the slash has nothing to do with this middleware but this is the best explanation I could find as to a possible reason

This helps define the structure of your website. Although django can support anything entered after the domain that is passed to the server doing it in this way allows you to easily add "subpages" to the url without it looking like accounts/loginreset?id=alkfjahgouasfjvn25jk1k25
That being said in the case above it may make sense to leave it out.

"URLs should be beautiful"!!!
I want to be able to control URLs. It's nothing nice when everything is about to be overwritten. Under circumstances, I make a redirect loop which is not funny.
from django.http import HttpResponseRedirect as rdrct
url(r'^sitemap.xml$', 'my_app.views.custom_sm'),
url(r'^sitemap.xml/$', lambda x: rdrct('/sitemap.xml')),

Using external URLs in Django's TEMPLATE_DIRS

Django's TEMPLATE_DIRS in Settings.py calls for unix style slashes.
Because of this, when I call
get_template('some/template.html')
in a view, the result always starts at the root, and results in a call to
/home/username/projectname/public/some/template.html
The problem is that I'd like to use templates hosted on an entirely different site. This works fine for other Settings.py fields (MEDIA_URL and STATIC_URL), where it will take an absolute http path with no objection.
Given an http path,
TEMPLATE_DIRS ('http://example.com/',)
in Settings.py will force
get_template('some/template.html')
in a view to try and find
/home/username/projectname/public/http://example.com/some/template.html
I've tried to circumvent this like so
TEMPLATE_DIRS ('../../../../http://example.com/',)
But it still forces a leading slash, so I get "/http://example.com", which is useless.
My questions:
Is there a way to trick this into pulling the template files from
another server?
Is that even feasible, given that the template files need to be
processed for the view?
Is it possible to create an alternate to 'django.template.loaders.filesystem.Loader' that doesn't call for unix style slashes?

You don't need to use the template directory is you dont want to. If you have a server that is serving template files, you can simply fetch them remotely using urllib2 and create and render the template with a context manually:
import urllib2
from django.template import Context, Template
tpl_html = urllib2.urlopen("http://mysite.com")
tpl = Template(tpl_html)
return tpl.render(Context({
'some_variable' : 'some_val',
})
If you are going to do this, you have to incorporate some caching, as for every request to using this template, you need to make an external request. Alternatively you could write this into a custom loader but it will suffer the same limitations.

You can't do this.
It has nothing to do with path names. It's just that the filesystem template loader needs to load things from the filesystem, hence the name.
This is completely different from the case of MEDIA_URL: that simply adds a path into your HTML, which your browser then loads. Django doesn't care where that file lives: although in fact the opposite applies, in that if you pass it a filepath that isn't a URL (ie served by a webserver somewhere), it simply won't work.
Now, you could write a template loader that gets its templates from another server. Template loaders are pluggable - you just need to put the name of your new loader in the TEMPLATE_LOADERS setting. The loader itself would need to use something like urllib.urlopen to get the template from the external server.
But think very carefully before you do this. This means that every single template request now requires a call to an external server before you can serve the page. In the typical case of a template that extends other templates and includes calls to included template tags, that might be five or ten calls. And, unlike media files, it can't be done in parallel: the page simply won't be served until the whole process is finished. This is likely to make your webserver very very slow.
I don't know why you think you need to do this. Templates are part of your application code, so they would normally live on the same server as your Python code. If you really have some reason to keep them externally, one solution might be to mount the external filesystem onto your webserver via something like sshfs. It's still likely to be very slow though. Think again.

No - it's not possible to trick it into pulling files from another server via http
Yes - you certainly could subclass django.template.loaders.filesystem.Loader (and by altering the load_template_source method appropriately) so that you could load templates via http
Once you had done 3 then the answer to 2 would be yes - it would be feasible - ultimately Django's template language doesn't care where it gets the file from, as long it's in the right format.
However, it seems like a very inefficient way of loading templates and more likely there is a much better way of achieving the same result.

What is a good Django workflow?

I'm a beginner to Python and Django.
When starting a new project what do you do first before diving into the code?
For example, one could take the following steps:
Configure the settings.py file first
Configure models.py to lay out data structure
Create template files
Define the views/pages
Syncdb
etc
So my question is, what is a good workflow to get through the required steps for a Django application? This also serves as a checklist of things to do. In the definitive guide to Django, the author talks about approaching top down or bottom up. Can anyone expand further on this and perhaps share their process?
Thanks.

Follow the Agile approach. Finish one small case, from the start to the end. From the models to the tests to user experience. Then build on it. Iterate.
Thats the right way to software development.
To do it efficiently, you need: (don't bother right away, you will need it.)
Automated schema migration, automated build system, auto updating and deployment. - None of these, django has got anything to do with. Use pip, fabric, hudson, twill and south appropriately.
Take care not to over burden yourself with all these right away, particularly since you say, you are beginning.

the required steps for a Django application?
There are two required steps.
Write the settings. Write the urls.py
The rest of the steps are optional.
This also serves as a checklist of things to do.
Bad policy. You don't need a checklist of Django features. You need a collection of use cases or user stories which you must implement.
For some reason, you've omitted the two most important and valuable features of Django. Configure the default admin interface and write unit tests. The default admin interface is very high value. Unit testing is absolutely central.
You do it like this.
Gather use cases.
Prioritize the use cases.
Define the actors. The classes of actors becomes groups in the security model.
Define enough "applications" to satisfy the first release of use cases. Define the url structure. Cool URL's don't change.
Build the first use case: models (including security), admin, urls, tests, forms, views and templates. Note that these are the file names (models.py, admin.py, ...) except for templates. Also note that forms and admin should be defined in separate modules even though this isn't required. Also note that templates will be split between a generic templates directory for top-level stuff and application-specific templates.
Build the second use case: models (including security), admin, urls, tests, forms, views and templates.
...
n. Package for release. Tweak up the settings. Configure database and mod-wsgi

I personally can't make a template without writing the views (unless it's a photoshop draft) but in general that's the way I go after I have a plan.
What's extremely important for me is that I don't dive head-first into the code, and that I spend time mocking up the model structure based on the "screens" or "pages" that the user will see.
Once I have a user experience defined, I make sure the backend is robust enough to handle that experience. If I don't visualize the user experience, details get left out that are certainly accomplishable in the shell but not ideal for the website, default django admin, etc.
There are always tradeoffs between agile development and a huge spec: I think there's an important balance. Agile is good: there's no point planning every detail before writing your first line of code, as your needs will change by the time you get to the end. You don't know how your users will really use the site.
On the other hand, without a plan, you can end up with a messy foundation that affects all future code.
An educated guess is a good start. Don't think or assume too much, but definitely have a clear idea how your users will interact with your site for stage 1.

Always try to remember about a DRY rule. For example, why to write RequestContext every new view is defined, where you can simply write a function once, which will add it for you. Good description is here in another topic.
Try to keep a code written one way. Each time you upgrade a schema of your view, edit it in all already written views. That will help keep your code clear and save a lot time for you in future.
Generally good rule, and how do I write my applications is the rule of small steps. Start with writing a settings and urls, then add one model and one view. When it works, modify - add another models or another views. You won't even notice, when your project becomes bigger and bigger.
And the last useful rule for clarity of all the source. Keep files in folders. If you have two subsites based one one (for example "accounts" and "blogs") create two directories names the same. Remeber to put init.py file in each directory. It's really easy to forget. With this practice it's easy to write models and views dedicated to each category. By the way it's a good practice to keep urls like in a tree structure. Main urls.py should contain only links like this one:
(r'^accounts/', include('your_main_name.accounts.urls')),
and of course all media, static, css and so on. In accounts directory urls keep:
urlpatterns = patterns('your_main_name.accounts.views',
url(r'^$', 'index', name='index'),
)
with all views subdirectories.
Last one - keep code clear with actuall django version. Remeber, that the 3.0 release is comming soon.
Hope this will help.

I find that my process varies depending on a lot of variables, mainly whether I know something will work or if I'm experimenting and also whether I'm developing on my production server or in a development environment.
For example, I often do my development directly on the deployment server (most of my work is for intranet projects so there isn't any security risk, etc). But when I do this I really need to make sure the settings and urls are setup first and that gunicorn and nginx are configured.
If I know something should work, or am setting up a generic base set of code, sometimes I'll do all that coding for views and models before I even get enough setup to even run the development server. But when experimenting with new code I find it's good to be able to test every step of the way, so in that case you need your servers running.
In general I do settings, models, syncdb, views, urls, templates, collectstatic, graphics/aesthetics
In general I leave my base.html very plain until the everything else is working, then I add css/js etc.
I guess my point here is that there isn't really a wrong answer for how you do it, and there isn't even only one best practice (as far as I'm concerned). When you do more work, you'll find what you are comfortable with and it'll even vary from project to project.
Good luck, hopefully you learn to love django!

here is something I do in general,
configure basic settings
configure root url.py
configure settings, url.py for static (media) files
create model
sync db
write views (use simple template, if needed)
once you are done with back end implementation
think about UI
prepare styles, scripts
start working on template implementation

Django: "projects" vs "apps"

I have a fairly complex "product" I'm getting ready to build using Django. I'm going to avoid using the terms "project" and "application" in this context, because I'm not clear on their specific meaning in Django.
Projects can have many apps. Apps can be shared among many projects. Fine.
I'm not reinventing the blog or forum - I don't see any portion of my product being reusable in any context. Intuitively, I would call this one "application." Do I then do all my work in a single "app" folder?
If so... in terms of Django's project.app namespace, my inclination is to use myproduct.myproduct, but of course this isn't allowed (but the application I'm building is my project, and my project is an application!). I'm therefore lead to believe that perhaps I'm supposed to approach Django by building one app per "significant" model, but I don't know where to draw the boundaries in my schema to separate it into apps - I have a lot of models with relatively complex relationships.
I'm hoping there's a common solution to this...

Once you graduate from using startproject and startapp, there's nothing to stop you from combining a "project" and "app" in the same Python package. A project is really nothing more than a settings module, and an app is really nothing more than a models module—everything else is optional.
For small sites, it's entirely reasonable to have something like:
site/
models.py
settings.py
tests.py
urls.py
views.py

Try to answer question: "What does my
application do?". If you cannot answer
in a single sentence, then maybe you can
split it into several apps with cleaner
logic.
I read this thought somewhere soon after I've started to work with django and I find that I ask this question of myself quite often and it helps me.
Your apps don't have to be reusable, they can depend on each other, but they should do one thing.

What is to stop you using myproduct.myproduct? What you need to achieve that roughly consists of doing this:
django-admin.py startproject myproduct
cd myproduct
mkdir myproduct
touch myproduct/__init__.py
touch myproduct/models.py
touch myproduct/views.py
and so on. Would it help if I said views.py doesn't have to be called views.py? Provided you can name, on the python path, a function (usually package.package.views.function_name) it will get handled. Simple as that. All this "project"/"app" stuff is just python packages.
Now, how are you supposed to do it? Or rather, how might I do it? Well, if you create a significant piece of reusable functionality, like say a markup editor, that's when you create a "top level app" which might contain widgets.py, fields.py, context_processors.py etc - all things you might want to import.
Similarly, if you can create something like a blog in a format that is pretty generic across installs, you can wrap it up in an app, with its own template, static content folder etc, and configure an instance of a django project to use that app's content.
There are no hard and fast rules saying you must do this, but it is one of the goals of the framework. The fact that everything, templates included, allows you to include from some common base means your blog should fit snugly into any other setup, simply by looking after its own part.
However, to address your actual concern, yes, nothing says you can't work with the top level project folder. That's what apps do and you can do it if you really want to. I tend not to, however, for several reasons:
Django's default setup doesn't do it.
Often, I want to create a main app, so I create one, usually called website. However, at a later date I might want to develop original functionality just for this site. With a view to making it removable (whether or not I ever do) I tend to then create a separate directory. This also means I can drop said functionality just by unlinking that package from the config and removing the folder, rather than a complex delete the right urls from a global urls.py folder.
Very often, even when I want to make something independent, it needs somewhere to live whilst I look after it / make it independent. Basically the above case, but for stuff I do intend to make generic.
My top level folder often contains a few other things, including but not limited to wsgi scripts, sql scripts etc.
django's management extensions rely on subdirectories. So it makes sense to name packages appropriately.
In short, the reason there is a convention is the same as any other convention - it helps when it comes to others working with your project. If I see fields.py I immediately expect code in it to subclass django's field, whereas if I see inputtypes.py I might not be so clear on what that means without looking at it.

I've found the following blog posts very useful about django applications and projects:
http://www.b-list.org/weblog/2006/sep/10/django-tips-laying-out-application/
http://web.archive.org/web/20080302205555/www.pointy-stick.com/blog/2007/11/09/django-tip-developing-without-projects/
In principle, you have a lot of freedom with django for organizing the source code of your product.

If so... in terms of Django's project.app namespace, my inclination is to usemyproduct.myproduct, but of course this isn't allowed
There is nothing like not allowed. Its your project, no one is restricting you. It is advisable to keep a reasonable name.
I don't see any portion of my product being reusable in any context. Intuitively, I would call this one "application." Do I then do all my work in a single "app" folder?
In a general django project there are many apps (contrib apps) which are used really in every project.
Let us say that your project does only one task and has only a single app (I name it main as thethe project revolves around it and is hardly pluggable). This project too still uses some other apps generally.
Now if you say that your project is using just the one app (INSTALLED_APPS='myproduct') so what is use of project defining the project as project.app, I think you should consider some points:
There are many other things that the code other than the app in a project handles (base static files, base templates, settings....i.e. provides the base).
In the general project.app approach django automatically defines sql schema from models.
Your project would be much easier to be built with the conventional approach.
You may define some different names for urls, views and other files as you wish, but I don't see the need.
You might need to add some applications in future which would be real easy with the conventional django projects which otherwise it may become equally or more difficult and tedious to do.
As far as most of the work being done in the app is concerned, I think that is the case with most of django projects.

Here Django creators points out that difference themselves.
I think that thinking about Apps as they have to be reusable in other projects is good. Also a good way of thinking about Apps in Django provide modern web applications.
Imagine that you are creating big dynamic web app basing on JavaScript.
You can create then in django App named e.g "FrontEnd" <-- in thins app you will display content.
Then you create some backend Apps. E.g App named "Comments" that will store user comments. And "Comments" App will not display anything itself. It will be just API for AJAX requests of your dynamic JS website.
In this way you can always reuse your "Comments" app. You can make it open source without opening source of whole project. And you keep clean logic of your project.

Where to place the call to a external API that has a dependency on a Model?

Suppose I have a Django app named "blog".
There's a model called Post and I have an external API call that returns the list of the most popular posts in a given time, e.g., the Google Analytics API.
My question is: Where is the expected place that I should place the code that makes the call to the external API, parses the id from each post, query the database and sort the list of models accordingly?
I don't think that it should live in the Manager or in a templatetag. Any tips or suggestions?
Thanks in advance!
EDIT: The desired result might be need in several places across the project, so if I place the code in view, I'll have duplication.

It should be done in a view, or better, if your view code is getting cluttered, put it in a helper module.
import util
def view(request):
util.process_post_rankings(request.user.id)
# ... write additional logic and render to template
However, be cautious with relying on external apis to render a page to your user. Things might go wrong, take an awful lot of time, API might not respond etc... Better to do that asynchronously with Javascript, and update the page when data is ready.

This sounds like it should be done in a View, because then you will be returning to a template with all the required context.