Django's TEMPLATE_DIRS in Settings.py calls for unix style slashes.
Because of this, when I call
get_template('some/template.html')
in a view, the result always starts at the root, and results in a call to
/home/username/projectname/public/some/template.html
The problem is that I'd like to use templates hosted on an entirely different site. This works fine for other Settings.py fields (MEDIA_URL and STATIC_URL), where it will take an absolute http path with no objection.
Given an http path,
TEMPLATE_DIRS ('http://example.com/',)
in Settings.py will force
get_template('some/template.html')
in a view to try and find
/home/username/projectname/public/http://example.com/some/template.html
I've tried to circumvent this like so
TEMPLATE_DIRS ('../../../../http://example.com/',)
But it still forces a leading slash, so I get "/http://example.com", which is useless.
My questions:
Is there a way to trick this into pulling the template files from
another server?
Is that even feasible, given that the template files need to be
processed for the view?
Is it possible to create an alternate to 'django.template.loaders.filesystem.Loader' that doesn't call for unix style slashes?
You don't need to use the template directory is you dont want to. If you have a server that is serving template files, you can simply fetch them remotely using urllib2 and create and render the template with a context manually:
import urllib2
from django.template import Context, Template
tpl_html = urllib2.urlopen("http://mysite.com")
tpl = Template(tpl_html)
return tpl.render(Context({
'some_variable' : 'some_val',
})
If you are going to do this, you have to incorporate some caching, as for every request to using this template, you need to make an external request. Alternatively you could write this into a custom loader but it will suffer the same limitations.
You can't do this.
It has nothing to do with path names. It's just that the filesystem template loader needs to load things from the filesystem, hence the name.
This is completely different from the case of MEDIA_URL: that simply adds a path into your HTML, which your browser then loads. Django doesn't care where that file lives: although in fact the opposite applies, in that if you pass it a filepath that isn't a URL (ie served by a webserver somewhere), it simply won't work.
Now, you could write a template loader that gets its templates from another server. Template loaders are pluggable - you just need to put the name of your new loader in the TEMPLATE_LOADERS setting. The loader itself would need to use something like urllib.urlopen to get the template from the external server.
But think very carefully before you do this. This means that every single template request now requires a call to an external server before you can serve the page. In the typical case of a template that extends other templates and includes calls to included template tags, that might be five or ten calls. And, unlike media files, it can't be done in parallel: the page simply won't be served until the whole process is finished. This is likely to make your webserver very very slow.
I don't know why you think you need to do this. Templates are part of your application code, so they would normally live on the same server as your Python code. If you really have some reason to keep them externally, one solution might be to mount the external filesystem onto your webserver via something like sshfs. It's still likely to be very slow though. Think again.
No - it's not possible to trick it into pulling files from another server via http
Yes - you certainly could subclass django.template.loaders.filesystem.Loader (and by altering the load_template_source method appropriately) so that you could load templates via http
Once you had done 3 then the answer to 2 would be yes - it would be feasible - ultimately Django's template language doesn't care where it gets the file from, as long it's in the right format.
However, it seems like a very inefficient way of loading templates and more likely there is a much better way of achieving the same result.
Related
I'm porting a web app from PHP (Codeigniter) to Python (powered by Django.) I already migrated the database and am now using Django models. I populated the database with images using Django's ImageField field and the images are in its proper folders under MEDIA_ROOT.
I'm in development so I'm using Django's server to test the web app as I go and I can serve the images through image.url.
image.url serves the image from:
MEDIA_ROOT/folder/subfolder/numberImage_folder_subfolder.jpg
But I need it to be served from MEDIA_ROOT/numberImage_folder_subfolder.jpg.
In production I know I can serve the image using nginx's XSendFile or something like that. Django's server (runserver command) is very easy to use and gives me what I need at the moment. While in development, how can I deliver the image from the URL I require? Preferably using Django's server...
Any help will be so much appreciated.
I'm not entirely clear on what your URLs are and how they map to file paths (and don't have the reputation to ask for clarification yet). So correct me if I've understood it wrong.
But in the meantime, to rephrase the question slightly, I think you're saying that you have the files stored on disk in those subfolders like MEDIA_ROOT/folder/subfolder/numberImage_folder_subfolder.jpg, but when you actually want the URLs served, you don't want the folders in the URL, so the URL will have MEDIA_ROOT/numberImage_folder_subfolder.jpg? And only with the ./manage.py runserver?
Going ahead on those assumptions, we have two steps: map the URLs we want back to a file path, and then also make sure we're generating the URLs we want.
For the first, normally urls.py will have something like this to serve media files in development:
from django.conf.urls.static import static
if settings.DEBUG is True:
urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
which is basically adding url(r'^media/(?P<path>.*)$', django.views.static.serve) to your urlconf, where serve takes all paths below settings.MEDIA_URL and serves up the equivalent file under MEDIA_ROOT.
The simplest way to do this would be to create a custom function modelled on that same django.conf.urls.static which takes the path and adds the folders back in before passing it on to django.views.static.serve.
For the second, if you want the URLs generated from the ImageField.url to differ from the underlying file storage path, you'll need to have a custom ImageField subclass overriding the FileField.url property (or a custom Storage class that does a similar thing).
Like so:
class CustomURLImageField(django.forms.fields.ImageField):
def _get_url(self):
"""
Override the standard FileField.url property to mangle the URL from
MEDIA_ROOT/folder/subfolder/numberImage.jpg
to
MEDIA_ROOT/numberImage_folder_subfolder.jpg
"""
self._require_file()
#return self.storage.url(self.name)
path_components = self.name.split('/')
folder = path_components[-3]
subfolder = path_components[-2]
numberImage, ext = os.path.splitext(path_components[-1])
return '/{numberImage}_{folder}_{subfolder}{ext}'.format(**locals())
url = property(_get_url)
This will be likely true in production too... how are you generating your URLs?)
Django official documentation and other tutorials on the web always use a trailing slash at the end of url. ex:
url(r'^accounts/login/', views.login) # login view in turn calls login.html
# instead of
url(r'^accounts/login', views.login)
Since accounts is the directory and login (login.html) is the file, shouldn't we be using second url? This will also make GET parameters look more structured:
accounts/login?name='abc' # login is a file that is accepting parameters
vs.
accounts/login/?name='abc' # login directory (maybe index) is accepting parameters??
One of Django’s core design philosophies is URLs should be beautiful.
So some url like accounts/detail?name='abc' should be mapped as accounts/detail/abc/. You can capture it using regex at your url configurations. Here the URL is pretty neat and user friendly. This will help the search engines to index your pages correctly (now you can forget about rel=canonical) and will help in seo.
Now the reason for a trailing slash, consider a view (in any framework) which relatively resolves about.html for a user at path, users/awesomeUser
since users/awesomeUser and users/awesomeUser/ are different,
If the user is at users/awesomeUser, the browser will resolve it as users/about.html because there ain't a trailing slash which we don't want
If the user is at users/awesomeUser/, the browser will resolve it as users/awesomeUser/about.html because there is a trailing slash
child relative to family/parent/ is family/parent/child.
child relative to family/parent is family/child.
Django Design philosophy on Definitive URLs reads,
Technically, foo.com/bar and foo.com/bar/ are two different URLs, and search-engine robots (and some Web traffic-analyzing tools) would treat them as separate pages. Django should make an effort to “normalize” URLs so that search-engine robots don’t get confused.
This is the reasoning behind the APPEND_SLASH setting. (APPEND_SLASH lets you force append slashes to a URL)
Still not convinced?
Since django observes both the urls as different, if you are caching your app, Django will keep two copies for same page at user/awesomeUser and user/awesomeUser/.
You gotta have issues with HTTP methods other than GET if you don't append slash to a URL (If you ever plan to build a REST API).
Update
You can't make POST/PUT/PATCH/DELETE methods to work with rest_framework unless you explicitly define APPEND_SLASH=False in settings and trailing_slash=False for each and every router you gotta use(if you use Routers). It is like you basically gonna skip this most times and you gotta waste a hell lot of time debugging this. Django recommends append slashes and doesn't force it.
Its up to the developer to append slashes or not.
From the docs for the middleware that uses APPEND_SLASH
a search-engine indexer would treat them as separate URLs – so it’s best practice to normalize URLs.
Its not required by django, its just trying to help your SEO by suggesting a standard way of doing urls.
Yes, I know that the slash has nothing to do with this middleware but this is the best explanation I could find as to a possible reason
This helps define the structure of your website. Although django can support anything entered after the domain that is passed to the server doing it in this way allows you to easily add "subpages" to the url without it looking like accounts/loginreset?id=alkfjahgouasfjvn25jk1k25
That being said in the case above it may make sense to leave it out.
"URLs should be beautiful"!!!
I want to be able to control URLs. It's nothing nice when everything is about to be overwritten. Under circumstances, I make a redirect loop which is not funny.
from django.http import HttpResponseRedirect as rdrct
url(r'^sitemap.xml$', 'my_app.views.custom_sm'),
url(r'^sitemap.xml/$', lambda x: rdrct('/sitemap.xml')),
My project's URLs are automatically generated in urls.py using a for loop (the URLs look like AppName/ViewName). According to the docs, urls.py is loaded upon every request. This appears to be slowing my site down since it requires a bunch of introspection code, so I want to generate the URLs less frequently. I could of course manually run a script to re-generate urls.py (or a file imported by urls.py) as needed, but would prefer if it happened automatically as part of project validation/startup (like the server starting up or the database being synced). I'm open-sourcing this project, and many people will be running it on their own servers, so I want to do this in a robust way. Any recommendations?
The docs do not say what you claim they do (or rather, you're reading too much into a phrase which only means "loads that Python module (if it has not already been loaded)".
Generally, the only things that happen on every request are running the middleware and the specific view code associated with that request. Even then, nothing is reloaded on every request. URLs, like all Python code, is only loaded when a new process is started, and when that happens depends on your server setup. Your problem is elsewhere: you should profile your application carefully to find out exactly where.
For example you can look for django-json-rpc where author has realized url-generating via decorators. There are main controller which receive all requests and urls dict {'pattern': method}. urls dict filled automatically by decorators like #jsonrpc_method which receive a function and put them to urls.
I think it must run faster than the for and I believe that this approach will be able to apply for django.urls
I'm having a rather strange problem using a jinja2.ChoiceLoader (also tried with multiple paths with FileSystemLoader, no joy) in Flask.
I have several "theme" directories, like so.
/templates/
themes/
default/
layout.html
menu.html
blue/
layout.html
grey/
menu.html
...
And I'd like to fallback to default/ if the selected theme doesn't have the required template, so I used a ChoiceLoader, like so.
#app.before_request
def setup_request():
current_theme = get_theme()
logging.info('Using theme %s'%(current_theme))
app.jinja_loader = jinja2.ChoiceLoader([
jinja2.FileSystemLoader('/templates/themes/%s/'%(current_theme)),
jinja2.FileSystemLoader('/templates/themes/default/')
])
That's great, but if I change the <current_theme> it still loads the theme from the old folder until I reload Apache or restart the Flask development server.
It should be using the new theme. Logging says that it's using the changed theme, but apparently app.jinja_loader is a bit like honey badger... it's completely ignoring it until I reload Apache.
Edit: This appears to be related to Flask considering all files of the same name to be the same file. I can reproduce with the builtin server (with DEBUG=True), Cherry, and mod_wsgi. This person seems to have a similar problem, but no simple solution: flask blueprint template folder My situation is different in the sense that I require cascading templates for a single app. His problem is related to cascading templates between blueprints, but it may be the same issue under the hood.
Here's the code that's in the "get_theme()" call:
def get_theme():
# I know this is unsafe, testing only
return request.args.get('theme','default')
Edit 2: I need to change the HTML and JavaScript between themes, not just the CSS. This is why I'm not just loading different CSS files. Also, some of these themes are for mobile devices, and have very little in common with the other themes.
Edit 3: Two solutions. Solution 1: Name the files uniquely, like "blue.layout.html" and "default.layout.html". This works perfectly, but it doesn't cascade as required. Solution 2: Use relative paths, so instead of include 'file.html', use include 'theme/blue/file.html. I achieved cascading by creating a get_theme_file() function that checks for the active theme, checks if the file exists (if not, fallback to "default" theme), and returns the relative path. I just have to make sure everything I include looks like {% include get_theme_file('file.html') %}. This is not elegant, but I find it to be more elegant that the low-level fiddling with Flask used here.
By the way, you can pass multiple locations to FileSystemLoader, and it is the recommended way to load templates
This is expected behavior in Apache with mod_wsgi (which I assume you are using). File system changes don't trigger a reload of all the processes. See this entry in the Flask docs that talks about this, and offers a workaround which is to add:
WSGIScriptReloading On
To the configuration section for your application and then touching the wsgi file to trigger a reload of the child processes.
Are you sure this is what you intend? Most theme switching tricks rely on the cascading part of cascading style sheets (CSS) to control themes.
Well, I'm not the only one to encounter this problem. The issue is that Flask caches based on filename, and if you don't include the relative path it just caches the first one to be loaded. There are three ways to accomplish dynamic, cascading, templates.
Override Jinja builtins. Which I found to be quite confusing. I'm not smart enough for this solution.
Serve a different WSGI process for each file. The setup here seems a bit too much for a dynamic site. Flask caches the filenames per WSGI process, so you can do something with multiple Cherry WSGI servers, for example.
Include the theme in the load path. Create a function, load it in with context_processor, and only load template files using that function. The function needs to check for a non-default theme, check if the file exists, and return it. So a template call would be get_theme_file('layout.html') which would return the relative path (like themes/blue/layout.html).
An example of Option 3.
def get_theme_file(fname):
theme = get_theme()
if os.path.exists(os.path.join(theme.theme_dir, fname)):
return os.path.join('themes', theme.name, fname)
return os.path.join('themes', 'default', fname)
...
# Each render_template should reference this
return render_template(get_theme_file('layout.html'))
If you include theme files in templates:
{% include get_theme_file('layout.html') %}
Unfortunately, this doesn't cache, but I can see a few ways to optimize it. Maybe cache an os.listdir of the get_theme().theme_dir, and instead of reading the filesystem for each get_theme_file call, just do an in check against the cached listdir list.
It's worth noting that this is Flask specific. I was unable to reproduce this behavior with plain Jinja2 and my own WSGI server. One might say that Flask was a poor choice for this particular project, but I'd argue that the savings from everything else Flask does was well worth it.
Is there any way to make Django's template loader run all templates it loads (i.e. directly or via extend/include) through SHPAML if it figures the HTML is out of date?
I know how to invoke SHPAML recursively over an entire directory, but I would prefer to be able to run it on demand so I don't have to remember to sync the HTML every time I change the SHPAML source.
I guess invoking SHPAML from manage.py would work too (at least for test servers), but being able to hack into Django's template engine and make it run every file it loads through a preprocessor would be nicer.
I suspect you can achieve what you want by inheriting from django.template.loaders.app_directories.Loader (or whatever loader you use) and overwriting the load_template_source method, e.g.:
from django.template.loaders.app_directories import Loader
from shpaml import convert_text
class SHPAMLLoader(Loader):
def load_template_source(self, *args, **kwargs):
shpaml_source = super(SHPAMLLoader, self).load_template_source(*args, **kwargs)
html = convert_text(shpaml_source)
return html
Then put your loader at the beginning of the TEMPLATE_LOADERS tuple in your settings.py. Of course, you will be doing the SHPAML to HTML dance every time a template is loaded, so you may see some overhead. The upcoming Django 1.2 features template caching, which could help mitigating that overhead...
Disclaimer: this code is completely untested, sorry.
Just created a project based on the snippet in piquadrat's answer. It's a little more feature complete and supports django 1.1 and 1.2 (probably 1.0 as well)
django-shpaml-template-loader on bitbucket
Thought it might come in handy for the future :)