I'm using the Django manage.py runserver for developing my application (obviously), but it takes 10 seconds to completely load a page because the development server is very, very slow at serving static media.
Is there any way to speed it up or some kind of workaround? I'm using Windows 7.
Consider using mod_wsgi instead, and having httpd handle the static media.
Development server is simple unsafe single-threaded application, so you cannot do much.
One trick you could try is to redirect /site_media to second development server, but this is ugly and probably wouldn't help that much. So you could try bundling/compressing multiple assets into one css/js (e.g. using YUI Compressor).
And in any case, you should have separate static media server, that can serve multiple assets at once.
Install Firefox (if you haven't already), and install the Firebug Add-On. Restart your browser. In the lower-right corner click the "bug" icon and make sure that in the "Network" tab (it's a dropdown) of the Firebug panel that opens at the bottom of the browser, the network monitor is active.
Now with the network tab of Firebug open, open your Django-generated page that you observed to load slowly. Take a look at the timeline bars. You'll notice that the colored fragment(s) of each bar indicate(s) the reason for each request's total "load" time. Violet, for instance, means that actually the browser is waiting for the server to generate the response. Gray means it's receiving content. And so on. Hovering over the bars will display a color legend.
With Firebug's network monitor you should be able to pinpoint how exactly your browser and/or server are spending their 10 seconds.
Run lighttpd to serve the static content and use the MEDIA_URL to point the pages at the lighttpd server that servers the static stuff.
You can try using django-extensions runserver_plus command with the --threaded option as a replacement for Django's runserver command. Under the hood, it uses Werkzeug as the threaded WSGI server. You may notice a huge improvement in loading times for static files.
Lightning fast ans easy on resources when using NGINX for serving static and media files. Here's how it goes. However, you'll need to adapt some paths according to your use case and system. But I think this will get you started:
1) Download NGINX for your system, in your case Windows:
http://nginx.org/
2) Unpack the zip file. Here's how your NGINX config file may look like. This file lives inside nginx/conf/:
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
# root path to your project
# use "..." if spaces are in the path
# you may add a drive letter if required, e.g. root c:/foo
# use / instead of \. It's simpler and works
root /example/path/django/project/;
server {
# any free port number will do; Django dev server usually runs on port 80
listen 8000 default;
server_name localhost; # or 127.0.0.1
}
}
3) Start NGINX by calling the nginx.exe - no options needed.
4) Tweak your Djnago project's settings.py file:
if DEBUG:
STATIC_URL = 'http://localhost:8000/static/'
# against Django recommendation, I often still use the static
# directory for user uploads; old-style Django ;-)
MEDIA_URL = 'http://localhost:8000/static/uploads/'
Now, the static URLs in Django should look something like this:
http://localhost:8000/static/js/base.js
... 5) Stop the server by calling:
taskkill /f /IM nginx.exe
Well, that's it. I typed this quickly, so just let me know if anything's unclear or not working for you. I may improve this answer accordingly.
Related
I cannot get a bokeh plot to work on a deployed server because of cross-domain issues. I have asked this question in a few forms and am not really getting anywhere.
I always get the error
XMLHttpRequest cannot load http://127.0.0.1:5006/bokeh/objinfo/0257493b-cce5-450d-8036-2bc57233b1dc/bd1791f4-4d28-4faa-8c9d-a6fe5a1721c1. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://my_ip_address' is therefore not allowed access. The response had HTTP status code 500.
Whether I am running an applet or trying to embed a single plot.
Here I am trying to fetch a plot script from within a Flask view
#perf.route('/_fetch_heatmap', methods=['POST'])
#login_required
def fetch_sd_heatmap():
document = Document()
session = Session(root_url='http://127.0.0.1:5006', configdir=current_app.config['BASE_DIRECTORY'])
session.use_doc('sd_viz')
session.load_document(document)
...
plots = VBox(hm_duration, hm_frequency)
document.add(plots)
push(session, document)
script = autoload_server(plots, session)
return jsonify({'script': script})
This script is returned to an ajax call within my javascript. This script is then appended to the corresponding <div>
This runs fine on my development machine.
Below is my nginx configuration for production
server {
listen my_ip default_server;
charset utf-8;
client_max_body_size 30M;
location ~ ^/(app_config.py|.git) {
deny all;
return 404;
}
location / {
index index.html index.htm;
root /home/myuser/app_directory;
try_files $uri #app;
}
location /static {
alias /home/myuser/app_directory/webapp/static;
}
location #app {
include uwsgi_params;
uwsgi_pass unix:/home/myuser/app_directory/uwsgi.sock;
uwsgi_connect_timeout 18000;
...
}
Has anyone successfully made a flask application with embedded bokeh plots from the bokeh server that runs in a production environment?
Hi just to update this discussion, as of the new Bokeh server in 0.11 there is much more extensive documentation about deployments:
http://docs.bokeh.org/en/0.11.1/docs/user_guide/server.html
Including information about running behind reverse proxies, using load-balancers and process manager and automating with tools like Salt. The never server is much more robust, scalable, and easy to use. You can see a gallery of live Bokeh server examples that have been "production" deployed continuously since January 2016 here:
http://demo.bokeh.org
As a reference, the full automated deployment is available for study on GitHub:
https://github.com/bokeh/demo.bokeh.org
Additionally, a fairly sophisticated example of embedding a session-specific Bokeh server app is demonstrated in the "Happiness" example here:
https://github.com/bokeh/bokeh/tree/master/examples/embed/server_session
But lastly I should say that the upcoiming 0.12 release will have the ability to set a custom Jinja template for Bokeh apps, meaning that things like Single Page Apps that build heavily around Bokeh documents can be served directly from the Bokeh server, without needing to embed in another web-server (if that is desired).
Yep, I want it to work like in Flask framework - there I could set parameters like this:
static_folder=os.getcwd()+"/static/", static_url_path=""
and all the files that lies in ./static/files/blabla.bla would be accessible by mysite.com/files/blabla.bla address. I really don't want to add static after mysite.com.
But if I set STATIC_URL = '/' in Django then I could get my static files by this address, but suddenly I could not fetch my pages that described in urls.py.
I believe this is not supported out of the box. Off the top of my head, one way to do it would be with a special 404 handler that, having failed to match against any of the defined URLs, treats the request as a request for a static resource. This would be reasonably easy to do in the development environment but significantly more difficult when nginx, Apache, and/or gunicorn get involved.
In other words, don't do this. Nest your statics (or put them on a different sub domain) but don't mix the URL hierarchy in this way.
This is really easy to accomplish with Nginx using try_files. Using the pseudo-settings below will make Nginx try to find a static file first, and if it fails then execution is passed to your django app.
server {
...
root /some/path/to/assets/;
try_files $uri #django;
location #django {
...
proxy_pass http://unix:/some/path/to/server.sock;
}
}
Example: The file /some/path/to/assets/myfile.ext will be available as http://mydomain/myfile.ext
I'm trying to set a reasonable cache expiry for my JS files while in development. I have the standard setup, where HTML, CSS and JS are living under the static directory.
The docs do mention this, but for the life of me I cannot get this to work. I've tried both methods implied, first
class MyFlask(flask.Flask):
def get_send_file_max_age(self, name):
if name.lower().endswith('.js'):
return 60
return flask.Flask.get_send_file_max_age(self, name)
app = MyFlask(__name__)
and
app.config['SEND_FILE_MAX_AGE_DEFAULT'] = 60
Both have had no effect, my JS files under /static are still coming back with the default cache timeout,
Cache-Control: public, max-age=43200
Any pointers appreciated.
You may want to look at webassets to manage the cache expiry. It works in both development and production environment.
I had this problem and couldn't find an answer online that worked for me.
Then I realised that my static files are not being served from Flask at all! Flask only generates my HTML. The static files are served directly by my web server (Apache in my case, yours might be Nginx or something else).
Here are the instructions for Apache.
First install the mod_expires module:
sudo a2enmod expires
Then add something like this to your .htaccess file:
ExpiresByType text/css "access plus 1 year"
ExpiresByType application/javascript "access plus 1 year"
ExpiresByType image/* "access plus 1 year"
More details on how to configure it in the Apache manual.
I am developing on a python webserver (Tornado). I plan to place this in a production instance with nginx in front. This will be my first time placing something into a production environment on my own. My question is how to setup files/directories for static serving. For instance my application, allows users to upload photos to the web. I recieve the requests in Tornado, and save to disk. However when a user visits their items page, I would rather the images be pulled from a static server. My question is what is the best practice for getting the images from my dynamic server to the static server? Do I rsync the image directory to the static server, then run a cron that delete the images from the dynamic server?
Best practice is use shared storage, but if can't use it, than you can use "proxy_store" option from nginx. Example from nginx doc:
location /images/ {
root /data/www;
error_page 404 = #fetch;
}
location #fetch {
internal;
proxy_pass http://backend;
proxy_store on;
proxy_store_access user:rw group:rw all:r;
proxy_temp_path /data/temp;
root /data/www;
}
I can't, for the life of me, get Django to find my JavaScript files! I am trying to plug in a custom widget to the admin page, like so:
class MarkItUpWidget(forms.Textarea):
class Media:
js = (
'js/jquery.js',
'js/markitup/jquery.markitup.js',
'js/markitup/sets/markdown/set.js',
'js/markitup.init.js',
)
css = {
'screen': (
'js/markitup/skins/simple/style.css',
'js/markitup/sets/markdown/style.css',
)
}
However, when this code is inserted on my admin page, all of the links are broken, I can't get to any of my JavaScript files. My settings file looks like this: http://pastebin.com/qFVqUXyG
Is there something I am doing wrong? I'm somewhat new to Django.
I guess you're using django-admin runserver to test your website. In that case, have a look at "How to serve static files" (but don't ignore the big fat disclaimer).
Once you're ready to deploy, this chapter contains all the information (provided you go the standard route of Apache/mod_wsgi)
I don't know what the "django way" to include js files is, but I just use a simple regular include in the template, its great since you can dynamically generate the location / filename for these.. oh and if you are using django's test server I don't know how to get it to recognize these or if it even can but all you need is to run an apache server on your local machine and then put the files in the localhost directory and you can include them through localhost by including their full path in the template (i.e., http://localhost/myfiledirectory/myfile.js), also something I do is use amazon s3 to host files since you then get a url for them on there (not that you asked about this but its a quick way to host files if you don't have apache running locally)
Basically, my understanding of Django is that it works differently than a PHP framework, for example, as the files for Django don't sit (or don't need to at least) in the web path (i.e. they do not need to be accessible through the host path but can be anywhere on the machine) this is good for providing extra security, etc but it also means, that to give files a url to download them they need to be in the web hosting path, others may know how to make django do this directly but my quick and dirty method of putting them on the localhost path will work if thats all you're after.