which would you recommend?
which is faster, reliable?
apache mod_python or nginx/lighttpd FastCGI?
I've done both, and Apache/mod_python tended to be easier to work with and more stable. But these days I've jumped over to Apache/mod_wsgi, which is everything I've ever wanted and more:
Easy management of daemon processes.
As a result, much better process isolation (running multiple sites in the same Apache config with mod_python almost always ends in trouble -- environment variables and C extensions leak across sites when you do that).
Easy code reloads (set it up right and you can just touch the .wsgi file to reload instead of restarting Apache).
More predictable resource usage. With mod_python, a given Apache child process' memory use can jump around a lot. With mod_wsgi it's pretty stable: once everything's loaded, you know that's how much memory it'll use.
lighttpd with FastCGI will be nominally faster, but really the time it takes to run your python code and any database hits it does is going to absolutely dwarf any performance benefit you get between web servers.
mod_python and apache will give you a bit more flexibility feature-wise if you want to write code outside of django that does stuff like digest auth, or any fancy HTTP header getting/setting. Perhaps you want to use other builtin features of apache such as mod_rewrite.
If memory is a concern, staying away form apache/mod_python will help a lot. Apache tends to use a lot of RAM, and the mod_python code that glues into all of the apache functionality occupies a lot of memory-space as well. Not to mention the multiprocess nature of apache tends to eat up more RAM, as each process grows to the size of it's most intensive request.
Nginx with mod_wsgi
I'm using it with nginx. not sure if it's really faster, but certainly less RAM/CPU load. Also it's easier to run several Django processes and have nginx map each URL prefix to a different socket. still not taking full advantage of nginx's memcached module, but first tests show huge speed advantage.
There's also mod_wsgi, it seems to be faster than mod_python and the daemon mode operates similar to FastCGI
Personally I've had it working with FastCGI for some time now (6 months or so) and the response times 'seem' quicker when loading a page that way vs mod___python. The critical reason for me though is that I couldn't see an obvious way to do multiple sites from the same apache / mod_python install whereas FastCGI was a relative no-brainer.
I've not conducted any particularly thorough experiments though :-)
[Edit] Speaking from experience though, setting up FastCGI can be a bit of a pain the first time around. I keep meaning to write a guide..!
I'd recommend WSGI configurations; I keep meaning to ditch apache, but there is always some legacy app on the server that seems to require it. Additionally, the WSGI app ecology is very diverse, and it allows neat tricks such as daisy-chaining WSGI "middleware" between the server and the app.
However, there are currently known issues with some apps and apache mod_wsgi, particularly some ctypes apps, so be wary if you are trying to run, say, geodjango which uses ctypes extensively. I'm currently working around those issues by going back to fastcgi myself.
Related
I've made a small web app using web.py that I now want to put into production. I dont anticipate this will have very high concurrent use (probably max of 5 or so users at any given time, if that).
That said, I dont want to go with the cherry.py server that comes with web.py (and which i have been using for debugging), because one of my main motivations for the app was to learn how to properly put apps in production.
Reading up on options - there seems to be dizzying array of stuff. Tornoado, nginx, lighttpd etc etc. Also stuff like Gunicorn, which I cant quite grasp the use of so far.
It seems WSGI is the way to go - and I wanted some help with what would be an appropriate, relatively easy to administer setup that i can run on an EC2 instance (ubuntu), perhaps using nginx/wsgi. Specifically, do i need gunicorn (or equivalent), and are there any good intros anybody may know of so i can actually get my web.py code running and at least start to understand this jigsaw of various technologies/options?
Many thanks
Available options are:
apache + mod_python
apache + mod_wsgi
lighttpd + mod_fastcgi
lighttpd + gunicorn
nginx + gunicorn
I suggest you to go for gunicorn.
CherryPy is a pretty good choice for deployment. It's a good WSGI server, and is known to work on EC2. Straightforward mapping of HTTP requests onto your python code. I have run it behind Apache, behind lighttpd, and by itself.
I suggest you use Apache + modpython. Even though you expect less load, it's always good to be prepared :)
Plus this is a tried and tested setup.
We are deploying django application, I found in the documentation that it is recommended to use WSGI appoach for doing that.
Before deploying I wanted to know, why it is recommended over other two approaches i.e. using mod_python and fastcgi...
Thanks a lot.
wsgi is usually preferred because it decouples your choice of framework from your choice of web server: if tomorrow you want to move, say, from Apache to nginx, or whatever, the move is trivially easy with wsgi, not so easy otherwise.
Furthermore, using wsgi affords you the option to add some middleware that's framework-independent, rather than having to rely on every possible functionality you want having already been implemented and made available for your framework of choice.
We tried mod_python. It's slower and harder to configure. It doesn't offer the daemon feature.
We couldn't get fast_cgi built for our combination of Apache, Red Hat and Python. I'm not sure specifically what was wrong, but we couldn't get it built properly. It wouldn't dispatch requests to Django properly, and we couldn't diagnose the problem.
We tried mod_wsgi third. It built nicely. It has the daemon option. It's very easy to configure. It allows trivial restart of the Django applications without restarting all of Apache.
I use mod_wsgi for any production Django app. It's fast, stable, and very configurable.
You may also want to look in to the FastCGI method a bit more. Eric Florenzano just did a great write up of Django with FastCGI for the Django Advent: http://djangoadvent.com/1.2/deploying-django-site-using-fastcgi/
We currently run a small shared hosting service for a couple of hundred small PHP sites on our servers. We'd like to offer Python support too, but from our initial research at least, a server restart seems to be required after each source code change.
Is this really the case? If so, we're just not going to be able to offer Python hosting support. Giving our clients the ability to upload files is easy, but we can't have them restart the (shared) server process!
PHP is easy -- you upload a new version of a file, the new version is run.
I've a lot of respect for the Python language and community, so find it hard to believe that it really requires such a crazy process to update a site's code. Please tell me I'm wrong! :-)
Python is a compiled language; the compiled byte code is cached by the Python process for later use, to improve performance. PHP, by default, is interpreted. It's a tradeoff between usability and speed.
If you're using a standard WSGI module, such as Apache's mod_wsgi, then you don't have to restart the server -- just touch the .wsgi file and the code will be reloaded. If you're using some weird server which doesn't support WSGI, you're sort of on your own usability-wise.
Depends on how you deploy the Python application. If it is as a pure Python CGI script, no restarts are necessary (not advised at all though, because it will be super slow). If you are using modwsgi in Apache, there are valid ways of reloading the source. modpython apparently has some support and accompanying issues for module reloading.
There are ways other than Apache to host Python application, including the CherryPy server, Paste Server, Zope, Twisted, and Tornado.
However, unless you have a specific reason not to use it (an since you are coming from presumably an Apache/PHP shop), I would highly recommed mod_wsgi on Apache. I know that Django recommends modwsgi on Apache and most of the other major Python frameworks will work on modwsgi.
Is this really the case?
It Depends. Code reloading is highly specific to the hosting solution. Most servers provide some way to automatically reload the WSGI script itself, but there's no standardisation; indeed, the question of how a WSGI Application object is connected to a web server at all differs widely across varying hosting environments. (You can just about make a single script file that works as deployment glue for CGI, mod_wsgi, passenger and ISAPI_WSGI, but it's not wholly trivial.)
What Python really struggles with, though, is module reloading. Which is problematic for WSGI applications because any non-trivial webapp will be encapsulating its functionality into modules and packages rather than simple standalone scripts. It turns out reloading modules is quite tricky, because if you reload() them one by one they can easily end up with bad references to old versions. Ideally the way forward would be to reload the whole Python interpreter when any file is updated, but in practice it seems some C extensions seem not to like this so it isn't generally done.
There are workarounds to reload a group of modules at once which can reliably update an application when one of its modules is touched. I use a deployment module that does this (which I haven't got around to publishing, but can chuck you a copy if you're interested) and it works great for my own webapps. But you do need a little discipline to make sure you don't accidentally start leaving references to your old modules' objects in other modules you aren't reloading; if you're talking loads of sites written by third parties whose code may be leaky, this might not be ideal.
In that case you might want to look at something like running mod_wsgi in daemon mode with an application group for each party and process-level reloading, and touch the WSGI script file when you've updated any of the modules.
You're right to complain; this (and many other WSGI deployment issues) could do with some standardisation help.
Or should I be using a totally different server?
Nginx with mod_wsgi requires the use of a non-blocking asynchronous framework and setup and isn't likely to work out of box with Pylons.
I usually go with the proxy route to a stand-alone Pylons process using the PasteScript#cherrypy WSGI server (as its higher performing than the Paste#http one, though it won't recycle threads if you have leaks...).
If you're set on using Apache and its your server (so you can compile and run Apache mod_wsgi), I'd suggest using that setup as its less maintenance to effectively utilize multiple cores. With a proxy setup, you'd have to use the mod_proxy_balancer with multiple paste processes to effectively utilize multiple cores/cpus.
If you're deploying to someone else's Apache (shared hosting), mod_proxy is generally the easier solution as its stock in Apache 2.2 and above.
Personally, I usually deploy with nginx + proxy to multiple paster processes.
I've also used mod_fastcgi + flup to great success several times now. There are a couple of recipes floating around for setting this up, but unfortunately it will probably still require some tweaking on your part to get everything working:
http://wiki.pylonshq.com/display/pylonscookbook/Production+Deployment+Using+Apache,+FastCGI+and+mod_rewrite
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I'm about to deploy a mediumsized site powered by Django. I have a dedicated Ubuntu Server.
I'm really confused over which serversoftware to use. So i thought to myself: why not ask stackoverflow.
What i'm looking for is:
Easy to set up
Fast and easy on resources
Can serve mediafiles
Able to serve multiple djangosites on same server
I would rather not install PHP or anything else that sucks resources, and for which I have no use for.
I have heard of mod_wsgi and mod_python on Apache, nginx and lighty. Which are the pros and cons of these and have i missed someone?
#Barry: Somehow i feel like Apache is to bloated for me. What about the alternatives?
#BrianLy: Ok I'll check out mod_wsgi some more. But why do i need Apache if i serve static files with lighty? I have also managed to serve the django app itself with lighty. Is that bad in anyway? Sorry for beeing so stupid :-)
UPDATE: What about lighty and nginx - which are the uses-cases when these are the perfect choice?
Since I was looking for some more in-depth answers, I decided to research the issue myself in depth. Please let me know if I've misunderstood anything.
Some general recommendation are to use a separate webserver for handling media. By separate, I mean a webserver which is not running Django. This server can be for instance:
Lighttpd (Lighty)
Nginx (EngineX)
Or some other light-weight server
Then, for Django, you can go down different paths. You can either:
Serve Django via Apache and:
mod_python
This is the stable and recommended/well documented way. Cons: uses a lot of memory.
mod_wsgi
From what I understand, mod_wsgi is a newer alternative. It appears to be faster and easier on resources.
mod_fastcgi
When using FastCGI you are delegating the serving of Django to another process. Since mod_python includes a python interpreter in every request it uses a lot of memory. This is a way to bypass that problem. Also there is some security concerns.
What you do is that you start your Django FastCGI server in a separate process and then configures apache via rewrites to call this process when needed.
Or you can:
Serve Django without using Apache but with another server that supports FastCGI natively:
(The documentation mentions that you can do this if you don't have any Apache specific needs. I guess the reason must be to save memory.)
Lighttpd
This is the server that runs Youtube. It seems fast and easy to use, however i've seen reports on memoryleaks.
nginx
I've seen benchmarks claiming that this server is even faster than lighttpd. It's mostly documented in Russian though.
Another thing, due to limitations in Python your server should be running in forked mode, not threaded.
So this is my current research, but I want more opinions and experiences.
I'm using Cherokee.
According to their benchmarks (grain of salt with them), it handles load better than both Lighttpd and nginx... But that's not why I use it.
I use it because if you type cherokee-admin, it starts a new server that you can log into (with a one-time password) and configure the whole server through a beautifully-done webmin. That's a killer feature. It has already saved me a lot of time. And it's saving my server a lot of resources!
As for django, I'm running it as a threaded SCGI process. Works well. Cherokee can keep it running too. Again, very nice feature.
The current Ubuntu repo version is very old so I'd advise you use their PPA. Good luck.
As #Barry said, the documentation uses mod_python. I haven't used Ubuntu as a server, but had a good experience with mod_wsgi on Solaris. You can find documentation for mod_wsgi and Django on the mod_wsgi site.
A quick review of your requirements:
Easy to setup I've found apache 2.2 fairly easy to build and install.
Fast and easy on resources I would say that this depends on your usage and traffic. * You may not want to server all files using Apache and use LightTPD (lighty) to server static files.
Can serve media files I assume you mean images, flash files? Apache can do this.
Multiple sites on same server Virtual server hosting on Apache.
Rather not install other extensions Comment out anything you don't want in the Apache config.
The officially recommended way to deploy a django project is to use mod_python with apache. This is described in the documentation. The main pro with this is that it is the best documented, most supported, and most common way to deploy. The con is that it probably isn't the fastest.
The best configuration is not so known I think. But here is:
Use nginx for serving requests (dynamic to app, static content directly).
Use python web server for serving dynamic content.
Two most speedy solutions for python-based web server is:
cogen
fapws2
You need to look into google to find current best configuration for django (still in development).
I’m using nginx (0.6.32 taken from Sid) with mod_wsgi. It works very well, though I can’t say whether it’s better than the alternatives because I never tried any. Nginx has memcached support built in, which can perhaps interoperate with the Django caching middleware (I don’t actually use it, instead I fill the cache manually using python-memcache and invalidate it when changes are made), so cache hits completely bypass Django (my development machine can serve about 3000 requests per second).
A caveat: nginx’ mod_wsgi highly dislikes named locations (it tries to pass them in SCRIPT_NAME), so the obvious ‘error_page 404 = #django’ will cause numerous obscure errors. I had to patch mod_wsgi source to fix that.
I'm struggling to understand all the options as well. In this blog post I found some benefits of mod_wsgi compared to mod_python explained.
Multiple low-traffic sites on a small VPS make RAM consumption the primary concern, and mod_python seems like a bad option there. Using lighttpd and FastCGI, I've managed to get the minimum memory usage of a simple Django site down to 58MiB virtual and 6.5MiB resident (after restarting and serving a single non-RAM-heavy request).
I've noticed that upgrading from Python 2.4 to 2.5 on Debian Etch increased the minimum memory footprint of the Python processes by a few percent. On the other hand, 2.5's better memory management might have a bigger opposite effect on long-running processes.
Keep it simple: Django recommends Apache and mod_wsgi (or mod_python). If serving media files is a very big part of your service, consider Amazon S3 or Rackspace CloudFiles.
There are many ways, approach to do this.For that reason, I recommend to read carefully the article related to the deployment process on DjangoAdvent.com:
Eric Florenzano - Deploying Django with FastCGI: http://djangoadvent.com/1.2/deploying-django-site-using-fastcgi/
Read too:
Mike Malone - Scaling Django
Stochastictechnologies Blog: The perfect Django Setup
Mikkel Hoegh Blog: 35 % Response-time-improvement-switching-uwsgi-nginx
Regards
In my opinion best/fastest stack is varnish-nginx-uwsgi-django.
And I'm successfully using it.
If you're using lighthttpd, you can also use FastCGI for serving Django. I'm not sure how the speed compares to mod_wsgi, but if memory serves correctly, you get a couple of the benefits that you would get with mod_wsgi that you wouldn't get with mod_python. The main one being that you can give each application its own process (which is really helpful for keeping memory of different apps separated as well as for taking advantage of multi-core computers.
Edit: Just to add in regards to your update about nginix, if memory serves correctly again, nginix uses "greenlets" to handle concurrency. This means that you may need to be a little bit more careful to make sure that one app doesn't eat up all the server's time.
We use nginx and FastCGI for all of our Django deployments. This is mostly because we usually deploy over at Slicehost, and don't want to donate all of our memory to Apache. I guess this would be our "use case".
As for the remarks about the documentation being mostly in Russian -- I've found most of the information on the English wiki to be very useful and accurate. This site has sample configurations for Django too, from which you can tweak your own nginx configuration.
I have a warning for using Cherokee. When you make changes to Django Cherokee maintains the OLD process, instead of killing it and starting a new one.
On Apache i strongly recommend this article.
http://www.djangofoo.com/17/django-mod_wsgi-deploy-exampl
Its easy to setup, easy to kill or reset after making changes.
Just type in terminal
sudo /etc/init.d/apache2 restart
and changes are seen instantly.