I am developping a django app on Windows, SQLite and the django dev server . I have deployed it to my host server which is running Linux, Apache, FastCgi, MySQL.
Unfortunately, I have an error returned by the server on the prod while everything ok on the dev machine. I've asked my provider for a pre-production solution in order to be able to debug and understand the problem.
Anyway, what are according to you the most likely errors that can happen when moving a django app from dev to prod?
Best
Update: I think that a pre-prod is the best way to address this kind of problem. But I would like to build a check list of what must be done before to put in production.
Thanks for the very valuable answers that I received until now :)
Update: FYI, I 've implemented the preprod server and the email notification as suggested by shanyu and I can see that the error comes from the smart_if templatetag that I am using on this new version. Any trick with template tags?
Update: I think I've fixed the pb which was caused I think by the Filezilla FTP sending. I was using the "replace if newer" option which I guess is causing some unexpected results. Using the "replace all" option fix the issue. However, it was an opportunity for me to learn more about deployment. Thansk for your answers.
Problems I typically have include:
Misconfigured productions settings, whether in my production localsettings.py, wsgi/cgi, or apache site files in /etc/sites-available
Database differences. I use South for migrations and have run into some subtle issues when performing my migration on PostgreSQL when it worked smoothly in sqlite.
Static file hosting since I cheat and use the Django server in development
Permissions, both on the file system and within the database
Rare, but possible, network issues preventing me from getting my dependencies, whether on PyPi or some 3rd party site
Ways that I have mitigated these issues:
Use the same database in production and development (in your case, MySQL everywhere)
I've found it is useful to have a "test" environment which mimics production in every way possible (it can be on lower end hardware, or even the same machine). This way, if there are any issues in this "production-like" enivornment, I can solve them without taking my production server offline.
Script everything for repeatable deployments. I use fabric, but zc.buildout or Paver would also work. These tools help reduce typos while deploying and reduce the time to deploy my app.
Use version control (mercurial, git, subversion) and a schema migration tool (like South), so if something does go wrong when you deploy to production, you have the possibility of backing out the changes and allowing production to run on the old code with the old database schema.
I haven't set up an "egg proxy" yet, but I am considering it, to avoid issues when downloading dependencies.
I've found pip's freezing dependencies to be useful, in case a new, incompatible change to a library occurred since I downloaded it initially
Use a web testing framework like Windmill or Selenium to test my application in my "test" environment, so that I can get a lot of test coverage of my system very quickly.
Regarding your case, I can think of 2 simple things that may help you:
You can enable Django to send messages when exceptions occur giving details about them. Look at here for details.
You'll be better off if you set up a test environment on the prod server (say, test.example.com) so that you can check if things will go smoothly or not before you deploy the app.
I believe these were the podcasts I listened to recently (from Pycon 2009):
Locate Django in the Real World (PyCon 2009):
http://advocacy.python.org/podcasts/pycon.rss
Parts 1 to 3
Very good introduction to designing your apps for deployment, in particular for reuse and redeployment.
Regs.
Related
I'm a newbie in django and have a project that involves distributed remote storage and I'm suggested to use mod x-sendfile as part of the project procedure.
In have a django app that receives a file and transforms it into N segments each to be stored on a distinct server. Those servers having a django app receiving and storing the segments.
But since mod x-sendfile works need apache and I am just in developing and trying stage this question occurred to me.
I googled a lot but found nothing in that regard.
So my question being: Is it possible to use apache as django web server during the development of django apps? Does it make sense in development mode to replace apache with django built-in web server?
There should be nothing stopping you from installing a copy of Apache on your workstation and using it for developing, and since you're working on something that depends on some of that functionality it makes perfect sense for you to use that for your development server instead of ./manage.py runserver.
Most people use Djangos built-in server because they don't need more than that for what they're trying to do - it sounds like your solution does.
Heck, since you're testing distributed you may even want to consider grabbing a virtualization tool (qemu, virtualbox, et al) so you can have a faux-distributed setup to work with (I'd suggest doing a bit of scripting to make it easy to deploy / restart them all at once though - it'll save you from having to track down issues where the code that's running is older than you thought it was).
Your development environment can be what you need it to be for what you're doing.
As a Django / Python newbie, should I try to debug on a server running 4 year old software versions, try to recreate the old software installations on my local, or just try to run the software in current version of Django/Python/Postgres/PostGIS on my local Mac OS X 10.9.5?
Background:
On a project where I was supposed to just load data into Postgres/PostGIS, I need to debug why a 2010 year old Django / Postgres / Postgis project is getting an error. I'm a LAMP developer who's never used Django or done much in Python, but I've been able to get a staging site working on the server, and make one or two changes. I thought it would make sense to debug locally on my Mac OS X 10.9.5. So I've used homebrew to install Django 1.7 and Postgres 9.3. Looking at the version differences, I'm worried it will be a more of a hassle now to try to migrate and upgrade the project than to attempt to debug it on the staging site instance running on the server.
FWIW, I know the lines of code that I'd like to investigate (seems like maybe an object is not getting loaded properly from db, since it is in the db), but I'm not positive what sort of echo statements to use. I like using IDE's to inspect things. The project is a bit of an orphan, as the first professional project of a developer who is no longer available to help. And of course, the deadline is last week. :(
Differences between your production and development environments can cause a myriad of headaches.
I recommend you use a tool such as Vagrant to set up a development environment running inside of a virtual machine that mirrors your production server.
Use VirtualEnv to emulate the necessary Django version. PostgreSQL is trickier, in theory you can have a second instance with the required version running simultaneously, but that can also cause very subtle conflicts. It would be better to have it running on another machine (virtual or physical) and access it through your local network.
The simplest way I think is to look at using unittest and mock object to set up some unit tests on the functions that you suspect are the cause of the problem. By using unittest and mock objects, you can control how the existing code interacts with Django and Postgres objects and allow for version differences by setting the expected return values.
With mock object, you can mock all or just part of an existing Python object, which reduces the dependencies you require for your development environment. Depending on how the code is structured, you might not need to install either Django or Postgres at all or a webserver for that matter. This blog explains Mock object in detail.
Even though you're pressed for time, you could do worse than setting up unittests for the whole project, future developers will thank you.
In terms of debugging, I personally can't reccomend pudb enough, it's an interactive command line debugger which you can use with unittest to zero in on what part of the code is causing the problem.
If you do need to install Django and Postgres, I would suggest looking at virtualenv which allows you to set up a virtual environment for Python. That way you can just install the specific dependencies you need without interfering with your global system wide installation. You can also install earlier versions of packages which would do the trick to emulate the existing system's state.
I need to write a very light database (sqlite is fine) app that will initially be run locally on a clients windows PC but could, should it ever be necessary, be upgraded to work over the public interwebs without a complete rewrite.
My end user is not very technically inclined and I'd like to keep things as simple as posible. To that end I really want to avoid having to install a local webserver, however "easy" that may seem to you or I. Django specifically warns not to use it's inbuilt webserver in production so my two options seem to be...
a) Use django's built in server anyway while the app is running locally on windows and, if it ever needs to be upgraded to work over the net just stick it behind apache on a linux box somewhere in the cloud.
b) Use a framework that has a more robust built in web server from the start.
My understanding is that the only two disadvantages of django's built in server are a lack of security testing (moot if running only locally) and it's single threaded nature (not likely to be a big deal either for a low/zero concurrency single user app running locally). Am I way off base?
If so, then can I get some other "full stack" framework recommendations please - I strongly prefer python but I'm open to PHP and ruby based solutions too if there's no clear python winner. I'm probably going to have to support this app for a decade or more so I'd rather not use anything too new or esoteric unless it's from developers with some serious pedigree.
Thanks for your advice :)
Roger
I find Django's admin very easy to use for non-technical clients. In fact, that is the major consideration for my using Django as of late. Once set up properly, non-technical people can very easily update information, which can reflected on the front end immediately.
The client feels empowered.
Use Django. It's very simple for you to get started. Also, they have the best documentation. Follow the step by step app creating tutorial. Django supports all the databases that exist. Also, the built in server is very simple to use for the development and production server. I would highly recommend Django.
What I want to know are actually 2 things.
Part 1:
I have a developed source code for an application in python that uses Django framework.
What I want is to be able to run the code on a developer machine and see the result. What do I need for it?
(my guesses):
Python development enironment (Eclipse/PyDev/Aptana Studio 3 seem to be the better ones for windows not sure linux yet),
I also have a postgre database already setup (I know there's a file where I have to specify connection information)
- something installed from django or would this be already included in the code that I have?
Part II:
I also want to make a dev server accessible through internet.
- this is the major part of the question and the most important. How do I publish the app?
- I have a linux machine that I would do this on, but unsure of all the things I need. Apache server?
To answer your questions:
What you need: A list of requirements and instructions to get started with Django is available here: http://djangobook.com/en/2.0/chapter02/.
Database: that chapter also includes a section on configuring access to your database, with a specific section on postgreSQL.
Dev server: To start a basic development server, see this tutorial section
Deploying django (production): For instructions on how to deploy Django for production, see chapter on deploying Django.
Publishing on internet: as for making your dev server accessible through the internet, ask on https://serverfault.com/. Make sure you provide more information about your network setup, what you've tried, what isn't working, etc. (Briefly, you need to make sure that the host you are running your server on is on a publicly accessible IP, or has port 80 forwarded to it from such a host. If in doubt, speak to your sys/network admin if you have one. Or use a django hosting service such as those listed on http://djangohosting.com)
IDE : Regarding IDE, it's down to personal preference. What you mentioned are fine and can run on Linux too.
As a first step, I suggest you follow the tutorial which guides you through the process of starting a development server and developing a basic app.
Even if your goal is to deploy an existing app, the tutorial will give you an idea of how the different components work together (apps, models, urls, templates, etc) which will help with debugging when something goes wrong with your deployment.
Good luck.
You need Python, Django, a WSGI container (e.g. mod_wsgi, uWSGI, Paste Deploy), and a database server. You make the Django project available as a WSGI app, bound to the appropriate interface on the machine.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I'm about to deploy a mediumsized site powered by Django. I have a dedicated Ubuntu Server.
I'm really confused over which serversoftware to use. So i thought to myself: why not ask stackoverflow.
What i'm looking for is:
Easy to set up
Fast and easy on resources
Can serve mediafiles
Able to serve multiple djangosites on same server
I would rather not install PHP or anything else that sucks resources, and for which I have no use for.
I have heard of mod_wsgi and mod_python on Apache, nginx and lighty. Which are the pros and cons of these and have i missed someone?
#Barry: Somehow i feel like Apache is to bloated for me. What about the alternatives?
#BrianLy: Ok I'll check out mod_wsgi some more. But why do i need Apache if i serve static files with lighty? I have also managed to serve the django app itself with lighty. Is that bad in anyway? Sorry for beeing so stupid :-)
UPDATE: What about lighty and nginx - which are the uses-cases when these are the perfect choice?
Since I was looking for some more in-depth answers, I decided to research the issue myself in depth. Please let me know if I've misunderstood anything.
Some general recommendation are to use a separate webserver for handling media. By separate, I mean a webserver which is not running Django. This server can be for instance:
Lighttpd (Lighty)
Nginx (EngineX)
Or some other light-weight server
Then, for Django, you can go down different paths. You can either:
Serve Django via Apache and:
mod_python
This is the stable and recommended/well documented way. Cons: uses a lot of memory.
mod_wsgi
From what I understand, mod_wsgi is a newer alternative. It appears to be faster and easier on resources.
mod_fastcgi
When using FastCGI you are delegating the serving of Django to another process. Since mod_python includes a python interpreter in every request it uses a lot of memory. This is a way to bypass that problem. Also there is some security concerns.
What you do is that you start your Django FastCGI server in a separate process and then configures apache via rewrites to call this process when needed.
Or you can:
Serve Django without using Apache but with another server that supports FastCGI natively:
(The documentation mentions that you can do this if you don't have any Apache specific needs. I guess the reason must be to save memory.)
Lighttpd
This is the server that runs Youtube. It seems fast and easy to use, however i've seen reports on memoryleaks.
nginx
I've seen benchmarks claiming that this server is even faster than lighttpd. It's mostly documented in Russian though.
Another thing, due to limitations in Python your server should be running in forked mode, not threaded.
So this is my current research, but I want more opinions and experiences.
I'm using Cherokee.
According to their benchmarks (grain of salt with them), it handles load better than both Lighttpd and nginx... But that's not why I use it.
I use it because if you type cherokee-admin, it starts a new server that you can log into (with a one-time password) and configure the whole server through a beautifully-done webmin. That's a killer feature. It has already saved me a lot of time. And it's saving my server a lot of resources!
As for django, I'm running it as a threaded SCGI process. Works well. Cherokee can keep it running too. Again, very nice feature.
The current Ubuntu repo version is very old so I'd advise you use their PPA. Good luck.
As #Barry said, the documentation uses mod_python. I haven't used Ubuntu as a server, but had a good experience with mod_wsgi on Solaris. You can find documentation for mod_wsgi and Django on the mod_wsgi site.
A quick review of your requirements:
Easy to setup I've found apache 2.2 fairly easy to build and install.
Fast and easy on resources I would say that this depends on your usage and traffic. * You may not want to server all files using Apache and use LightTPD (lighty) to server static files.
Can serve media files I assume you mean images, flash files? Apache can do this.
Multiple sites on same server Virtual server hosting on Apache.
Rather not install other extensions Comment out anything you don't want in the Apache config.
The officially recommended way to deploy a django project is to use mod_python with apache. This is described in the documentation. The main pro with this is that it is the best documented, most supported, and most common way to deploy. The con is that it probably isn't the fastest.
The best configuration is not so known I think. But here is:
Use nginx for serving requests (dynamic to app, static content directly).
Use python web server for serving dynamic content.
Two most speedy solutions for python-based web server is:
cogen
fapws2
You need to look into google to find current best configuration for django (still in development).
I’m using nginx (0.6.32 taken from Sid) with mod_wsgi. It works very well, though I can’t say whether it’s better than the alternatives because I never tried any. Nginx has memcached support built in, which can perhaps interoperate with the Django caching middleware (I don’t actually use it, instead I fill the cache manually using python-memcache and invalidate it when changes are made), so cache hits completely bypass Django (my development machine can serve about 3000 requests per second).
A caveat: nginx’ mod_wsgi highly dislikes named locations (it tries to pass them in SCRIPT_NAME), so the obvious ‘error_page 404 = #django’ will cause numerous obscure errors. I had to patch mod_wsgi source to fix that.
I'm struggling to understand all the options as well. In this blog post I found some benefits of mod_wsgi compared to mod_python explained.
Multiple low-traffic sites on a small VPS make RAM consumption the primary concern, and mod_python seems like a bad option there. Using lighttpd and FastCGI, I've managed to get the minimum memory usage of a simple Django site down to 58MiB virtual and 6.5MiB resident (after restarting and serving a single non-RAM-heavy request).
I've noticed that upgrading from Python 2.4 to 2.5 on Debian Etch increased the minimum memory footprint of the Python processes by a few percent. On the other hand, 2.5's better memory management might have a bigger opposite effect on long-running processes.
Keep it simple: Django recommends Apache and mod_wsgi (or mod_python). If serving media files is a very big part of your service, consider Amazon S3 or Rackspace CloudFiles.
There are many ways, approach to do this.For that reason, I recommend to read carefully the article related to the deployment process on DjangoAdvent.com:
Eric Florenzano - Deploying Django with FastCGI: http://djangoadvent.com/1.2/deploying-django-site-using-fastcgi/
Read too:
Mike Malone - Scaling Django
Stochastictechnologies Blog: The perfect Django Setup
Mikkel Hoegh Blog: 35 % Response-time-improvement-switching-uwsgi-nginx
Regards
In my opinion best/fastest stack is varnish-nginx-uwsgi-django.
And I'm successfully using it.
If you're using lighthttpd, you can also use FastCGI for serving Django. I'm not sure how the speed compares to mod_wsgi, but if memory serves correctly, you get a couple of the benefits that you would get with mod_wsgi that you wouldn't get with mod_python. The main one being that you can give each application its own process (which is really helpful for keeping memory of different apps separated as well as for taking advantage of multi-core computers.
Edit: Just to add in regards to your update about nginix, if memory serves correctly again, nginix uses "greenlets" to handle concurrency. This means that you may need to be a little bit more careful to make sure that one app doesn't eat up all the server's time.
We use nginx and FastCGI for all of our Django deployments. This is mostly because we usually deploy over at Slicehost, and don't want to donate all of our memory to Apache. I guess this would be our "use case".
As for the remarks about the documentation being mostly in Russian -- I've found most of the information on the English wiki to be very useful and accurate. This site has sample configurations for Django too, from which you can tweak your own nginx configuration.
I have a warning for using Cherokee. When you make changes to Django Cherokee maintains the OLD process, instead of killing it and starting a new one.
On Apache i strongly recommend this article.
http://www.djangofoo.com/17/django-mod_wsgi-deploy-exampl
Its easy to setup, easy to kill or reset after making changes.
Just type in terminal
sudo /etc/init.d/apache2 restart
and changes are seen instantly.