I'm debugging a really weird problem with a mod_wsgi-deployed application resulting in Django/Apache (not yet known which) giving status 500 errors to some users instead of the correct 404. I want to exclude Apache from the debugging environment to determine which part of the setup is at fault and send my requests manually to the WSGI handler.
I suspect it's as easy as setting the environment and running python wsgi_handler.py, but is this correct? What should the enviroment contain additionally? Any pointers to existing documentation?
it's as easy as setting the environment and running python wsgi_handler.py,
Correct.
What should the enviroment contain additionally? Any pointers to existing documentation?
Did you read this? http://docs.python.org/library/wsgiref.html
You can easily run a web server from your desktop for testing. It's a very few lines of code.
http://docs.python.org/library/wsgiref.html#wsgiref.simple_server.make_server
Related
I started to write a small REST API using Python with Falcon and Gunicorn. I would like to write some integration tests and I am not sure how to set up a proper test environment (for example to switch to another database). Do you have some good advice or tutorials?
My current idea is to maybe introduce some middleware and to provide a header. If the header is set, I could switch to my test configuration.
Definitely don't add middleware for the sole purpose of integration testing. What you should do is set up some configuration files for your server to use. Dev, Test, and Prod is a decent setup. Each file can point to a different database and have different ports for your server. I'm sure you will even be able to have Dev and Test servers up and running at the same time on your personal computer without any issues. Python has a build in config module that you can use. You can set environment variables in your shell so your server knows which configuration file to use. E.G. in bash FALCON_ENV='DEV' Then in python you can use the os module to get the environment variable - os.environ['FALCON_ENV']. Hope that helps, feel free to ask any more questions.
You might want try using the virtual testing environment and testing helpers provided by falcon core:
http://falcon.readthedocs.io/en/stable/api/testing.html
I have experience with PHP but now have to manage a python application. It is already developed and running on live server. On live server we have beta.domain.com as well. which has its own copy of same source, other than the main domain application.
Now, when we print something within template it effects but before that, following MVC, if we try to print or use sys.exit() in manage.py or later imported "settings.py" or then views.py, nothing effects these files at all. After any change in just mentioned files, Website still renders everything and display related template.
Another thing that if even we remove
return render_to_response('home.html', RequestContext(request, context))
inside the views.py for testing, nothing effects and website still gets rendered with template.
Do I need to think that if website code is already hosted then in order to have changes to reflect new one in the code in any file (.py), needs to be reinitialized by executing any project related file?
I am not sure how python code needs to be updated at all, any quick help is much appreciated.
Anytime you change your code, you need to restart Apache server.
<path to apache>/bin/apachectl restart
The python interpreter of the process has already loaded your python modules in previous web requests. And once the module is loaded, it is stored in memory. Next time when a request comes, the Python interpreter will simply use the version of the module that is already loaded in memory. So your changed code will not be picked up.
Please, please, please don't try and edit files on your live server. You're only going to get yourself into terrible difficulties. Use a local copy for development, preferably cloned via a version control system, and deploy at regular intervals rather than on every change.
To answer your question, you usually need to restart the WSGI process in order to see Python code changes. An easy way to do this is to touch the .wsgi file, at which point mod_wsgi will detect that it has changed, and reload everything. Otherwise you can simply reload Apache.
Unfortunately I don't understand your references to manage.py or sys.exit.
The solution is excatly restarting the apache server as answered by Sudipta .Each time the py files are interpretted .pvc files are created for each corresponding .py files.When the Apache server is restarted all the python source files are interprretted again and new .pvc files are generated and only these files will run till the apache restarts again.
I am developping a django app on Windows, SQLite and the django dev server . I have deployed it to my host server which is running Linux, Apache, FastCgi, MySQL.
Unfortunately, I have an error returned by the server on the prod while everything ok on the dev machine. I've asked my provider for a pre-production solution in order to be able to debug and understand the problem.
Anyway, what are according to you the most likely errors that can happen when moving a django app from dev to prod?
Best
Update: I think that a pre-prod is the best way to address this kind of problem. But I would like to build a check list of what must be done before to put in production.
Thanks for the very valuable answers that I received until now :)
Update: FYI, I 've implemented the preprod server and the email notification as suggested by shanyu and I can see that the error comes from the smart_if templatetag that I am using on this new version. Any trick with template tags?
Update: I think I've fixed the pb which was caused I think by the Filezilla FTP sending. I was using the "replace if newer" option which I guess is causing some unexpected results. Using the "replace all" option fix the issue. However, it was an opportunity for me to learn more about deployment. Thansk for your answers.
Problems I typically have include:
Misconfigured productions settings, whether in my production localsettings.py, wsgi/cgi, or apache site files in /etc/sites-available
Database differences. I use South for migrations and have run into some subtle issues when performing my migration on PostgreSQL when it worked smoothly in sqlite.
Static file hosting since I cheat and use the Django server in development
Permissions, both on the file system and within the database
Rare, but possible, network issues preventing me from getting my dependencies, whether on PyPi or some 3rd party site
Ways that I have mitigated these issues:
Use the same database in production and development (in your case, MySQL everywhere)
I've found it is useful to have a "test" environment which mimics production in every way possible (it can be on lower end hardware, or even the same machine). This way, if there are any issues in this "production-like" enivornment, I can solve them without taking my production server offline.
Script everything for repeatable deployments. I use fabric, but zc.buildout or Paver would also work. These tools help reduce typos while deploying and reduce the time to deploy my app.
Use version control (mercurial, git, subversion) and a schema migration tool (like South), so if something does go wrong when you deploy to production, you have the possibility of backing out the changes and allowing production to run on the old code with the old database schema.
I haven't set up an "egg proxy" yet, but I am considering it, to avoid issues when downloading dependencies.
I've found pip's freezing dependencies to be useful, in case a new, incompatible change to a library occurred since I downloaded it initially
Use a web testing framework like Windmill or Selenium to test my application in my "test" environment, so that I can get a lot of test coverage of my system very quickly.
Regarding your case, I can think of 2 simple things that may help you:
You can enable Django to send messages when exceptions occur giving details about them. Look at here for details.
You'll be better off if you set up a test environment on the prod server (say, test.example.com) so that you can check if things will go smoothly or not before you deploy the app.
I believe these were the podcasts I listened to recently (from Pycon 2009):
Locate Django in the Real World (PyCon 2009):
http://advocacy.python.org/podcasts/pycon.rss
Parts 1 to 3
Very good introduction to designing your apps for deployment, in particular for reuse and redeployment.
Regs.
I'm running a Django app on Apache + mod_python. When I make some changes to the code, sometimes they have effect immediately, other times they don't, until I restart Apache. However I don't really want to do that since it's a production server running other stuff too. Is there some other way to force that?
Just to make it clear, since I see some people get it wrong, I'm talking about a production environment. For development I'm using Django's development server, of course.
If possible, you should switch to mod_wsgi. This is now the recommended way to serve Django anyway, and is much more efficient in terms of memory and server resources.
In mod_wsgi, each site has a .wsgi file associated with it. To restart a site, just touch the relevant file, and only that code will be reloaded.
As others have suggested, use mod_wsgi instead. To get the ability for automatic reloading, through touching the WSGI script file, or through a monitor that looks for code changes, you must be using daemon mode on UNIX. A slight of hand can be used to achieve same on Windows when using embedded mode. All the details can be found in:
http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode
You can reduce number of connections to 1 by setting "MaxRequestsPerChild 1" in your httpd.conf file. But do it only on test server, not production.
or
If you don't want to kill existing connections and still restart apache you can restart it "gracefully" by performing "apache2ctl gracefully" - all existing connections will be allowed to complete.
Use a test server included in Django. (like ./manage.py runserver 0.0.0.0:8080) It will do most things you would need during development. The only drawback is that it cannot handle simultaneous requests with multi-threading.
I've heard that there is a trick that setting Apache's max instances to 1 so that every code change is reflected immediately--but because you said you're running other services, so this may not be your case.
This question already has an answer here:
Testing for mysterious load errors in python/django
(1 answer)
Closed 3 years ago.
I am hosting a Django app on Apache using mod_python. Occasionally, I get some cryptic mod_python errors, usually of the ImportError variety, although not usually referring to the same module. The thing is, these seem to come up for a single forked subprocess, while the others operate fine, even when I force behavior that requires using the module that the problem process has errored on. Once the process encounters the error, it will always just serve the same traceback every time Apache chooses it to handle a request. (This is also a hassle, since my users don't necessarily report the error on the first occurrence, and once the process encounters the error.)
I know more about configuring Django than configuring Apache, but that won't get me anywhere since the request never reaches Django for processing. Ideally, I should solve the root problem, and that might involve my code, project, or machine configuration, but until then, I need help diagnosing and mitigating the problem.
Is there any way to configure the Apache logs to include a subprocess id?
Is there any way to force a subprocess to respawn if it has hit an error?
Are there any known issues relating to this that I should know about?
As a workaround, and assuming you are free to install new Apache modules on the server, you might try one of
mod_scgi
mod_fastcgi
mod_wsgi
instead. I use SCGI to connect an nginx frontend webserver to my Django apps, which highlights a major benefit (decoupling from the webserver). All of these packages are available in Debian, probably on RHELx as well.