Develop Reference

Can the Django development server be started programmatically?

I'm trying to start the django development server from another module in my package. My module can import manage.py, and I want to execute the equivalent of manage.py runserver without using subprocess or anything of that sort (why? see below).
Currently the best solution I could come up with is to use subprocess:
def run_with_default_settings():
import inspect
import subprocess
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
subprocess.Popen(['python', 'manage.py', 'runserver'], cwd=currentdir)
However this solution seems to me rather overcomplicated, and more importantly it is not platform independent (for example if someone has both python 2 and python 3 and python is defined as python 3; or if python is not defined in the environment PATH... etc.).
I couldn't find any solutions online, and every way I tried to run execute_from_command_line() failed miserably.
Any ideas?

Yes. Just do what's in the manage.py:
import os
from django.core.management import execute_from_command_line
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'web.settings')
execute_from_command_line(list_of_args)
This should work fine. Just remember that execute_from_command_line accepts originally sys.argv as argument, so the command runserver is on the index 1:
list_of_args = ['', 'runserver']

Python does not show code changes from imported file

I am using a linux python shell and each time I make changes to the imported file I need restart the shell (I tried reimporting the file but the changes were not reflected)
I have a definition in a file called handlers.py
def testme():
print "Hello I am here"
I import the file in the python shell
>> import handlers as a
>> a.testme()
>> "Hello I am here"
I then change print statement to "Hello I am there", reimport handlers, it does not show the change?
Using Python 2.7 with Mint 17.1

You need to explicitly reload the module, as in:
import lib # first import
# later ....
import imp
imp.reload(lib) # lib being the module which was imported before
note that imp module is pending depreciation in favor of importlib and in python 3.4 one should use: importlib.reload.

You should use reload every time you make a change and then import again:
reload( handlers )
import handlers a a

As an alternative answer inside reload you can use
watchdog
.
A simple program that uses watchdog to monitor directories specified as command-line arguments and logs events generated:
From the website
Supported Platforms
Linux 2.6 (inotify)
Mac OS X (FSEvents, kqueue)
FreeBSD/BSD (kqueue)
Windows (ReadDirectoryChangesW with I/O completion ports; ReadDirectoryChangesW worker threads)
OS-independent (polling the disk for directory snapshots and comparing them periodically; slow and not recommended)

how to start django shell with ipython in qtconsole mode?

When i start django shell by typing python manage.py shell
the ipython shell is started. Is it possible to make Django start ipython in qtconsole mode? (i.e. make it run ipython qtconsole)
Arek
edit:
so I'm trying what Andrew Wilkinson suggested in his answer - extending my django app with a command which is based on original django shell command. As far as I understand code which starts ipython in original version is this:
from django.core.management.base import NoArgsCommand
class Command(NoArgsCommand):
requires_model_validation = False
def handle_noargs(self, **options):
from IPython.frontend.terminal.embed import TerminalInteractiveShell
shell = TerminalInteractiveShell()
shell.mainloop()
any advice how to change this code to start ipython in qtconsole mode?
second edit:
what i found and works so far is - start 'ipython qtconsole' from the location where settings.py of my project is (or set the sys.path if starting from different location), and then execute this:
import settings
import django.core.management
django.core.management.setup_environ(settings)
and now can i import my models, list all instances etc.

The docs here say:
If you'd rather not use manage.py, no problem. Just set the
DJANGO_SETTINGS_MODULE environment variable to mysite.settings and run
python from the same directory manage.py is in (or ensure that
directory is on the Python path, so that import mysite works).
So it should be enough to set that environment variable and then run ipython qtconsole. You could make a simple script to do this for you automatically.

I created a shell script with the following:
/path/to/ipython
qtconsole --pylab inline -c "run /path/to/my/site/shell.py"
You only need the --pylab inline part if you want the cool inline matplotlib graphs.
And I created a python script shell.py in /path/to/my/site with:
import os
working_dir = os.path.dirname(__file__)
os.chdir(working_dir)
import settings
import django.core.management
django.core.management.setup_environ(settings)
Running my shell script gets me an ipython qtconsole with the benefits of the django shell.

You can check the code that runs the shell here. You'll see that there is no where to configure what shell is run.
What you could do is copy this file, rename it as shell_qt.py and place it in your own project's management/commands directory. Change it to run the QT console and then you can run manage.py shell_qt.

Since Django version 1.4, usage of django.core.management.setup_environ() is deprecated. A solution that works for both the IPython notebook and the QTconsole is this (just execute this from within your Django project directory):
In [1]: from django.conf import settings
In [2]: from mydjangoproject.settings import DATABASES as MYDATABASES
In [3]: settings.configure(DATABASES=MYDATABASES)
Update: If you work with Django 1.7, you additionally need to execute the following:
In [4]: import django; django.setup()
Using django.conf.settings.configure(), you specify the database settings of your project and then you can access all your models in the usual way.
If you want to automate these imports, you can e.g. create an IPython profile by running:
ipython profile create mydjangoproject
Each profile contains a directory called startup. You can put arbitrary Python scripts in there and they will be executed just after IPython has started. In this example, you find it under
~/.ipython/profile_<mydjangoproject>/startup/
Just put a script in there which contains the code shown above, probably enclosed by a try..except clause to handle ImportErrors. You can then start IPython with the given profile like this:
ipython qtconsole --profile=mydjangoproject
or
ipython notebook --profile=mydjangoproject

I also wanted to open the Django shell in qtconsole. Looking inside manage.py solve the problem for me:
Launch IPython qtconsole, cd to the project base directory and run:
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")
Dont forget to change 'myproject' to your project name.

You can create a command that extends the base shell command and imports the IPythonQtConsoleApp like so:
create file qtshell.py in yourapp/management/commands with:
from django.core.management.commands import shell
class Command(shell.Command):
def _ipython(self):
"""Start IPython Qt console"""
from IPython.qt.console.qtconsoleapp import IPythonQtConsoleApp
app = IPythonQtConsoleApp.instance()
app.initialize(argv=[])
app.start()
then just use python manage.py qtshell

A somewhat undocumented feature of shell_plus is the ability to run it in "kernel only mode". This allows us to connect to it from another shell, such as one running qtconsole.
For example, in one shell do:
django-admin shell_plus --kernel
# or == ./manage.py shell_plus --kernel
This will print out something like:
# Shell Plus imports ...
...
To connect another client to this kernel, use:
--existing kernel-23600.json
Then, in another shell run:
ipython qtconsole --existing kernel-23600.json
This should now open a QtConsole. One other tip, instead of running another shell, you can also hit Ctrl+Z, and run bg to tell current process to run in background.

You can install django extensions and then run
python manage.py shell_plus --ipython

Twisted application without twistd

I've wrote a nice app for myself using the Twisted framework. I launch it using a command like:
twistd -y myapp.py --pidfile=/var/run/myapp.pid --logfile=/var/run/myapp.log
It works great =)
To launch my app I wrote a script with this command because I'm lazy^^
But since I launch my app with the same twistd option, and I tink the script shell solution is ugly, how I can do the same but inside my app? I'd like to launch my app by just doing ./myapp and without a shell work around.
I've tried to search about it in twisted documentation and by reading twisted source but I don't understand it since it's my first app in Python (wonderful language btw!)
Thanks in advance for anyhelp.

You need to import the twistd script as a module from Twisted and invoke it. The simplest solution for this, using your existing command-line, would be to import the sys module to replace the argv command line to look like how you want twistd to run, and then run it.
Here's a simple example script that will take your existing command-line and run it with a Python script instead of a shell script:
#!/usr/bin/python
from twisted.scripts.twistd import run
from sys import argv
argv[1:] = [
'-y', 'myapp.py',
'--pidfile', '/var/run/myapp.pid',
'--logfile', '/var/run/myapp.log'
]
run()
If you want to bundle this up nicely into a package rather than hard-coding paths, you can determine the path to myapp.py by looking at the special __file__ variable set by Python in each module. Adding this to the example looks like so:
#!/usr/bin/python
from twisted.scripts.twistd import run
from my.application import some_module
from os.path import join, dirname
from sys import argv
argv[1:] = [
'-y', join(dirname(some_module.__file__), "myapp.py"),
'--pidfile', '/var/run/myapp.pid',
'--logfile', '/var/run/myapp.log'
]
run()
and you could obviously do similar things to compute appropriate pidfile and logfile paths.
A more comprehensive solution is to write a plugin for twistd. The axiomatic command-line program from the Axiom object-database project serves as a tested, production-worthy example of how to do similar command-line manipulation of twistd to what is described above, but with more comprehensive handling of command-line options, different non-twistd-running utility functionality, and so on.

You can also create the options / config for a twisted command and pass it to the twisted runner.
#!/usr/bin/env python3
import twisted.scripts.twistd
import twisted.scripts._twistd_unix
config = twisted.scripts._twistd_unix.ServerOptions()
config.parseOptions([
"--nodaemon",
"web",
"--listen=tcp:80",
"--path=/some/path"
])
twisted.scripts._twistd_unix.UnixApplicationRunner(config).run()

Setting up Django on an internal server (os.environ() not working as expected?)

I'm trying to setup Django on an internal company server. (No external connection to the Internet.)
Looking over the server setup documentation it appears that the "Running Django on a shared-hosting provider with Apache" method seems to be the most-likely to work in this situation.
Here's the server information:
Can't install mod_python
no root access
Server is SunOs 5.6
Python 2.5
Apache/2.0.46
I've installed Django (and flup) using the --prefix option (reading again I probably should've used --home, but at the moment it doesn't seem to matter)
I've added the .htaccess file and mysite.fcgi file to my root web directory as mentioned here.
When I run the mysite.fcgi script from the server I get my expected output (the correct site HTML output). But, it won't when trying to access it from a browser.
It seems that it may be a problem with the PYTHONPATH setting since I'm using the prefix option.
I've noticed that if I run mysite.fcgi from the command-line without setting the PYTHONPATH enviornment variable it throws the following error:
prompt$ python2.5 mysite.fcgi
ERROR:
No module named flup Unable to load
the flup package. In order to run
django as a FastCGI application, you
will need to get flup from
http://www.saddi.com/software/flup/
If you've already installed flup,
then make sure you have it in your
PYTHONPATH.
I've added sys.path.append(prefixpath) and os.environ['PYTHONPATH'] = prefixpath to mysite.fcgi, but if I set the enviornment variable to be empty on the command-line then run mysite.fcgi, I still get the above error.
Here are some command-line results:
>>> os.environ['PYTHONPATH'] = 'Null'
>>>
>>> os.system('echo $PYTHONPATH')
Null
>>> os.environ['PYTHONPATH'] = '/prefix/path'
>>>
>>> os.system('echo $PYTHONPATH')
/prefix/path
>>> exit()
prompt$ echo $PYTHONPATH
Null
It looks like Python is setting the variable OK, but the variable is only applicable inside of the script. Flup appears to be distributed as an .egg file, and my guess is that the egg implementation doesn't take into account variables added by os.environ['key'] = value (?) at least when installing via the --prefix option.
I'm not that familiar with .pth files, but it seems that the easy-install.pth file is the one that points to flup:
import sys; sys.__plen = len(sys.path)
./setuptools-0.6c6-py2.5.egg
./flup-1.0.1-py2.5.egg
import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sy
s.path[p:p]=new; sys.__egginsert = p+len(new)
It looks like it's doing something funky, anyway to edit this or add something to my code so it will find flup?

In your settings you have to point go actual egg file, not directory where egg file is located. It should look something like:
sys.path.append('/path/to/flup/egg/flup-1.0.1-py2.5.egg')

Try using a utility called virtualenv. According to the official package page, "virtualenv is a tool to create isolated Python environments."
It'll take care of the PYTHONPATH stuff for you and make it easy to correctly install Django and flup.

Use site.addsitedir() not os.environ['PYTHONPATH'] or sys.path.append().
site.addsitedir interprets the .pth files. Modifying os.environ or sys.path does not. Not in a FastCGI environment anyway.
#!/user/bin/python2.6
import site
# adds a directory to sys.path and processes its .pth files
site.addsitedir('/path/to/local/prefix/site-packages/')
# avoids permissions error writing to system egg-cache
os.environ['PYTHON_EGG_CACHE'] = '/path/to/local/prefix/egg-cache'

To modify the PYTHONPATH from a python script you should use:
sys.path.append("prefixpath")
Try this instead of modifying with os.environ().
And I would recommend to run Django with mod_python instead of using FastCGI...