I'm trying to start the django development server from another module in my package. My module can import manage.py, and I want to execute the equivalent of manage.py runserver without using subprocess or anything of that sort (why? see below).
Currently the best solution I could come up with is to use subprocess:
def run_with_default_settings():
import inspect
import subprocess
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
subprocess.Popen(['python', 'manage.py', 'runserver'], cwd=currentdir)
However this solution seems to me rather overcomplicated, and more importantly it is not platform independent (for example if someone has both python 2 and python 3 and python is defined as python 3; or if python is not defined in the environment PATH... etc.).
I couldn't find any solutions online, and every way I tried to run execute_from_command_line() failed miserably.
Any ideas?
Yes. Just do what's in the manage.py:
import os
from django.core.management import execute_from_command_line
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'web.settings')
execute_from_command_line(list_of_args)
This should work fine. Just remember that execute_from_command_line accepts originally sys.argv as argument, so the command runserver is on the index 1:
list_of_args = ['', 'runserver']
Related
I am currently using the subprocess module in python for scripting purposes, and have been unable to execute the command 'module list' despite this working when I run it in shell and despite any other kind of command working fine when using the subprocess module to execute commands.
Two variations I have tried:
p = subprocess.Popen('module list', shell=True)
print(p.communicate()[0])
and
p = Popen(["module", "list"], stdout=subprocess.PIPE)
print(p.communicate()[0])
For commands such as 'echo hello world' and even longer commands with multiple arguments, either of these formats works just fine. Is the terminal I run commands from different from the shell used to execute commands using subprocess? The error I get is as follows:
/bin/bash: line 1: module: command not found
Based on what you've said in the comments, I believe you're going about using environment modules in Python the wrong way: There is actually a method in Modules itself to import module functionality into Python, as explained here:
>>> execfile('/usr/local/Modules/default/init/python.py')
>>> module('list')
No Modulefiles Currently Loaded.
>>> module('load','foo')
>>> module('list')
Currently Loaded Modulefiles:
1) foo/1.0
Of course, it's not very safe to use execfile(), so I slightly prefer the import method described here (slightly altered for Python 3 support):
import os
if 'PYTHONPATH' in os.environ:
os.environ['PYTHONPATH'] +=':'+os.environ['MODULESHOME']+"/init"
else:
os.environ['PYTHONPATH'] = os.environ['MODULESHOME']+"/init"
from python import module
The documentation of the Environment Modules software provides a recommendation on how to initialize the module command in Python (that should work on either Python 2 or 3):
import os
exec(open('/usr/share/Modules/init/python.py').read())
Once initialized, the module function is available and could be used in the following way:
module('sub-command', 'arg1', 'arg2', ...)
For example:
module('load', 'foo', 'bar')
module('list')
module('avail')
I'm trying to start a python program as a subprocess using the following code but the subprocess outputs ImportErrors to stderr.
The code
import subprocess
import sys
import os
environment = os.environ
command = ["python", "-u", "/test/my_python_program.py"]
p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, env=environment)
for line in iter(p.stdout.readline, ''):
line = line.replace('\r', '').replace('\n', '')
print line
sys.stdout.flush()
The error
Traceback (most recent call last):
File "/test/my_python_program.py", line 31, in <module>
from PySide import QtGui, QtCore, QtNetwork
ImportError: dlopen(/Applications/Autodesk/maya2014/Maya.app/Contents/Frameworks/Python.framework/Versions/Current/lib/python2.7/site-packages/PySide/QtGui.so, 2): Library not loaded: #executable_path/libpyside-python2.7.1.1.dylib
Referenced from: /Applications/Autodesk/maya2014/Maya.app/Contents/Frameworks/Python.framework/Versions/Current/lib/python2.7/site-packages/PySide/QtGui.so
Reason: image not found
I'm running my code from a version of python embedded into another program (Autodesk Maya). The module's that cause the ImportError are importable from the main processes python interpreter. My assumption was that I could supply the main process's python's env to the subprocess to give it access to the modules that the main python instance had access too.
Does the approach make sense? and if so what am I doing wrong?
Edit1
I also tried to use the embedded version of python used in the popen call and the errors remained. If I remove the Import the embedded version of python runs just fine.
Edit 2
These are the environment variables that os.environ returns
MAYA_MODULE_PATH
AUTOLOADER_LAPS
LOGNAME
USER
SUBSTANCES_LOCATION
PATH
HOME
MAYA_SCRIPT_BASE
MENTALRAY_INCLUDE_LOCATION
SHELL
MAYA_LICENSE_METHOD
MAYA_LICENSE
QT_MAC_NO_NATIVE_MENUBAR
MAYA_SCRIPT_PATH
MAYA_REVERSE_FILEFORMAT_EXT
WF_IMF_CIN_WHITE_POINT
MAYA_LOCATION
PYTHONPATH
SSH_AUTH_SOCK
MENTALRAY_SHADERS_LOCATION
Apple_PubSub_Socket_Render
MAYA_PRESET_PATH
XBMLANGPATH
MAYA_RENDER_DESC_PATH
MAYA_SHADER_LIBRARY_PATH
MENTALRAY_LOCATION
TMPDIR
__KMP_REGISTERED_LIB_5123
MAYA_PLUG_IN_PATH
MAYA_APP_DIR
PYTHONHOME
MAYA_PLUG_IN_RESOURCE_PATH
__CF_USER_TEXT_ENCODING
IMF_PLUG_IN_PATH
__CHECKFIX1436934
WF_IMF_CIN_CORRECTION
Edit 3
Here's the contents of PYHOME
/Applications/Autodesk/maya2014/Maya.app/Contents/Frameworks/Python.framework/Versions/Current
The Python running inside Maya is not completely standard...
When run from within Maya, you might need to use the mayapy command to launch Python. This bootstraps Python according to Autodesk's requirements and should ensure the environment required to run Python is correct. (Although I think this may be what you are referring to in Edit1).
Have you also tried not passing through the environment to the popen command? I think the default behaviour of popen is to inherit the current process' environment which is what you want. I would expect the environment to be setting DYLD_LIBRARY_PATH and/or DYLD_FRAMEWORK_PATH at some point to tell it where Qt is.
I'm trying to automate the test rerun after a change while developing. After searching around a little sniffer seemed fine. But if I run it my tests fail with this error:
ERROR: Failure: ImportError (Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.)
if I run them manually they pass. Do you have a clue why sniffer won't work?
Something like the following as your scent.py should work:
from subprocess import call
from sniffer.api import runnable
#runnable
def execute_tests(*args):
fn = [ 'python', 'manage.py', 'test' ]
fn += args[1:]
return call(fn) == 0
Which you can then call as sniffer -x appName.
You can get sniffer to read your settings by creating a scent.py file in the same directory as manage.py.
Here's what mine looks like:
import os
os.environ["DJANGO_SETTINGS_MODULE"] = 'myapp.settings'
Which will get you as far as sniffer reading your settings, but then you'll run into other problems — basically, sniffer just runs your tests using nose, which isn't the same thing that the manage.py test does when django-nose is installed.
Anybody know what else needs to be in scent.py for snigger to with with Django?
Trying to guess where the problem may reside: it seems you need to explicitly set the position of your settings.py file.
if you're running your test from a subprocess' call you can use the following command:
call(["django-admin.py", "test --settings=your_project.settings"])
otherwise you can set environment variables with the following command:
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'your_project.settings'
(change your_project with the name of your django project)
if you're running a command like "./manage.py tests" you can add the former lines at the beginning of manage.py (there are other ways but I need to see the code to provide a more precise solution)
I'm trying to run a custom django command as a scheduled task on Heroku. I am able to execute the custom command locally via: python manage.py send_daily_email. (note: I do NOT have any problems with the custom management command itself)
However, Heroku is giving me the following exception when trying to "Run" the task through Heroku Scheduler addon:
Traceback (most recent call last):
File "bin/send_daily_visit_email.py", line 2, in <module>
from django.conf import settings
ImportError: No module named django.conf
I placed a python script in /bin/send_daily_email.py, and it is the following:
#! /usr/bin/python
from django.conf import settings
settings.configure()
from django.core import management
management.call_command('send_daily_email') #delegates off to custom command
Within Heroku, however, I am able to run heroku run bin/python - launch the python shell - and successfully import settings from django.conf
I am pretty sure it has something to do with my PYTHON_PATH or visibility to Django's SETTINGS_MODULE, but I'm unsure how to resolve the issue. Could someone point me in the right direction? Is there an easier way to accomplish what I'm trying to do here?
Thank you so much for your tips and advice in advance! New to Heroku! :)
EDIT:
Per Nix's comment, I made some adjustments, and did discover that specifying my exact python path, I did get past the Django setup.
I now receive:
File "/app/lib/python2.7/site-packages/django/core/management/__init__.py", line 155, in call_command
raise CommandError("Unknown command: %r" % name)
django.core.management.base.CommandError: Unknown command: 'send_daily_email'
Although, I can see 'send_daily_email' when I run ``heroku run bin/python app/manage.py```.
I'll keep an update if I come across the answer.
You are probably using a different interpreter.
Check to make sure shell python is the same as the one you reference in your script /usr/bin/python . It could be that there is a different one in your path, which would explain why it works when you run python manage.py but not your shell scrip which you explicitly reference /usr/bin/python.
Typing which python will tell you what interpreter is being found on your path.
In addition, this can also be resolved by adding your home directory to your Python path. A quick and unobtrusive way to accomplish that is to add it to the PYTHONPATH environment variable (which is generally /app on the Heroku Cedar stack).
Add it via the heroku config command:
$ heroku config:add PYTHONPATH=/app
That should do it! For more details: http://tomatohater.com/2012/01/17/custom-django-management-commands-on-heroku/
I've wrote a nice app for myself using the Twisted framework. I launch it using a command like:
twistd -y myapp.py --pidfile=/var/run/myapp.pid --logfile=/var/run/myapp.log
It works great =)
To launch my app I wrote a script with this command because I'm lazy^^
But since I launch my app with the same twistd option, and I tink the script shell solution is ugly, how I can do the same but inside my app? I'd like to launch my app by just doing ./myapp and without a shell work around.
I've tried to search about it in twisted documentation and by reading twisted source but I don't understand it since it's my first app in Python (wonderful language btw!)
Thanks in advance for anyhelp.
You need to import the twistd script as a module from Twisted and invoke it. The simplest solution for this, using your existing command-line, would be to import the sys module to replace the argv command line to look like how you want twistd to run, and then run it.
Here's a simple example script that will take your existing command-line and run it with a Python script instead of a shell script:
#!/usr/bin/python
from twisted.scripts.twistd import run
from sys import argv
argv[1:] = [
'-y', 'myapp.py',
'--pidfile', '/var/run/myapp.pid',
'--logfile', '/var/run/myapp.log'
]
run()
If you want to bundle this up nicely into a package rather than hard-coding paths, you can determine the path to myapp.py by looking at the special __file__ variable set by Python in each module. Adding this to the example looks like so:
#!/usr/bin/python
from twisted.scripts.twistd import run
from my.application import some_module
from os.path import join, dirname
from sys import argv
argv[1:] = [
'-y', join(dirname(some_module.__file__), "myapp.py"),
'--pidfile', '/var/run/myapp.pid',
'--logfile', '/var/run/myapp.log'
]
run()
and you could obviously do similar things to compute appropriate pidfile and logfile paths.
A more comprehensive solution is to write a plugin for twistd. The axiomatic command-line program from the Axiom object-database project serves as a tested, production-worthy example of how to do similar command-line manipulation of twistd to what is described above, but with more comprehensive handling of command-line options, different non-twistd-running utility functionality, and so on.
You can also create the options / config for a twisted command and pass it to the twisted runner.
#!/usr/bin/env python3
import twisted.scripts.twistd
import twisted.scripts._twistd_unix
config = twisted.scripts._twistd_unix.ServerOptions()
config.parseOptions([
"--nodaemon",
"web",
"--listen=tcp:80",
"--path=/some/path"
])
twisted.scripts._twistd_unix.UnixApplicationRunner(config).run()