Develop Reference

sys.path.append causing 500 error in Flask

I have a flask app that is giving me 500 when I import a file using sys.path.append('path/to/file.py)
Here is my file located in /var/www/html/ip.py which flask is calling:
import sys
sys.path.append('/auto/conf/')
from config import config
server_username = config['server_username']
server_password = config['server_prod_password']
def check_ip(ip_addr):
return "test"
Here is my /auto/conf/config.py:
import os
import ConfigParser
# Define names of dir and file
CRED_DIR = 'cred'
FILE_NAME = 'cred.conf'
# Path to cred file
my_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
cred_dir = os.path.join(my_dir, CRED_DIR)
file_path = os.path.join(cred_dir, FILE_NAME)
# Load the conf file with ConfigParser
config = ConfigParser.SafeConfigParser()
config.read(file_path)
# Build a config dictionary
config = {
'server_username': config.get('server', 'username'),
'server_password': config.get('server', 'password'),
}
and cred.conf is located in /auto/cred/cred.conf and contains server info.
Now, here is the issue. If I run python ip.py, it runs fine. I added print statement and it was fetching proper server username and password. But when I run it via Flask, it gives me 500 error.
Here are some of the things I tried:
-- If I comment out "from config import config" from ip.py, Flask runs returns "test" meaning it worked. It will not get server un and pw but atleast it does not 500.
-- If I move cred.conf and config.py to same directory as ip.py and comment out "sys.path.append('/auto/conf/')" and uncomment "from config import config", Flask works.
Any ideas why its happening? I am thinking Flask does not like sys.path.append. Is there any alternative I can use so Flask works?
Edit:
I changed ip.py to this:
import sys
sys.path.append('/auto/conf/')
import config
and removed all code in config.py and it is still gving me error. If I comment out "import config", Flask works. Definitely it does not like importing in this fashion..

Python gives errors when running Django tests

I got errors while running runtests.py.
I use Ubuntu 16.04 and Django version 1.6.5. I copied code from github.
./runtests.py: line 3: os.environ[DJANGO_SETTINGS_MODULE]: command not found
./runtests.py: line 4: syntax error near unexpected token `(`
./runtests.py: line 4: `test_dir = os.path.dirname(__file__)`
My runtest.py file:
import os
import sys
os.environ['DJANGO_SETTINGS_MODULE'] = 'test-settings'
test_dir = os.path.dirname(__file__)
sys.path.insert(0, test_dir)
import django
from django.test.utils import get_runner
from django.conf import settings
def runtests():
if django.VERSION >= (1, 7):
django.setup()
TestRunner = get_runner(settings)
test_runner = TestRunner(verbosity=1, interactive=True)
failures = test_runner.run_tests(
['quiz', 'essay', 'multichoice', 'true_false']
)
sys.exit(bool(failures))
if __name__ == '__main__':
runtests()
How could I solve these errors?

Looks like you're trying to execute this script on the command line as in ./runtests.py, which is invoking a bash interpreter and hence why you're getting all kinds of weird errors. 2 ways to solve this:
Run it like python runtests.py
Put a shebang at the top of the file #!/usr/bin/env python then run it ./runtests.py.

According to this post you have to change
os.environ['DJANGO_SETTINGS_MODULE'] = 'test-settings'
to
os.environ['DJANGO_SETTINGS_MODULE'] = 'test.settings'
Note
test must be your django project name. If it is not, change it accordingly.

How do i set django setting to a file which is outside the django tree?

I have this python file tasks.py
import os
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'DjangoWebProject.settings')
from django.contrib.auth.models import User
from logreg.models import ActivationCode
import datetime
def remove_users():
print 'hello worldddddddddddddddddddddddddddddddddd'
inactive_users = []
activation_codes = ActivationCode.objects.all()
for activation_code in activation_codes:
if datetime.datetime.date(activation_code.key_expires) < datetime.datetime.date(datetime.datetime.now()):
inactive_users.append(activation_code.user_id)
for inactive_user in inactive_users:
User.objects.filter(id=inactive_user).delete()
But this is in the root folder and when i try to execute it, it gives me the following error
File
"C:\Users\deybala1\AppData\Local\Continuum\Anaconda2\lib\site-packages\dj
ango\apps\registry.py", line 124, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.") django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
How do i fix this?

If you're creating any script that is using your django project, it is absolutely necessary to set path to settings of your project before any import from django or your project. And you're importing user model from django in 1st line and model from your project in second.
Also, you will need to call django.setup() first.
To fix that, move import os and setting path to django settings to the very beginning of your script, and put django.setup() just after that (with proper import), like this:
# first, set path to project settings
import os
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'DjangoWebProject.settings')
import django
django.setup()
# now you can import anything else
from django.contrib.auth.models import User
from logreg.models import ActivationCode
import datetime
def remove_users():
print 'hello worldddddddddddddddddddddddddddddddddd'
inactive_users = []
activation_codes = ActivationCode.objects.all()
for activation_code in activation_codes:
if datetime.datetime.date(activation_code.key_expires) < datetime.datetime.date(datetime.datetime.now()):
inactive_users.append(activation_code.user_id)
for inactive_user in inactive_users:
User.objects.filter(id=inactive_user).delete()

Note that you're trying to add a settings module inside a script that already requires it.
Wouldn't it be easier if you add a specific django command? Thanks to it you'd be able to start your task with python manage.py --settings=<path_to_your_settings>.
Another tip:
Move every django import statement below
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'DjangoWebProject.settings')

Using environment dependent django settings inside fabric task

I'm trying to use the database configuration set on settings files to make a database dump using fabric.
There's more than one settings file, so I'd like to be able to do so based on the environment I choose.
by now, my task is like this
def dump_database():
with cd('~/project_folder'), prefix(WORKON_VIRTUALENV):
django.settings_module(env.settings)
from django.conf import settings
dbname = settings.DATABASES['default']['NAME']
dbuser = settings.DATABASES['default']['USER']
dbpassword = settings.DATABASES['default']['PASSWORD']
fname = '/tmp/{0}-backup-{1}.sql.gz'.format(
dbname,
time.strftime('%Y%m%d%H%M%S')
)
run('mysqldump -u %s -p=%s %s | gzip -9 /tmp/backup-%s.sql.gz' % (
dbuser,
dbpassword,
dbname,
fname))
But I'm getting an ImportError:
ImportError: Could not import settings 'project.settings.production'
I've tried to use shell_env() to set the DJANGO_SETTINGS_MODULE instead of django.settings_module(env.settings), with the same result.
I use a task to change the environment based on a environment dict:
def environment(name):
env.update(environments[name])
env.environment = name
This way, I want to be able to create a dump from multiple hosts like:
fab environment:live dump_database
fab environment:otherhost dump_database
Without having to reproduce database settings from all hosts on fabfile.

Importing your Django settings file in fabric is explained here.
http://fabric.readthedocs.org/en/1.3.3/api/contrib/django.html
Quoting from the above link:
from fabric.api import run
from fabric.contrib import django
django.settings_module('myproject.settings')
from django.conf import settings
def dump_production_database():
run('mysqldump -u %s -p=%s %s > /tmp/prod-db.sql' % (
settings.DATABASE_USER,
settings.DATABASE_PASSWORD,
settings.DATABASE_NAME
))

NOTE: I don't answer the question but offer a different solution
I had the same problem.. so I did custom .py script like that:
I created a file named dump_db.py (placed next to fabfile.py for example, that is on the remote machine)
import os
import sys
from datetime import datetime
from django.conf import settings
def dump_mysql():
os.environ.setdefault("DJANGO_SETTINGS_MODULE", SETTINGS_MODULE)
DB_NAME = settings.DATABASES['default']['NAME']
DB_USER = settings.DATABASES['default']['USER']
DB_PASSWORD = settings.DATABASES['default']['PASSWORD']
dump_file_name = '{time}_{db_name}.sql'.format(
time=datetime.now().strftime('%Y_%m_%d'),
db_name=DB_NAME,
)
os.system('mysqldump -u {db_user} -p{db_password} {db_name} > {to_file}'.format(
db_user=DB_USER,
db_password=DB_PASSWORD,
db_name=DB_NAME,
to_file=dump_file_name,
))
return dump_file_name
if __name__ == '__main__':
try:
SETTINGS_MODULE = sys.argv[1:].pop()
except IndexError:
SETTINGS_MODULE = 'project_name.settings'
print dump_mysql()
As you see sys.argv[1:].pop() tries to take optional argument (the setting module in this case).
So in my fabfile:
import os
from fabric.api import env, local, run, prefix, cd
.....
def dump():
current_dir = os.getcwd()
with prefix('source {}bin/activate'.format(env.venv)), cd('{}'.format(env.home)):
dumped_file = run('python dump_db.py {}'.format(env.environment)) # the optional argument given
file_path = os.path.join(env.home, dumped_file)
copy_to = os.path.join(current_dir, dumped_file)
scp(file_path, copy_to)
def scp(file_path, copy_to):
local('scp {}:{} {}'.format(env.host, file_path, copy_to))
where env.environment = 'project_name.settings.env_module'
And this is how I dump my DB and copy it back to me.
Hope it comes handy to someone! :)

app engine python is this bug: when open switch interactive console

While I am working at localhost:8080, when I open interactive console and do some operations, like getting list of Kind etc (address: http://localhost:8080/_ah/admin/interactive) then it gives me this error:
<class 'google.appengine.dist._library.UnacceptableVersionError'>: django 1.2 was requested, but 0.96.4.None is already in use
This errors happened several times, in similar cases. It is stuck until restart localhost by dev_appserver.py
Is this a bug or what I am doing wrong?
Example for what I did at interactive console:
from myapp.models import *
for room in Room.all():
room.update_time = room.create_time
room.put()
Note:
This is my django_bootstrap :
import os
import sys
import logging
import __builtin__
from google.appengine.ext.webapp import util
import pickle
sys.modules['cPicle'] =pickle
logging.getLogger().setLevel(logging.INFO)
sys.path.insert(0, os.path.abspath((os.path.dirname(__file__))))
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
from google.appengine.dist import use_library
use_library('django', '1.2')
import django.core.handlers.wsgi
def main():
application = django.core.handlers.wsgi.WSGIHandler()
util.run_wsgi_app(application)
if __name__ == '__main__':
main()

my index.ymal in root folder says:
# AUTOGENERATED
# This index.yaml is automatically updated whenever the dev_appserver
# detects that a new type of query is run. If you want to manage the
# index.yaml file manually, remove the above marker line (the line
# saying "# AUTOGENERATED"). If you want to manage some indexes
# manually, move them above the marker line. The index.yaml file is
# automatically uploaded to the admin console when you next deploy
# your application using appcfg.py.
Thus each time I open http://localhost:8080/_ah/admin/datastore, this file updated: which is still has the same content but timestamp of file on operating system says it is updated.
I think here, As the http://localhost:8080 sees that models.py is not the same then it could load it then can not start django_bootstrap.
However if I first open http://localhost:8080/_ah/admin/datastore and then http://localhost:8080, it works. So this is why sometimes I get error sometimes not: It depends of order urls respective