I have some usernames, passwords and other configurations setup in the environment variables of an ec2 instance. I have created a virtualenv setup and active where I run my django server. In the settings file of that django server I access the environment variables as os.environ['variable'].
Outside the virtualenv the site could access those variables fine. When I run printenv, I see all the variables and the values.
However, the server cannot find them and is throwing key errors as a result when I call os.environ on them.
setup = ec2 instance - mod_wsgi - nginx - apache
UPDATE
This started working by setting the variables in django.wsgi in the following way.
os.environ['SQL_PASSWORD'] = 'password'
That alone stopped working once I upgraded to the new ec2 hardware. I am not sure how that was related.
Now, what worked finally was setting the variables individually using SetEnv in the apache config file. Still not optimal because I have to keep the config file checked out on the production machine but it unblocks me.
SetEnv SQL_PASSWORD password
I use nginx and uwsgi on Ec2 and have a chef recipe that builds out my servers. To solve this problem, in chef I have roles that contain credentials not stored in the app's repo.
file at /home/user/web/site/environment that chef creates based on roles.
MYSQL_DATABASE=databasename
MYSQL_USER=databaseuser
MYSQL_PASSWORD=databasepassword
MYSQL_HOST=databaseip
MYSQL_PORT=3306
REDIS_HOST=redishost
REDIS_PASSWORD=redispassword
REDIS_PORT=6379
REDIS_DB=0
MEDIA_ROOT=/home/user/web/site/media
STATIC_ROOT=/home/user/web/site/static
at the beginning of my production/staging/etc settings file I have the following block to read the environment file
import os, re
try:
dirname = os.path.dirname(os.path.abspath(__file__))
# my environment file is always in the same place relative to my project's settings file
env_path = os.path.normpath(os.path.join(dirname, '..', '..', '..', '..', 'environment'))
with open(env_path) as f:
content = f.read()
for line in content.splitlines():
m1 = re.match(r'\A([A-Za-z_0-9]+)=(.*)\Z', line)
if m1:
key, val = m1.group(1), m1.group(2)
m2 = re.match(r"\A'(.*)'\Z", val)
if m2:
val = m2.group(1)
m3 = re.match(r'\A"(.*)"\Z', val)
if m3:
val = re.sub(r'\\(.)', r'\1', m3.group(1))
os.environ.setdefault(key, val)
except IOError:
pass
Voila, settings are source controlled in a separately managed repo (chef's) and loaded into the app. This may not be the BEST way, but it is fairly secure b/c of how permissions are locked down on the chef repo and the target servers and easy to deploy.
Related
I'm setting up my first Django VM with Vanguard following this guide. Ran into an error:
==> djangovm: [2014-10-20T15:04:09+02:00] ERROR: Running exception handlers
==> djangovm: [2014-10-20T15:04:09+02:00] ERROR: Exception handlers complete
==> djangovm: [2014-10-20T15:04:09+02:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
==> djangovm: [2014-10-20T15:04:09+02:00] FATAL: NameError: uninitialized constant Chef::Resource::LWRPBase
Found the answer here: Vagrant & Chef: uninitialized constant Chef::Resource::LWRPBase which says to refer to this answer: How to control the version of Chef that Vagrant uses to provision VMs?
I'm just not sure where do I put:
config.omnibus.chef_version = :latest
in my Vagrantfile?
My Vagrant file looks like that tutorial link exactly at the moment:
Vagrant::Config.run do |config|
config.vm.define :djangovm do |django_config|
# Every Vagrant virtual environment requires a box to build off of.
django_config.vm.box = "lucid64"
# The url from where the 'config.vm.box' box will be fetched if it
# doesn't already exist on the user's system.
django_config.vm.box_url = "http://files.vagrantup.com/lucid64.box"
# Forward a port from the guest to the host, which allows for outside
# computers to access the VM, whereas host only networking does not.
django_config.vm.forward_port 80, 8080
django_config.vm.forward_port 8000, 8001
# Enable provisioning with chef solo, specifying a cookbooks path (relative
# to this Vagrantfile), and adding some recipes and/or roles.
#
django_config.vm.provision :chef_solo do |chef|
chef.cookbooks_path = "cookbooks"
chef.add_recipe "apt"
chef.add_recipe "apache2::mod_wsgi"
chef.add_recipe "build-essential"
chef.add_recipe "git"
chef.add_recipe "vim"
#
# # You may also specify custom JSON attributes:
# chef.json = { :mysql_password => "foo" }
end
end
end
I tried adding it in like so:
# The url from where the 'config.vm.box' box will be fetched if it
# doesn't already exist on the user's system.
django_config.vm.box_url = "http://files.vagrantup.com/lucid64.box"
# This gets the latest version of Omnibus
config.omnibus.chef_version = :latest
Then did Vagrant Destroy then Vagrant Up and still got the same error.
You can put it inside the :djangovm definition as you tried, but there you need to use
django_config.omnibus.chef_version = :latest
Of course you also have to install the vagrant-omnibus too:
vagrant plugin install vagrant-omnibus
Btw, it is safer to lock down the Chef version to a known version so your provision won't explode when Chef releases new major versions. So for example:
django_config.omnibus.chef_version = "11.6.4"
I wanted to create a simple app using webapp2. Because I have Google App Engine installed, and I want to use it outside of GAE, I followed the instructions on this page: http://webapp-improved.appspot.com/tutorials/quickstart.nogae.html
This all went well, my main.py is running, it is handling requests correctly. However, I can't access resources directly.
http://localhost:8080/myimage.jpg or http://localhost:8080/mydata.json
always returns a 404 resource not found page.
It doesn't matter if I put the resources on the WebServer/Documents/ or in the folder where the virtualenv is active.
Please help! :-)
(I am on a Mac 10.6 with Python 2.7)
(Adapted from this question)
Looks like webapp2 doesn't have a static file handler; you'll have to roll your own. Here's a simple one:
import mimetypes
class StaticFileHandler(webapp2.RequestHandler):
def get(self, path):
# edit the next line to change the static files directory
abs_path = os.path.join(os.path.dirname(__file__), path)
try:
f = open(abs_path, 'r')
self.response.headers.add_header('Content-Type', mimetypes.guess_type(abs_path)[0])
self.response.out.write(f.read())
f.close()
except IOError: # file doesn't exist
self.response.set_status(404)
And in your app object, add a route for StaticFileHandler:
app = webapp2.WSGIApplication([('/', MainHandler), # or whatever it's called
(r'/static/(.+)', StaticFileHandler), # add this
# other routes
])
Now http://localhost:8080/static/mydata.json (say) will load mydata.json.
Keep in mind that this code is a potential security risk: It allows any visitors to your website to read everything in your static directory. For this reason, you should keep all your static files to a directory that doesn't contain anything you'd like to restrict access to (e.g. the source code).
I have a Pyramid web application managed with zc.buildout. In it, I need to read a file on disk, which is located in a sub-directory of buildout directory.
The problem is with determining the path to the file - I do not want to hard-code the absolute path and just providing a relative path does not work when serving the app in production (supposedly because the working directory is different).
So the promising "hooks" I am thinking about are:
the "root" buildout directory, which I can address in buildout.cfg as ${buildout:directory} - however, I can't figure out how can I "export" it so it can be accessed by the Python code
the location of the Paster's .ini file which starts the app
Like #MartijnPieters suggests in a comment on your own answer, I'd use collective.recipe.template to generate an entry in the .ini. I wondered myself how I could then access that data in my project, so I worked it out :-)
Let's work our way backwards to what you need. First in your view code where you want the buildout directory:
def your_view(request):
buildout_dir = request.registry.settings['buildout_dir']
....
request.registry.settings (see documentation) is a "dictonary-like deployment settings object". See deployment settings, that's the **settings that gets passed into your main method like def main(global_config, **settings)
Those settings are what's in the [app:main] part of your deployment.ini or production.ini file. So add the buildout directory there:
[app:main]
use = egg:your_app
buildout_dir = /home/you/wherever/it/is
pyramid.reload_templates = true
pyramid.debug_authorization = false
...
But, and this is the last step, you don't want to have that hardcoded path in there. So generate the .ini with a template. The template development.ini.in uses a ${partname:variable} expansion language. in your case you need${buildout:directory}:
[app:main]
use = egg:your_app
buildout_dir = ${buildout:dir}
# ^^^^^^^^^^^^^^^
pyramid.reload_templates = true
pyramid.debug_authorization = false
...
Add a buildout part in buildout.cfg to generate development.ini from development.ini.in:
[buildout]
...
parts =
...
inifile
...
[inifile]
recipe = collective.recipe.template
input = ${buildout:directory}/development.ini.in
output = ${buildout:directory}/development.ini
Note that you can do all sorts of cool stuff with collective.recipe.template. ${serverconfig:portnumber} to generate a matching port number in your production.ini and in your your_site_name.nginx.conf, for instance. Have fun!
If the path to the file relative to the buildout root or location of paster.ini is always the same, which it seems it is from your question, you could set it in paster.ini:
[app:main]
...
config_file = %(here)s/path/to/file.txt
Then access it from the registry as in Reinout's answer:
def your_view(request):
config_file = request.registry.settings['config_file']
Here's a rather clumsy solution I've devised:
In buildout.cfg I used extra-paths option of zc.recipe.egg to add the buildout directory to sys.path:
....
[webserver]
recipe = zc.recipe.egg:scripts
eggs = ${buildout:eggs}
extra-paths = ${buildout:directory}
then I put a file called app_config.py into the buildout directory:
# This remembers the root of the installation (similar to {buildout:directory}
# so we can import it and use where we need access to the filesystem.
# Note: we could use os.getcwd() for that but it feels kinda wonky
# This is not directly related to Celery, we may want to move it somewhere
import os.path
INSTALLATION_ROOT = os.path.dirname(__file__)
Now we can import it in our Python code:
from app_config import INSTALLATION_ROOT
filename = os.path.join(INSTALLATION_ROOT, "somefile.ext")
do_stuff_with_file(filename)
If anyone knows a nicer solution you're welcome :)
How do I run OpenERP on uWSGI?
I found this wsgi script online, but I'm not sure where to place it?
import openerp
try:
import uwsgi
uwsgi.port_fork_hook = openerp.wsgi.core.on_starting
except:
openerp.wsgi.core.on_starting()
# Equivalent of --load command-line option
openerp.conf.server_wide_modules = ['web']
# internal TODO: use openerp.conf.xxx when available
conf = openerp.tools.config
# Path to the OpenERP Addons repository (comma-separated for
# multiple locations)
conf['addons_path'] = '/home/openerp/addons/trunk,/home/openerp/web/trunk/addons'
# Optional database config if not using local socket
#conf['db_name'] = 'mycompany'
#conf['db_host'] = 'localhost'
#conf['db_user'] = 'foo'
#conf['db_port'] = 5432
#conf['db_password'] = 'secret'
# OpenERP Log Level
# DEBUG=10, DEBUG_RPC=8, DEBUG_RPC_ANSWER=6, DEBUG_SQL=5, INFO=20,
# WARNING=30, ERROR=40, CRITICAL=50
# conf['log_level'] = 20
# If --static-http-enable is used, path for the static web directory
#conf['static_http_document_root'] = '/var/www'
# vim:expandtab:smartindent:tabstop=4:softtabstop=4:shiftwidth=4:
application = openerp.wsgi.core.application
I installed OpenERP in a virtual environment in /var/www/openerp/venv and I can run it by calling $ openerp-server.
Thanks in advance.
you can just put the script file in the same directory with the openerp-server.py file.
however when I test it it doesnot work since gunicorn cannot find the openerp in the
import openerp sentence. the reason is that openerp is not installed as a python module to the system with the installation procedures around.
I think it will work when you do a openerp install with the DEB package. (when you make such install you should disable the start script so it will just work from gunicorn.
let me also make a test install and share the result.
I'm using the standard routing module with pylons to try and setup a default route for the home page of my website.
I've followed the instructions in the docs and here http://routes.groovie.org/recipes.html but when I try http://127.0.0.1:5000/ I just get the 'Welcome to Pylons' default page.
My config/routing.py file looks like this
from pylons import config
from routes import Mapper
def make_map():
"""Create, configure and return the routes Mapper"""
map = Mapper(directory=config['pylons.paths']['controllers'],
always_scan=config['debug'])
map.minimization = False
map.connect('/error/{action}', controller='error')
map.connect('/error/{action}/{id}', controller='error')
# CUSTOM ROUTES HERE
map.connect( '', controller='main', action='index' )
map.connect('/{controller}/{action}')
map.connect('/{controller}/{action}/{id}')
return map
I've also tried
map.connect( '/', controller='main', action='index' )
and (using http://127.0.0.1:5000/homepage/)
map.connect( 'homepage', controller='main', action='index' )
But nothing works at all. I know its reloading my config file as I used
paster serve --reload development.ini
to start the server
system info
$ paster --version
PasteScript 1.7.3 from /Library/Python/2.5/site-packages/PasteScript-1.7.3-py2.5.egg (python 2.5.1 (r251:54863, Feb 6 2009, 19:02:12))
You have to delete the static page (myapp/public/index.html). Static
files take priority due to the Cascade configuration at the end of
middleware.py.