Let's say I have a settings.py file in my app's root folder (/myapp/myapp/settings.py) with just a bunch of variables in it:
var1 = ''
var2 = ''
Can I automatically set one of those variables from the .ini file? I tried this:
myapp.settings.var1 = 'this is from development.ini'
But when I call var1 it is still the empty string:
import myapp.settings as s
print s.var1
I know that I can do var1 = config.get('myapp.settings.var1') in settings.py but that just doesn't seem as elegant to me :).
You can find relevant information at this page:
Getting Data from the Configuration File
.ini is one way only communication. Django uses settings.py which is a normal python module that can be manipulated on-the-fly (causing many obscure errors). Setup your settings in .ini and use config.get to access variables.
Related
I want to get all the environment variables starting with a specific prefix and save them (without prefix) into a dictionary. Is there any better way than getting all the os.environ variables and searching through them?
I also need those to be merged with a config file, so if you know any library in python which is like Viper in go (which handles both environment variables and config files and merging of them with priority), it will be a huge help.
UPDATE:
my configs are not simple app config, they are users config with some structures in it, so it's not a simple key value pair. It also might be in different formats, such as YAML, INI, JSON, etc.
I don't know any other solution that using os.environ to load your environment variable and loop through them. It would look something like:
import os
import re
# Replace PREF by the prefix you want
prefix="PREF"
myPattern = re.compile(r'{prefix}\w+'.format(prefix=prefix))
my_env_variables = {key.replace(prefix,''):val for key, val in os.environ.items() if myPattern.match(key)}
print(my_env_variables)
Regarding config files management, I warmly recommend the python-dotenv library: documentation
Searching through os.environ is going to be the best way.
from typing import Dict
import os
prefix: str = "MY_PREFIX_"
variables: Dict[str, str] = {}
for key, value in os.environ.items():
if key[:len(prefix)] == prefix:
variables[key[len(prefix):]] = value
If you are reading the env vars from .env file or from terminal than environs package is the best
export KAFKA_HOST=kafka
export KAFKA_PORT=9092
Python code to read all the env variables starting with KAFKA_
from environs import Env
env = Env()
env.read_env()
with env.prefixed("KAFKA_"):
print(env("HOST"))
print(env("PORT"))
I'm forced to keep my .env file in a non-standard path outside the root of my project (in a separate directory altogether).
Let's say I have my Django project in /var/projects/my_project, though I have my .env file in /opt/envs/my-project/.env where my SECRET_KEY is stored. In my settings.py file, I'd like to explicitly use the .env file at that path so that I can still do this:
from decouple import config
secret_key = config('SECRET_KEY')
I figured it out.
Instead of importing decouple.config and doing the usual config('FOOBAR'), create a new decouple.Config object using RepositoryEnv('/path/to/env-file').
from decouple import Config, RepositoryEnv
DOTENV_FILE = '/opt/envs/my-project/.env'
env_config = Config(RepositoryEnv(DOTENV_FILE))
# use the Config().get() method as you normally would since
# decouple.config uses that internally.
# i.e. config('SECRET_KEY') = env_config.get('SECRET_KEY')
SECRET_KEY = env_config.get('SECRET_KEY')
Hopefully this helps someone.
If you look at the decouple implementation, config is just a pre-instantiated AutoConfig:
config = AutoConfig()
But AutoConfig takes as optional argument search_path so we can do the following:
from decouple import AutoConfig
config = AutoConfig(search_path='/opt/envs/my-project')
Then you can do as usual:
secret_key = config('SECRET_KEY')
Now, django-decouple==2.1 supports having settings.ini and .env files in any parent directory of the project dir.
(And the old methods don't work anymore. - from decouple import Config, RepositoryEnv does not work, AutoConfig does not have search_path as parameter.)
This is convenient because you would want to keep the settings.ini in the project folder on your local machine and you would want to have clean checkouts on the staging/prod server, thus the settings.ini is better located outside the project folder.
We are working on an add-on that writes to a log file and we need to figure out where the default var/log directory is located (the value of the ${buildout:directory} variable).
Is there an easy way to accomplish this?
In the past I had a similar use case.
I solved it by declaring the path inside the zope.conf:
zope-conf-additional +=
<product-config pd.prenotazioni>
logfile ${buildout:directory}/var/log/prenotazioni.log
</product-config>
See the README of this product:
https://github.com/PloneGov-IT/pd.prenotazioni/
This zope configuration can then be interpreted with this code:
from App.config import getConfiguration
product_config = getattr(getConfiguration(), 'product_config', {})
config = product_config.get('pd.prenotazioni', {})
logfile = config.get('logfile')
See the full example
here: https://github.com/PloneGov-IT/pd.prenotazioni/blob/9a32dc6d2863b5bfb5843d441e652101406d9a2c/pd/prenotazioni/init.py#L17
Worth noting is the fact that the initial return avoids multiple logging if the init function is mistakenly called more than once.
Anyway, if you do not want to play with buildout and custom zope configuration, you may want to get the default event log location.
It is specified in the zope.conf. You should have something like this:
<eventlog>
level INFO
<logfile>
path /path/to/plone/var/log/instance.log
level INFO
</logfile>
</eventlog>
I was able to obtain the path with this code:
from App.config import getConfiguration
import os
eventlog = getConfiguration().eventlog
logpath = eventlog.handler_factories[0].instance.baseFilename
logfolder = os.path.split(logpath)[0]
Probably looking at in the App module code you will find a more straightforward way of getting this value.
Another possible (IMHO weaker) solution would be store (through buildout or your prefered method) the logfile path into an environment variable.
You could let buildout set it in parts/instance/etc/zope.conf in an environment variable:
[instance]
recipe = plone.recipe.zope2instance
environment-vars =
BUILDOUT_DIRECTORY ${buildout:directory}
Check it in Python code with:
import os
buildout_directory = os.environ.get('BUILDOUT_DIRECTORY', '')
By default you already have the INSTANCE_HOME environment variable, which might be enough.
I am working with an ini file where the default section is named 'default' and not 'DEFAULT'. Pythons ConfigParser seams to handle these sections ok with no problem. However, for whatever reason, they dont allow you to add a 'default' section or write to it.
In the code, they have hardcoded that you can not create a section of any case-insensitive version of 'DEFAULT'.
So if I can not add the section 'default' how do I write to it? The spec says you write to 'DEFAULT' since the section always exists. However, how to I write to 'default'?
I want to stay consistent with the way the file is already written, so I want the default section to be written in lowercase and not all uppercase.
If the file doesnt exist yet, I want to add the [default] section. Writing to section 'default' gives me a ValueError: Invalid section name: default
(note: writing to default works fine when I use an already correctly formatted file)
Also, I am willing to listen to suggestions for other configuration libraries I can use. But they must not be 3rd party (I shouldn't need to pip install them)
Here's a workaround (setting ConfigParser.DEFAULTSECT temporarily) :
import ConfigParser
import sys
config = ConfigParser.ConfigParser()
config.set(ConfigParser.DEFAULTSECT, 'name', 'value')
ORIG_DEFAULTSECT = ConfigParser.DEFAULTSECT # <---
ConfigParser.DEFAULTSECT = 'default'
try:
config.write(sys.stdout)
finally:
ConfigParser.DEFAULTSECT = ORIG_DEFAULTSECT # <---
Alternative: pass a StringIO with desired section name to readfp
import ConfigParser
import sys
from StringIO import StringIO
config = ConfigParser.ConfigParser()
config.readfp(StringIO('[default]')) # <----
config.set('default', 'name', 'value')
config.write(sys.stdout)
Update for 3.7
I don't know if this has always been the case but I'm currently dealing with configparser in 3.7 and came across this post. The easiest way to get this done as of this writing is to just set your default section name during the __init__ of configparser. Pay close attention to the character case in the examples below.
Example:
config = configparser.ConfigParser(default_section='default')
Then you can simply add to the default section with:
config.set('default', 'option', 'value')
or
config['default'] = {'option1':'value1', 'option2':'value2'}
Note:
The second method above (the dict-type method) will overwrite anything that already exists in the section, including the section name case if you're not careful. Be consistent on your naming convention to avoid this.
I think that the simplest approach is this...
from configparser import ConfigParser
config_parser = ConfigParser()
config_parser.default_section = "default"
I am still new to Python so keep that in mind when reading this.
I have been hacking away at an existing Python script that was originally "put" together by a few different people.
The script was originally designed to load it's 'configuration' using a module named "conf/config.py" which is basically Python code.
SETTING_NAME='setting value'
I've modified this to instead read it's settings from a configuration file using ConfigParser:
import ConfigParser
config_file_parser = ConfigParser.ConfigParser()
CONFIG_FILE_NAME = "/etc/service_settings.conf"
config_file_parser.readfp(open(r'' + CONFIG_FILE_NAME))
SETTING_NAME = config_file_parser.get('Basic', 'SETTING_NAME')
The problem I am having is how to specify the configuration file to use. Currently I have managed to get it working (somewhat) by having multiple TAC files and setting the "CONFIG_FILE_NAME" variable there using another module to hold the variable value. For example, I have a module 'conf/ConfigLoader.py":
global CONFIG_FILE_NAME
Then my TAC file has:
import conf.ConfigLoader as ConfigLoader
ConfigLoader.CONFIG_FILE_NAME = '/etc/service_settings.conf'
So the conf/config.py module now looks like:
import ConfigLoader
config_file_parser = ConfigParser.ConfigParser()
config_file_parser.readfp(open(r'' + ConfigLoader.CONFIG_FILE_NAME))
It works, but it requires managing two files instead of a single conf file. I attempted to use the "usage.Options" feature as described on http://twistedmatrix.com/documents/current/core/howto/options.html. So I have twisted/plugins/Options.py
from twisted.python import usage
global CONFIG_FILE_NAME
class Options(usage.Options):
optParameters = [['conf', 'c', 'tidepool.conf', 'Configuration File']]
# Get config
config = Options()
config.parseOptions()
CONFIG_FILE_NAME = config.opts['conf']
That does not work at all. Any tips?
I don't know if I understood your problem.
If you want to load the configuration from multiple locations you could pass a list of filenames to the configparser: https://docs.python.org/2/library/configparser.html#ConfigParser.RawConfigParser.read
If you were trying to make a generic configuration manager, you could create a class of a functions the receives the filename or you could use set the configuration file name in an environment variable and read that variable in your script using something like os.environ.get('CONFIG_FILE_NAME').