I am working with an ini file where the default section is named 'default' and not 'DEFAULT'. Pythons ConfigParser seams to handle these sections ok with no problem. However, for whatever reason, they dont allow you to add a 'default' section or write to it.
In the code, they have hardcoded that you can not create a section of any case-insensitive version of 'DEFAULT'.
So if I can not add the section 'default' how do I write to it? The spec says you write to 'DEFAULT' since the section always exists. However, how to I write to 'default'?
I want to stay consistent with the way the file is already written, so I want the default section to be written in lowercase and not all uppercase.
If the file doesnt exist yet, I want to add the [default] section. Writing to section 'default' gives me a ValueError: Invalid section name: default
(note: writing to default works fine when I use an already correctly formatted file)
Also, I am willing to listen to suggestions for other configuration libraries I can use. But they must not be 3rd party (I shouldn't need to pip install them)
Here's a workaround (setting ConfigParser.DEFAULTSECT temporarily) :
import ConfigParser
import sys
config = ConfigParser.ConfigParser()
config.set(ConfigParser.DEFAULTSECT, 'name', 'value')
ORIG_DEFAULTSECT = ConfigParser.DEFAULTSECT # <---
ConfigParser.DEFAULTSECT = 'default'
try:
config.write(sys.stdout)
finally:
ConfigParser.DEFAULTSECT = ORIG_DEFAULTSECT # <---
Alternative: pass a StringIO with desired section name to readfp
import ConfigParser
import sys
from StringIO import StringIO
config = ConfigParser.ConfigParser()
config.readfp(StringIO('[default]')) # <----
config.set('default', 'name', 'value')
config.write(sys.stdout)
Update for 3.7
I don't know if this has always been the case but I'm currently dealing with configparser in 3.7 and came across this post. The easiest way to get this done as of this writing is to just set your default section name during the __init__ of configparser. Pay close attention to the character case in the examples below.
Example:
config = configparser.ConfigParser(default_section='default')
Then you can simply add to the default section with:
config.set('default', 'option', 'value')
or
config['default'] = {'option1':'value1', 'option2':'value2'}
Note:
The second method above (the dict-type method) will overwrite anything that already exists in the section, including the section name case if you're not careful. Be consistent on your naming convention to avoid this.
I think that the simplest approach is this...
from configparser import ConfigParser
config_parser = ConfigParser()
config_parser.default_section = "default"
Related
Is it possible to import a Python module from over the internet using the http(s), ftp, smb or any other protocol? If so, how? If not, why?
I guess it's about making Python use more the one protocol(reading the filesystem) and enabling it to use others as well. Yes I agree it would be many folds slower, but some optimization and larger future bandwidths would certainly balance it out.
E.g.:
import site
site.addsitedir("https://bitbucket.org/zzzeek/sqlalchemy/src/e8167548429b9d4937caaa09740ffe9bdab1ef61/lib")
import sqlalchemy
import sqlalchemy.engine
Another version,
I like this answer. when applied it, i simplified it a bit - similar to the look and feel of javascript includes over HTTP.
This is the result:
import os
import imp
import requests
def import_cdn(uri, name=None):
if not name:
name = os.path.basename(uri).lower().rstrip('.py')
r = requests.get(uri)
r.raise_for_status()
codeobj = compile(r.content, uri, 'exec')
module = imp.new_module(name)
exec (codeobj, module.__dict__)
return module
Usage:
redisdl = import_cdn("https://raw.githubusercontent.com/p/redis-dump-load/master/redisdl.py")
# Regular usage of the dynamic included library
json_text = redisdl.dumps(host='127.0.0.1')
Tip - place the import_cdn function in a common library, this way you could re-use this small function
Bear in mind It will fail when no connectivity to that file over http
In principle, yes, but all of the tools built-in which kinda support this go through the filesystem.
To do this, you're going to have to load the source from wherever, compile it with compile, and exec it with the __dict__ of a new module. See below.
I have left the actually grabbing text from the internet, and parsing uris etc as an exercise for the reader (for beginners: I suggest using requests)
In pep 302 terms, this would be the implementation behind a loader.load_module function (the parameters are different). See that document for details on how to integrate this with the import statement.
import imp
modulesource = 'a=1;b=2' #load from internet or wherever
def makemodule(modulesource,sourcestr='http://some/url/or/whatever',modname=None):
#if loading from the internet, you'd probably want to parse the uri,
# and use the last part as the modulename. It'll come up in tracebacks
# and the like.
if not modname: modname = 'newmodulename'
#must be exec mode
# every module needs a source to be identified, can be any value
# but if loading from the internet, you'd use the URI
codeobj = compile(modulesource, sourcestr, 'exec')
newmodule = imp.new_module(modname)
exec(codeobj,newmodule.__dict__)
return newmodule
newmodule = makemodule(modulesource)
print(newmodule.a)
At this point newmodule is already a module object in scope, so you don't need to import it or anything.
modulesource = '''
a = 'foo'
def myfun(astr):
return a + astr
'''
newmod = makemodule(modulesource)
print(newmod.myfun('bat'))
Ideone here: http://ideone.com/dXGziO
Tested with python 2, should work with python 3 (textually compatible print used;function-like exec syntax used).
This seems to be a use case for a self-written import hook. Look up in PEP 302 how exactly they work.
Essentially, you'll have to provide a finder object which, in turn, provides a loader object. I don't understand the process at the very first glance (otherwise I'd be more explicit), but the PEP contains all needed details for implementing the stuff.
As glglgl's has it this import hook has been implemented for Python2 and Python3 in a module called httpimport.
It uses a custom finder/loader object to locate resources using HTTP/S.
Additionally, the import_cdn function in Jossef Harush's answer is almost identically implemented in httpimport's github_repo, and bitbucket_repo functions.
#Marcin's answer contains a good portion of the code of the httpimport's loader class.
We are working on an add-on that writes to a log file and we need to figure out where the default var/log directory is located (the value of the ${buildout:directory} variable).
Is there an easy way to accomplish this?
In the past I had a similar use case.
I solved it by declaring the path inside the zope.conf:
zope-conf-additional +=
<product-config pd.prenotazioni>
logfile ${buildout:directory}/var/log/prenotazioni.log
</product-config>
See the README of this product:
https://github.com/PloneGov-IT/pd.prenotazioni/
This zope configuration can then be interpreted with this code:
from App.config import getConfiguration
product_config = getattr(getConfiguration(), 'product_config', {})
config = product_config.get('pd.prenotazioni', {})
logfile = config.get('logfile')
See the full example
here: https://github.com/PloneGov-IT/pd.prenotazioni/blob/9a32dc6d2863b5bfb5843d441e652101406d9a2c/pd/prenotazioni/init.py#L17
Worth noting is the fact that the initial return avoids multiple logging if the init function is mistakenly called more than once.
Anyway, if you do not want to play with buildout and custom zope configuration, you may want to get the default event log location.
It is specified in the zope.conf. You should have something like this:
<eventlog>
level INFO
<logfile>
path /path/to/plone/var/log/instance.log
level INFO
</logfile>
</eventlog>
I was able to obtain the path with this code:
from App.config import getConfiguration
import os
eventlog = getConfiguration().eventlog
logpath = eventlog.handler_factories[0].instance.baseFilename
logfolder = os.path.split(logpath)[0]
Probably looking at in the App module code you will find a more straightforward way of getting this value.
Another possible (IMHO weaker) solution would be store (through buildout or your prefered method) the logfile path into an environment variable.
You could let buildout set it in parts/instance/etc/zope.conf in an environment variable:
[instance]
recipe = plone.recipe.zope2instance
environment-vars =
BUILDOUT_DIRECTORY ${buildout:directory}
Check it in Python code with:
import os
buildout_directory = os.environ.get('BUILDOUT_DIRECTORY', '')
By default you already have the INSTANCE_HOME environment variable, which might be enough.
I am still new to Python so keep that in mind when reading this.
I have been hacking away at an existing Python script that was originally "put" together by a few different people.
The script was originally designed to load it's 'configuration' using a module named "conf/config.py" which is basically Python code.
SETTING_NAME='setting value'
I've modified this to instead read it's settings from a configuration file using ConfigParser:
import ConfigParser
config_file_parser = ConfigParser.ConfigParser()
CONFIG_FILE_NAME = "/etc/service_settings.conf"
config_file_parser.readfp(open(r'' + CONFIG_FILE_NAME))
SETTING_NAME = config_file_parser.get('Basic', 'SETTING_NAME')
The problem I am having is how to specify the configuration file to use. Currently I have managed to get it working (somewhat) by having multiple TAC files and setting the "CONFIG_FILE_NAME" variable there using another module to hold the variable value. For example, I have a module 'conf/ConfigLoader.py":
global CONFIG_FILE_NAME
Then my TAC file has:
import conf.ConfigLoader as ConfigLoader
ConfigLoader.CONFIG_FILE_NAME = '/etc/service_settings.conf'
So the conf/config.py module now looks like:
import ConfigLoader
config_file_parser = ConfigParser.ConfigParser()
config_file_parser.readfp(open(r'' + ConfigLoader.CONFIG_FILE_NAME))
It works, but it requires managing two files instead of a single conf file. I attempted to use the "usage.Options" feature as described on http://twistedmatrix.com/documents/current/core/howto/options.html. So I have twisted/plugins/Options.py
from twisted.python import usage
global CONFIG_FILE_NAME
class Options(usage.Options):
optParameters = [['conf', 'c', 'tidepool.conf', 'Configuration File']]
# Get config
config = Options()
config.parseOptions()
CONFIG_FILE_NAME = config.opts['conf']
That does not work at all. Any tips?
I don't know if I understood your problem.
If you want to load the configuration from multiple locations you could pass a list of filenames to the configparser: https://docs.python.org/2/library/configparser.html#ConfigParser.RawConfigParser.read
If you were trying to make a generic configuration manager, you could create a class of a functions the receives the filename or you could use set the configuration file name in an environment variable and read that variable in your script using something like os.environ.get('CONFIG_FILE_NAME').
Is it possible to import a Python module from over the internet using the http(s), ftp, smb or any other protocol? If so, how? If not, why?
I guess it's about making Python use more the one protocol(reading the filesystem) and enabling it to use others as well. Yes I agree it would be many folds slower, but some optimization and larger future bandwidths would certainly balance it out.
E.g.:
import site
site.addsitedir("https://bitbucket.org/zzzeek/sqlalchemy/src/e8167548429b9d4937caaa09740ffe9bdab1ef61/lib")
import sqlalchemy
import sqlalchemy.engine
Another version,
I like this answer. when applied it, i simplified it a bit - similar to the look and feel of javascript includes over HTTP.
This is the result:
import os
import imp
import requests
def import_cdn(uri, name=None):
if not name:
name = os.path.basename(uri).lower().rstrip('.py')
r = requests.get(uri)
r.raise_for_status()
codeobj = compile(r.content, uri, 'exec')
module = imp.new_module(name)
exec (codeobj, module.__dict__)
return module
Usage:
redisdl = import_cdn("https://raw.githubusercontent.com/p/redis-dump-load/master/redisdl.py")
# Regular usage of the dynamic included library
json_text = redisdl.dumps(host='127.0.0.1')
Tip - place the import_cdn function in a common library, this way you could re-use this small function
Bear in mind It will fail when no connectivity to that file over http
In principle, yes, but all of the tools built-in which kinda support this go through the filesystem.
To do this, you're going to have to load the source from wherever, compile it with compile, and exec it with the __dict__ of a new module. See below.
I have left the actually grabbing text from the internet, and parsing uris etc as an exercise for the reader (for beginners: I suggest using requests)
In pep 302 terms, this would be the implementation behind a loader.load_module function (the parameters are different). See that document for details on how to integrate this with the import statement.
import imp
modulesource = 'a=1;b=2' #load from internet or wherever
def makemodule(modulesource,sourcestr='http://some/url/or/whatever',modname=None):
#if loading from the internet, you'd probably want to parse the uri,
# and use the last part as the modulename. It'll come up in tracebacks
# and the like.
if not modname: modname = 'newmodulename'
#must be exec mode
# every module needs a source to be identified, can be any value
# but if loading from the internet, you'd use the URI
codeobj = compile(modulesource, sourcestr, 'exec')
newmodule = imp.new_module(modname)
exec(codeobj,newmodule.__dict__)
return newmodule
newmodule = makemodule(modulesource)
print(newmodule.a)
At this point newmodule is already a module object in scope, so you don't need to import it or anything.
modulesource = '''
a = 'foo'
def myfun(astr):
return a + astr
'''
newmod = makemodule(modulesource)
print(newmod.myfun('bat'))
Ideone here: http://ideone.com/dXGziO
Tested with python 2, should work with python 3 (textually compatible print used;function-like exec syntax used).
This seems to be a use case for a self-written import hook. Look up in PEP 302 how exactly they work.
Essentially, you'll have to provide a finder object which, in turn, provides a loader object. I don't understand the process at the very first glance (otherwise I'd be more explicit), but the PEP contains all needed details for implementing the stuff.
As glglgl's has it this import hook has been implemented for Python2 and Python3 in a module called httpimport.
It uses a custom finder/loader object to locate resources using HTTP/S.
Additionally, the import_cdn function in Jossef Harush's answer is almost identically implemented in httpimport's github_repo, and bitbucket_repo functions.
#Marcin's answer contains a good portion of the code of the httpimport's loader class.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Dynamic module import in Python
I intend to make a suite of files at some point soon, and the best way to organize it is to have a list, that list will be at the very top of a file, and after it will come a ridiculous amount of code to handle what that list controls and how it operates. I'm looking to write said list only once, and said list is a list of folder and file names in this format:
[(folder/filename, bool, bool, int), (folder/filename, bool, bool, int)]
As you can see, folder/filename are the same (sort of). File name is folder name with .py on the end, but doing import XXX you don't need to do import XXX.py, so I don't see this causing an issue.
The problem I'm facing is importing using this method...
for (testName, auto, hardware, bit) in testList:
print(testName)
paths = "\\" + testName
print paths
addpath(paths)
sys.modules[testName] = testName # One of a few options I've seen suggested on the net
print("Path Added")
test = testName + ".Helloworld()"
eval(test)
So for each test I have, print the name, assemble a string which contains the path ("\\testName"), for this example, print the test path, then add the path to the list (sys.path.append(path)), then print to confirm it happened, then assemble a string which will be executed by eval for the tests main module and eventually eval it.
As you can see, I'm currently having to have a list of imports at the top. I can't simply do import testName (the contents of testName are the name of the module I wish to import), as it will try to find a module called testName, not a module called the contents of testName.
I've seen a few examples of where this has been done, but can't find any which work in my circumstances. If someone could literally throw a chunk of code which does it that would be wonderful.
I'd also request that I'm not hung, drawn, nor quartered for use of eval, it is used in a very controlled environment (the list through which it cycles is within the .py file, so no "end user" should mess with it).
Not sure if I understood everything correctly, but you can import a module dynamically using __import__:
mod = __import__(testName)
mod.HelloWorld()
Edit: I wasn't aware that the use of __import__ was discouraged by the python docs for user code: __import__ documentation (as noted by Bakuriu)
This should also work and would be considered better style:
import importlib
mod = importlib.import_module(testName)
mod.HelloWorld()
Never, ever, ever mess with sys.modules directly if you don't know exactly what you are doing.
There are a lot of ways to do what you want:
The build-in __import__ function
Using imp.load_module
Using importlib.import_module
I'd avoid using __import__ directly, and go for importlib.import_module(which is also suggested at the end of the documentation of __import__).
Add the path where module resides to sys.path. Import the module using __import__ function which accepts a string variable as module name.
import sys
sys.path.insert(0, mypath) # mypath = path of module to be imported
imported_module = __import__("string_module_name") # __import__ accepts string
imported_module.myfunction() # All symbols in mymodule are now available normally