I am using Argparse for fetching the necessary value from config file.
For example:
python arg.py --event_conf=/opt/open-stack-tools/track_events.conf --openstack_conf=/etc/nova/nova.conf
I need to fetch value from two different files.
I can be able to get the results as needed for one local config file.
But In case of fetching the necessary values from nova.conf file, it results in following error:
Traceback (most recent call last):
File "arg.py", line 36, in <module>
oslo_messaging_rabbit= dict(config.items("oslo_messaging_rabbit"))
File "/usr/lib/python2.7/ConfigParser.py", line 655, in items
for option in options]
File "/usr/lib/python2.7/ConfigParser.py", line 691, in _interpolate
self._interpolate_some(option, L, rawval, section, vars, 1)
File "/usr/lib/python2.7/ConfigParser.py", line 723, in _interpolate_some
option, section, rest, var)
ConfigParser.InterpolationMissingOptionError: Bad value substitution:
section: [oslo_messaging_rabbit]
option : logging_exception_prefix
key : color
rawval : %(asctime)s.%(msecs)03d TRACE %(name)s %(instance)s
Is there any way to fix the same.
I have copied the necessary contents and created a new local file, I can see that it is working fine.
When I am using the nova.conf file it results in error.
I can't change the file which I am using.
So I need a fix for the particular error.
Note:
Adding more details as needed:
parser.add_argument("-c", "--event_conf",
help="Specify config file 1", metavar="FILE")
args1, remaining_argv1 = parser.parse_known_args()
parser.add_argument("-o", "--openstack_conf",
help="Specify config file 2", metavar="FILE")
args2, remaining_argv2 = parser.parse_known_args()
if args1.event_conf:
config = ConfigParser.SafeConfigParser()
print config.read([args1.event_conf])
config.read([args1.event_conf])
configdetails_section1 = dict(config.items("configdetails_section1"))
I found the solution for the same.
Actually issue was with the configparser which I have used.
Instead of SafeConfigParser I changed it to RawConfigParser.
Then I can be able to see that it is working fine.
Interpolation allows you to refer to values of the configuration while defining others. For example, in this config file:
[bug_tracker]
protocol = http
server = localhost
port = 8080
url = %(protocol)s://%(server)s:%(port)s/bugs/
username = dhellmann
password = SECRET
if you read like this:
from ConfigParser import SafeConfigParser
parser = SafeConfigParser()
parser.read('interpolation.ini')
print 'value =', parser.get('bug_tracker', 'url')
you will get:
value = http://localhost:8080/bugs/
that can be very useful, the problem seems that that you pass some values in run time. I mean you need the format string to actually substitute the values by yourself.
You can use RawConfigParser instead of SafeConfigParser but then you loose all the interpolations.
Instead you can suppress the interpolation for one specific value:
print 'value =', parser.get('bug_tracker', 'url', raw=True)
and the result would be:
value = %(protocol)s://%(server)s:%(port)s/bugs/
There is also the possibility that you need to combine interpolated values with some of them given in evaluation time. For example if you want to give the user in evaluation time, you can also include it into the config expression:
[bug_tracker]
protocol = http
server = localhost
port = 8080
url = %(protocol)s://%(user)s#%(server)s:%(port)s/bugs/
username = dhellmann
password = SECRET
and then you need to make something like this:
from ConfigParser import SafeConfigParser
parser = SafeConfigParser()
parser.read('interpolation.ini')
parser.set('bug_tracker', 'user', 'admin')
print 'value =', parser.get('bug_tracker', 'url')
and you'll get:
value = http://admin#localhost:8080/bugs/
Sorry that I didn't use your example. I did take the one in the documentation of another project. See the section: Combining Values with Interpolation
Related
I was trying to read a boolean value using config parser, It finds the section fine but it fails finding the option for some reason.
configparser.NoOptionError: No option 'firsttime' in section: 'DEFAULT'
In the INI i have:
[DEFAULT]
"firsttime" = "true"
And in my main folder i have:
import configparser
config = configparser.ConfigParser()
print(config.getboolean('DEFAULT', 'firsttime'))
ConfigParser() constructs the configuration parsing object, but doesn't read in any configuration file. To do that you have to invoke its read method, e.g. config.read(<your_config_file>). So if your config file was named example.ini, your code would look like:
import configparser
config = configparser.ConfigParser()
config.read('example.ini')
print(config.getboolean('DEFAULT', 'firsttime'))
As a side note putting quotations in the init file will screw up the parsing of it. So you'll want your init file to look like:
[DEFAULT]
firsttime = true
He is the whole code:
import ifcfg
import json
for name, interface in ifcfg.interfaces().items():
# do something with interface
print (interface['device']) # Device name
print (interface['inet']) # First IPv4 found
print (interface['inet4']) # List of ips
print(ifcfg.default_interface())
When I removeprint(ifcfg.default_interface()) the program runs fine.
Here is the error message I get:
Traceback: File "C:\Users\User\PycharmProjects\pythonProject\IFCONFIG.py", line 15, in
print(ifcfg.default_interface())
return Parser(ifconfig=ifconfig)._default_interface(route_output=route_output)
AttributeError: 'WindowsParser' object has no attribute '_default_interface'. Did you mean: 'default_interface'?
From what I was able to gather this is a Windows specific issue. Ifcfg class WindowsParser currently has no method _default_interface.
You can try and write your own here's the path.
C:..\Programs\Python\Python310\lib\site-packages\ifcfg\parser.py
For reference here's the UNUX version of _default_interface in parser.py
def _default_interface(self, route_output=None):
"""
:param route_output: For mocking actual output
"""
if not route_output:
out, __, __ = exec_cmd('/sbin/route -n')
lines = out.splitlines()
else:
lines = route_output.split("\n")
for line in lines[2:]:
line = line.split()
if '0.0.0.0' in line and \
'UG' in line:
iface = line[-1]
return self.interfaces.get(iface, None)
Alternatively use interfaces() this is generally sufficient for most users.
import ifcfg
import json
for name, interface in ifcfg.interfaces().items():
# do something with interface
print (interface['device']) # Device name
print (interface['inet']) # First IPv4 found
print (interface['inet4']) # List of ips
print(ifcfg.interfaces())
Further reading: https://github.com/ftao/python-ifcfg/issues/25
We have 3 different Datacenter environments, for reference, let us say, US, CA and IN. All three have a puppetdb master different to the other 2. So while automating some stuff, I want to a config.json file like below and reference it from the main code based on the parameters passed:
// config.json
{
"DEFAULT": {
"LOGFILE": "log/get_hname.log",
"LOCKDIR": "include/LOCK",
"NOOPRUN": "0"
},
"US": {
"PDB": "puppetdb100.us",
"VRFFILE": "include/vrf.txt",
"FQDN": "us.livevox"
},
"CA": {
"PDB": "puppet.ca.livevox.net",
"FQDN": "ca.livevox"
},
"IN": {
"PDB": "puppet100.in.livevox.net",
"FQDN": "in.livevox"
}
}
Now, for the main script, where I am trying to use a variable called "myenv", which would be one of US, or CA or IN to refer the key in the config which is of dict type. But I keep getting the error as below:
Traceback (most recent call last):
File "./get_hname.py", line 94, in <module>
print (config[myenv.upper()]['PDB'])
KeyError: 'NONE'
The script itself for your reference below:
#!/usr/bin/env python
import argparse
import json
import sys
import os
import logging
import time
from argparse import ArgumentParser
# Where Config File lives --------
CONFFILE = "include/get-hname-cfg.json"
# - DO NOT EDIT BELOW THIS LINE --
with open(CONFFILE, 'r') as f:
config = json.load(f)
logfile = config['DEFAULT']['LOGFILE']
myarguments = argparse.ArgumentParser(description="Arguments for Name Builder", usage="%(prog)s [options]")
myarguments.add_argument("-e", "--env", type=str, metavar="environment", nargs='*', help="Environment")
myarguments.add_argument("-t", "--type", type=str, metavar="servertype", nargs='*', help="Server type")
myarguments.add_argument("-n", "--noop", action="store_true", help="Stimulate the whole run, but don't execute. Similar to noop")
myarguments.parse_args()
args = myarguments.parse_args()
if not args.env and not args.type:
myarguments.print_help(sys.stderr)
myenv = str(args.env).lower()
pdbhost = "config" +myenv.upper()+ "['PDB']"
print ("%s" %(pdbhost))
if config[myenv.upper()]['PDB'] in globals():
puppetdbhost = config[myenv.upper()]['PDB']
How can I use the string type variable within a dict as a key?
EDIT : Please note, all necessary indentations which maybe missing in my question, have all been taken care of, since I'm using PyCharm.
Traceback (most recent call last):
File "./get_hname.py", line 94, in <module>
print (config[myenv.upper()]['PDB'])
KeyError: 'NONE'
It's mean that myenv variable is string 'None' in that moment. Check it.
you need to modify your add_arguments remove nargs='*' which is giving env in list
form see below example:
myarguments.add_argument("-e", "--env", type=str, metavar="environment", help="Environment")
refer this document for more info.
OK folks, I found what I was doing wrong. The arguments I was using
type=str and nargs='*'
which is why I was facing a type mismatch and getting the output as ['US'] instead of US (or CA or IN). Once I removed that, things are working fine.
Thank you all for your pointers. That helped.
I have a fairly simple use-case but i'm not understanding the error message i'm receiving.
I'm using the requests and pyral modules, pyral (http://pyral.readthedocs.io/en/latest/interface.html#) is really just a wrapper for Rally's Restful api. My goal is to get a file (attachment) from a Rally (a CA product) UserStory and store it to a local file system.
For context, here is my environment setup (authenticate to Rally and create an object). I've obviously removed authentication information.
from pyral import Rally, rallyWorkset
options = [arg for arg in sys.argv[1:] if arg.startswith('--')]
args = [arg for arg in sys.argv[1:] if arg not in options]
server, user, password, apikey, workspace, project = rallyWorkset(options)
rally = Rally(server='rally1.rallydev.com',
user='**********', password='***********',
apikey="**************",
workspace='**************', project='**************',
server_ping=False)
After that I get a response object for just one user story (see the query for US845), i do this just to simplify the problem.
r = rally.get('UserStory', fetch = True, projectScopeDown=True, query = 'FormattedID = US845')
and then I use the built-in iterator to get the user story from the RallyRESTResponse object.
us = r.next()
from there it feels like I should be able to easily use the getAttachment() method that accepts a artifact (us) and filename (name of an attachment). I'm able to use getAttachmentNames(us) to return a list of attachment names. The issue arrises when i try something like
attachment_names = rally.getAttachmentNames(us) #get attachments for this UserStory
attachment_file = rally.getAttachment(us, attachment_names[0]) #Try to get the first attachment
returns an error like this
Traceback (most recent call last):
File "<ipython-input-81-a4a342a59c5a>", line 1, in <module>
attachment_file = rally.getAttachment(us, attachment_names[0])
File "C:\Miniconda3\lib\site-packages\pyral\restapi.py", line 1700, in getAttachment
att.Content = base64.decodebytes(att_content.Content) # maybe further txfm to Unicode ?
File "C:\Miniconda3\lib\base64.py", line 552, in decodebytes
_input_type_check(s)
File "C:\Miniconda3\lib\base64.py", line 520, in _input_type_check
raise TypeError(msg) from err
TypeError: expected bytes-like object, not str
I receive a similar error if i try to use
test_obj = rally.getAttachments(us)
Which returns an error like this:
Traceback (most recent call last):
File "<ipython-input-82-06a8cd525177>", line 1, in <module>
rally.getAttachments(us)
File "C:\Miniconda3\lib\site-packages\pyral\restapi.py", line 1721, in getAttachments
attachments = [self.getAttachment(artifact, attachment_name) for attachment_name in attachment_names]
File "C:\Miniconda3\lib\site-packages\pyral\restapi.py", line 1721, in <listcomp>
attachments = [self.getAttachment(artifact, attachment_name) for attachment_name in attachment_names]
File "C:\Miniconda3\lib\site-packages\pyral\restapi.py", line 1700, in getAttachment
att.Content = base64.decodebytes(att_content.Content) # maybe further txfm to Unicode ?
File "C:\Miniconda3\lib\base64.py", line 552, in decodebytes
_input_type_check(s)
File "C:\Miniconda3\lib\base64.py", line 520, in _input_type_check
raise TypeError(msg) from err
TypeError: expected bytes-like object, not str
It seems that i'm fundamentally misunderstanding the parameters that this method requires? Has anyone been able to do this successfully before? For what it's worth i have no issues using the addAttachment() method with a workflow similar to the above. I've tried converting the filename (string) with the bytes() method to utf-8 but that didn't help.
I've also looked at this example in the pyral source, but i receive exactly the same error when trying to execute that.
https://github.com/klehman-rally/pyral/blob/master/examples/get_attachments.py
It looks like the issue in restapi.py script - there is no decodebytes method in base64 library:
att.Content = base64.decodebytes(att_content.Content)
All available methods are described at:
RFC 3548: Base16, Base32, Base64 Data Encodings
So, workaround is to replace decodebytes by base64.b64decode in restapi.py. At least, it works me.
E.g. location at Mac OS X:
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyral/restapi.py
I have used the below code to get all attachment since getAttachments is not working as expected. it will create a file in the current dir with the same name.
import sys
import string
import base64
from pyral import rallyWorkset, Rally,RallyRESTResponse
rally = Rally(server, user=USER_NAME, password=PASSWORD, workspace=workspace, project=project)
criterion = 'FormattedID = US57844'
response = rally.get('HierarchicalRequirement', query=criterion, order="FormattedID",pagesize=200, limit=400, projectScopeDown=True)
artifact = response.next()
context, augments = rally.contextHelper.identifyContext()
for att in artifact.Attachments:
resp = rally._getResourceByOID(context, 'AttachmentContent', att.Content.oid, project=None)
if resp.status_code not in [200, 201, 202]:
break
res = RallyRESTResponse(rally.session, context, "AttachmentContent.x", resp, "full", 1)
if res.errors or res.resultCount != 1:
print("breaking the for loop")
att_content = res.next()
cont = att_content.Content
x = base64.b64decode(cont)
output = open(att.Name, 'wb')
output.write(x)
So, I am passing a environment variable from bash to python;
#!/usr/bin/env python2
import os
#connect("weblogic", "weblogic", url=xxx.xxx.xxx.xxx:xxxx)
os.environ['bash_variable']
via wlst.sh I can print exported bash_variable, but how do I execute stored variable? Basically, I am trying to remove the original connect statement and pass a variable that has said information. Thanks
Question though, why wouldn't you called the script with the variable as an argument and use sys.argv[] ?
By example something like this.
import os
import sys
import traceback
from java.io import *
from java.lang import *
wlDomain = sys.argv[1]
wlDomPath = sys.argv[2]
wlNMHost = sys.argv[3]
wlNMPort = sys.argv[4]
wlDPath="%s/%s" %(wlDomPath,wlDomain)
wlNMprop="/apps/bea/wls/scripts/.shadow/NM.prop"
try:
print "Connection to Node Manager"
print ""
loadProperties(wlNMprop)
nmConnect(username=NMuser,password=NMpass,host=wlNMHost,port=wlNMPort,domainName=wlDomain,domainDir=wlDPath,mType='ssl',verbose='true')
except:
print "Fatal Error : No Connection to Node Manager"
exit()
print "Connected to Node Manager"
The NM.prop file is a 600 file with the username/password for the NM.
EDIT :
So from what I understand you want to do something like this :
URLS = ['t3s://Host1:Port1','t3s://Host2:Port2','t3s://Host3:Port3']
for urls in URLS:
connect('somebody','password',urls)
{bunch of commands}
disconnect()
And the values of the list URLS would be define by the environment.
The way I see it you have 3 choices :
Have 1 script per environment, more or less identical save for the URLS list
Have 1 script but with a conditionnal branching on sys.argv[1] (the environment as a parameter) and create the list there.
Have 1 script which use a parameter file for each environment according to the environment. Each parameter file containing the list in question.
Something like that :
propENV = sys.argv[1]
propPath = "/path1/path2"
propFile = "%s/%s" %(propPath,propENV)
loadProperties(propFile)
I would probably use the properties file option myself as it is more flexible from an operational standpoint...at least IMHO.