Is there a parser which reads and stores the types of data to write?
File format must to produce a readable.
Shelve does not offer.
Use the ConfigParser class to read configuration files in the ini file format:
http://docs.python.org/library/configparser.html#examples
The ini file format does not store the datatype of the values stored (you need to know them as you read the data back). You could overcome this limitation by encoding your values in json format:
import simplejson
from ConfigParser import ConfigParser
parser = ConfigParser()
parser.read('example.cfg')
value = 123
#or value = True
#or value = 'Test'
#Write any data to 'Section1->Foo' in the file:
parser.set('Section1', 'foo', simplejson.dumps(value))
#Now you can close the parser and start again...
#Retrieve the value from the file:
out_value = simplejson.loads(parser.get('Section1', 'foo'))
#It will match the input in both datatype and value:
value === out_value
Being json, the format of the stored value is human readable.
You could use the following function
def getvalue(parser, section, option):
try:
return parser.getint(section, option)
except ValueError:
pass
try:
return parser.getfloat(section, option)
except ValueError:
pass
try:
return parser.getbool(section, option)
except ValueError:
pass
return parser.get(section, option)
With configobj library, it becomes very simple.
import sys
import json
from configobj import ConfigObj
if(len(sys.argv) < 2):
print "USAGE: pass ini file as argument"
sys.exit(-1)
config = sys.argv[1]
config = ConfigObj(config)
Now you can use config as a dict to extract the desired configuration.
If you want to convert it to json, that's simple too.
config_json = json.dumps(config)
print config_json
Related
I want to read the json values from config json file. The json object path I use in my python code is like below:
jsonfile['Data'][0]['Tenants'][0]['TenPropertyGroup']
Now I want to pass this above path "['Data'][0]['Tenants'][0]['TenPropertyGroup']" from ini file to make sure I can make changes in ini file if the object path gets changed in json file.
My config.ini looks like:
[CONFIG]
TenPropertyGroup= ['Data'][0]['Tenants'][0]['TenPropertyGroups']
My python code after reading from ini file looks like
globalconfig = "config.ini"
config = configparser.ConfigParser()
config.read(globalconfig)
f = open(configfile, )
jsonfile = json.load(f)
TenPropertyGroup = config['CONFIG']['TenPropertyGroup']
TenPropertyGroups = (str(jsonfile ) + TenPropertyGroup)
But when I am reading in Python using configparser the above PropertyGroup is of string data type and I am not able to get the list from json file.
I am trying to read this ini properly from python code but not able to convert it to object.
I would suggest a different approach. You should avoid executing text read from a file for security reasons. If you use a different format for you ini file value, you could parse it and use the values to drill down into you json object. Here's a simple example:
path_from_ini = 'x/y/z'
json_dict = { 'x' : { 'y' : { 'z' : 42 } } }
parts = path_from_ini.split('/')
v = json_dict
# Drill down into the JSON dictonary
for p in parts:
v = v[p]
print(v)
I have a script that loops through file names in a json doc. In the event that there are no more filenames, I have a break statement that breaks the loop and stores the last read filename in a variable to print. Is there a way to save this variable so when I call the script again, it can use the last read filename as the starting point?
Thanks in advance
You could save it to a dotenv file or use configparser to save it to a ini and then retrieve it the same way.
To write just do
with open('yourfile.ini', 'wb') as config:
config.write(yourvariable_info)
This is a snippet I had to retrieve variables:
from configparser import ConfigParser
def config(filename='/home/pi/your/directory.ini', section='postgresql'):
# create a parser
parser = ConfigParser()
# read config file
parser.read(filename)
# get section, default to postgresql
db = {}# retrieves a dictionary
if parser.has_section(section):
params = parser.items(section)
for param in params:
db[param[0]] = param[1]
else:
raise Exception('Section {0} not found in the {1} file'.format(section, filename))
return db
I am having a lot of issues saving a YAML file to a python dictionary. I will display two routes I take below, both being suboptimal.
Yaml file:
modules:
module_1: True
module_2: True
module_3: False
scenarios:
constant:
start_date: "2018-09-30"
end_date: "2019-09-30"
adverse:
start_date: "2019-09-30"
end_date: "2022-09-30"
Route 1: Saving YAML file directly to a dictionary without specifying a loader which is now depreciated
import yaml
filepath = "C:\\user\\path\\file.yaml"
_dict = yaml.load(open(filepath))
print(type(_dict))
>>> <class 'dict'>
error message: Calling yaml.load() without loader=... is depreciated, as it is unsafe
Route 2: Loading in as a generator (which is not subscriptable)
import yaml
filepath = "C:\\user\\path\\file.yaml"
document = open(filepath, "r")
dictionary = yaml.safe_load_all(document)
print(type(dictionary)
>>> <generator object>
print(dictionary["modules"]["module_1"]
>>> generator object is not subscriptable
Is there a way I can import my yaml file into a dictionary safely? I wish to use the dictionary in my python project instead of creating global variables, etc.
Example:
if _dict["modules"]["module_1"]:
# Do something
Only calling without loader was depracted. You can always pass SafeLoader to the load function.
import yaml
with open(filepath, 'r') as stream:
dictionary = yaml.load(stream, Loader=yaml.SafeLoader)
This should return your dictionary.
edit:
And as for yaml.safe_load_all, you only need to call generator.__next__() to obtain the dictionary.
import yaml
filepath = "C:\\user\\path\\file.yaml"
document = open(filepath, "r")
generator = yaml.safe_load_all(document)
dictionary = generator.__next__()
I would recomend the first option for your use.
I am new on python. I am using a python script where I load a json file to set certain values to the script, but my idea is to import that file more dynamically using arguments (I think is the correct use), so I don´t need to always include the name of the json file in the python script, here is may code example:
import json
from pprint import pprint
with open("VariableSettings.json") as json_data:
data = json.load(json_data)
so my idea is to change the code: "with open("VariableSettings.json") as json_data" with args to open the json file dynamically.
I think that on command prompt I can use the command py test.py arg1 (this represent the file path).
So I know that probably my explanation is a bit confusing but if some can help I appreciate it.
You can use sys to do that. In the example below I created a file test.json with the content
{"foo": "bar"}
And modified your code as
import json
import sys
with open(sys.argv[1]) as json_data:
data = json.load(json_data)
print(data)
You need to call execute as
python test.py test.json
and the output will be
{'foo': 'bar'}
More details can be found in this other post
You can also use argparse:
import json
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-f", "--filename", required=True, type=str)
args = parser.parse_args()
with open(args.filename) as json_data:
data = json.load(json_data)
print(data)
Which can be called with the alias:
python test.py -f test.json
Or full argument name:
python test.py --filename test.json
And if you don't supply a file, you get:
usage: test.py [-h] -f FILENAME
test.py: error: the following arguments are required: -f/--filename
since I passed required=True. You can remove this if you want the argument to be optional.
Addtionally, you could also extend your program to check that if the JSON file has correct format by catching json.JSONDecodeError with try/except:
with open(args.filename) as json_data:
try:
data = json.load(json_data)
print(data)
except json.JSONDecodeError:
print('Invalid JSON format')
Use the sys module
Ex:
import sys
import json
from pprint import pprint
if len(sys.argv) < 2:
print("Input File Missing")
sys.exit()
with open(sys.argv[1]) as json_data:
data = json.load(json_data)
print(data)
To Run Use
python yourScriptName.py full_path_to.json
I'm trying to create a python script that will toggle my pi3 /boot/config.txt files gpu_mem=156 parameter for desktop and game mode. I have tried looking into ConfigParser but the config file I'm using uses a simpler format of simply:
var1=value0
var2=value1
I would appreciate some advice.
If you are using a text file to store your config you can parse it like
with open('config.txt', 'r') as config:
for line in config.read().splitlines():
var, val = line.split('=')
print var, val
But if you are storing your config that can be parsed by ConfigParser in the following format
[global]
var1=value0
var2=value1
Then you can parse it with ConfigParser like so
import ConfigParser
config_parser = ConfigParser.ConfigParser()
config_parser.read('config.txt')
print config_parser.get('global', 'var1')