I want to have a JSON config file where I can reference values internally. For example, consider this JSON config file at bellow:
{
"hdfs-base":"/user/SOME_HDFS_USER/SOME_PROJECT"
,"incoming-path":"$hdfs-base/incoming"
,"processing-path":"$hdfs-base/processing"
,"processed-path":"$hdfs-base/processed"
}
The main idea is to leverage values already stored in json object. In this case, replacing '$hdfs-base' to 'hdfs-base' attribute's value. Do you know something that do that already? I don't wanna use ConfigParser module because I want to use JSON.
Thanks!
Loop over your values, and substitute the keys if there's a match:
import json
js = open('input.json').read()
data = json.loads(js)
for k, v in data.items():
for key in data.keys():
if key in v:
data[k] = v.replace("$" + key, data[key])
BEFORE
hdfs-base /user/SOME_HDFS_USER/SOME_PROJECT
incoming-path $hdfs-base/incoming
processing-path $hdfs-base/processing
processed-path $hdfs-base/processed
AFTER
hdfs-base /user/SOME_HDFS_USER/SOME_PROJECT
incoming-path /user/SOME_HDFS_USER/SOME_PROJECT/incoming
processing-path /user/SOME_HDFS_USER/SOME_PROJECT/processing
processed-path /user/SOME_HDFS_USER/SOME_PROJECT/processed
REPL: https://repl.it/repls/NumbMammothTelephone
Related
Lets say I have a file named tools.py. From tools.py I have a dictionary with sample key/value pairs:
TOOLS ={0x100: "Hammer",
0x101: "Screw",
0x102: "Nail",
0x103: "Wedge",
0x104: "Drill",
0x105: "Socket",
}
I have another file named file.py where I import tools.py. I have a value of 0x101 and I want to access tools.py dictionary TOOLS such that I can create a new dictionary with 0x101 key being Screw. In other words I have defined
import tools
value = 10
placeholder = 0x101
new_dict = {}
I want to add the entry Screw:10 by looking up 0x101 in tools.py and getting the value Screw. How do I do this in python?
Here you have it as an answer:
import tools
new_dict[tools.TOOLS[placeholder]] = value
Here is another way to do it with the format that you had, if you want to add more to the dictionary.
from tools import TOOLS
value = 10
placeholder = 0x101
new_dict = {}
new_dict.update({TOOLS[placeholder]: value})
print(new_dict)
This way, you look up the placeholder you had in TOOLS, and use it as a key in the new_dict. For the value, you use your "value" variable.
How to extract 41677?
My json:
{"41677":{"key":"ilya","premium":"true"}}
My code:
params={"id": "ilya", "fmt": "json"}
r=requests.get("somesite", params=params )
data=json.loads(r.text)
By using loads, your JSON string will be converted to a dictionary which maps keys to values.
Since you need the key 41677, you can simply call data.keys()[0] to retrieve the first key of your dictionary.
EDIT:
Also, if you have a list of that JSON structure, you can iterate over the keys and values using the items function, like so:
for key, value in data.items():
print key # 41677
print value # {"key":"ilya","premium":"true"}
By using Requests' built-in json attribute:
data = requests.get("somesite", params=params ).json().keys()[0]
assuming the json it returns is {"41677":{"key":"ilya","premium":"true"}}:
>>>print data
"41677"
import json
s = {"41677":{"key":"ilya","premium":"true"}}
d = json.dumps(s)
l = json.loads(d)
l.keys()
I know its kind of trivial thing to ask...but i'm a new bee to python.
Here is a json string
reply = {u'f': [{u'v': u'0'}]}
How to parse out value 0 from it using python.
i tried like
count = reply['rows'][0]['v']
but it not working
count = reply['f'][0]['v'] should work I believe.
reply is a dictionary. As such, you need to use the dictionary keys to access the data. In this case, the key is 'f', not 'rows'.
If you had valid JSON, you could use the simplejson module:
from simplejson import loads, dumps
my_dict = loads(my_json_serialized_string)
and then you can access the python dict, e.g.,:
print my_dict.items()
print my_dict.keys()
print my_dict.values()
#lets assume 'rows' exists as a key, and the value is a list, and the first item of that list is a dict that contains the key 'v':
print my_dict['rows'][0]['v']
and you can even change the dict, and serialize it as a valid JSON string:
my_dict['my_key'] = 'my_value'
my_other_json_serialized_string = dumps(my_dict)
I have a json file ( ~3Gb ) that I need to load into mongodb. Quite a few of the json keys contain a . (dot), which causes the load into mongodb to fail. I want to the load the json file, and edit the key names in the process, say replace the dot with an empty space. Using the following python code
import json
def RemoveDotKey(dataPart):
for key in dataPart.iterkeys():
new_key = key.replace(".","")
if new_key != key:
newDataPart = deepcopy(dataPart)
newDataPart[new_key] = newDataPart[key]
del newDataPart[key]
return newDataPart
return dataPart
new_json = json.loads(data, object_hook=RemoveDotKey)
The object_hook called RemoveDotKey should iterate over all the keys, it a key contains a dot, create a copy, replace the dot with a space, and return the copy. Created a copy of dataPart, since not sure if I can iterate over dataPart's keys and insert/delete key value pairs at the same time.
There seems to be an error here, all the json keys with a dot in them are not getting edited. I am not very sure how json.load works. Also am new to python ( been using it for less than a week )
You almost had it:
import json
def remove_dot_key(obj):
for key in obj.keys():
new_key = key.replace(".","")
if new_key != key:
obj[new_key] = obj[key]
del obj[key]
return obj
new_json = json.loads(data, object_hook=remove_dot_key)
You were returning a dictionary inside your loop, so you'd only modify one key. And you don't need to make a copy of the values, just rename the keys.
My input file is going to be something like this
key "value"
key "value"
... the above lines repeat
What I do is read the file contents, populate an object with the data and return it. There are only a set number of keys that can be present in the file. Since I am a beginner in python, I feel that my code to read the file is not that good
My code is something like this
ObjInstance = CustomClass()
fields = ['key1', 'key2', 'key3']
for field in fields:
for line in f:
if line.find(field) >= 0:
if pgn_field == 'key1':
objInstance.DataOne = get_value_using_re(line)
elif pgn_field == 'key2':
objInstance.DataTwo = get_value_using_re(line)
return objInstance;
The function "get_value_using_re" is very simple, it looks for a string in between the double quotes and returns it.
I fear that I will have multiple if elif statements and I don't know if this is the right way or not.
Am I doing the right thing here?
A normal approach in Python would be something like:
for line in f:
mo = re.match(r'^(\S+)\s+"(.*?)"\s*$',line)
if not mo: continue
key, value = mo.groups()
setattr(objInstance, key, value)
If the key is not the right attribute name, in the last line line in lieu of key you might use something like translate.get(key, 'other') for some appropriate dict translate.
I'd suggest looking at the YAML parser for python. It can conveniently read a file very similar to that and input it into a python dictionary. With the YAML parser:
import yaml
map = yaml.load(file(filename))
Then you can access it like a normal dictionary with map[key] returning value. The yaml files would look like this:
key1: 'value'
key2: 'value'
This does require that all the keys be unique.