I had a problem on converting dictionaries to strings which has recursive features.
I had a map of routing such as the following;
urls = {
'/' : 'BaseController.hello',
'/api' : {
'/auth' : {
'/me' : 'ApiController.hello',
'/login' : {
'/guest' : 'ApiController.guest_login',
'/member': 'ApiController.member_login'
}
}
}
}
What I need to do is to generate a dictionary from that into the following;
url_map = {
'/' : 'BaseController.hello',
'/api/auth/me' : 'ApiController.hello',
'/api/auth/login/guest' : 'ApiController.guest_login',
'/api/auth/login/member': 'ApiController.member_login',
}
This feature is called route grouping but I haven't been able to write a function to generate that. Any ideas ?
You can recursively do it like this
def flatten(current_dict, current_key, result_dict):
# For every key in the dictionary
for key in current_dict:
# If the value is of type `dict`, then recurse with the value
if isinstance(current_dict[key], dict):
flatten(current_dict[key], current_key + key, result_dict)
# Otherwise, add the element to the result
else:
result_dict[current_key + key] = current_dict[key]
return result_dict
print flatten(urls, "", {})
Output
{
'/api/auth/me': 'ApiController.hello',
'/api/auth/login/guest': 'ApiController.guest_login',
'/': 'BaseController.hello',
'/api/auth/login/member': 'ApiController.member_login'
}
Related
I'm new to learning python and i've learned to push a config file data into nested dict. how is it done vice versa?
example fille would be like:
i need
dictonary = { 'section1' : { 'name' : 'abcd' , 'language' : 'python' } , 'section2' : { 'name' : 'aaaa' , 'language' : 'java' } }
into something like this
[section1]
name : abcd
language : python
[section2]
name : aaaa
language : java
Your expected output looks like a toml output. try:
import toml
toml.dumps(dictonary)
toml - PyPI
You can use the module configparser.
import configparser
dictonary = {
'section1' : { 'name' : 'abcd' , 'language' : 'python' } ,
'section2' : { 'name' : 'aaaa' , 'language' : 'java' } }
config = configparser.RawConfigParser()
for section, pairs in dictonary.items():
config.add_section(section)
for k,v in pairs.items():
config.set(section, k, v)
with open('example.cfg', 'w') as configfile:
config.write(configfile)
This will work and here's why:
our_dict = {
'section1':{
'name':'abcd',
'language':'python'
},
'section2':{
'name':'aaaa',
'language':'java'
}
}
def dictFunc(dictionary):
ret = ''
for i in dictionary:
value = dictionary.get(i)
ret += '\n' + i + ':\n'
for j in value:
k = value.get(j)
ret += j + ':' + k + '\n'
return ret
print(dictFunc(our_dict))
First, we declare our_dict. Then, we declare the function dictFunc(), with one argument; the dictionary. We make a variable named ret, which we will soon return. We start by looping the dictionary, and declaring the variable value. It's the dictionary's second-place key (i.e {'name':'aaaa','language':'java'}). We make sure to add the key (i.e section1) to ret. We loop our second-place key, or j, to get j and j's current key, or k (i.e, name.). Finally, we get j's current second-place key, and link them together in ret. We now return ret.
How to create folders with name of all dictionaries in nested dictionary?
The code should iterate through dictionary and create the directories structure like in nested dictionary with keeping all hierarchy.
dic = {
"root": {
'0_name': {
"0_name_a": {
"0_name_a_a": {
},
"0_name_a_b": {
"file": "file"
}
},
"0_name_b": {
}
},
"1_name": {
},
"2_name": {
},
"3_name": {
"3_name": {
},
}
}
}
Should make directories like:
root/0_name
root/0_name/0_name_a
root/0_name/0_name_a/0_name_a_a
root/0_name/0_name_a/0_name_a_b
root/0_name/0_name_b
root/1_name/1_name_a
root/2_name/
root/3_name/3_name(the same name)
The script needs to determine if the value is final, and create a folder with that path, then remove that value from a dictionary and start over. Also somehow to recognize "file" type and skip it. I couldn't determine a recursive way to iterate all values and add them to a path.
My approach (absolutely not working, just pasted it in to show something):
def rec(dic):
path = []
def nova(dic):
for k, v in dic.copy().items():
if isinstance(v, dict):
path.append(k)
if not v:
print(os.path.join(*path))
path.clear()
dic.pop(k)
nova(v)
if path == []:
nova(dic)
You're going to need a recursive program, and the value of "path" needs to be part of that recursion.
This isn't exactly what you want, but it's close.
def handle_problem(dic):
def one_directory(dic, path):
for name, info in dic.items():
next_path = path + "/" + name
if isinstance(info, dict):
print("Creating " + next_path) # actually use mkdir here!
one_directory(info, next_path)
one_directory(dic, '')
I have a dictionary which contains the following json elements.
myjsonDictionary = \
{
"Teams": {
"TeamA": {
"#oid": "123.0.0.1",
"dataRequestList": {
"state": {
"#default": "0",
"#oid": "2"
}
},
"TeamSub": {
"#oid": "3",
"dataRequestList": {
"state": {
"#default": "0",
"#oid": "2"
}
}
}
},
# ....many nested layers
}
}
I have the following issue and am currently very confused on how to solve this problem.
I want to be able to parse this dictionary and get the concatenation of the "#oid" value and the respective "#oid" when I request the "key" such as "TeamA" or "TeamSub".
I have a function which takes in the gettheiDLevelConcatoid(myjsonDictionary, key).
I can call this function like this:
gettheiDLevelConcatoid(myjsonDictionary, key) where "key" is like "TeamA"
And the expected output should be "123.0.0.1.2". Note the 2 appended to the 123.0.0.1.
gettheiDLevelConcatoid(myjsonDictionary, key) where "key" is like TeamSub
Output is "123.0.0.1.3.2". Note the "3.2" added to the "123.0.0.1".
My current implementation:
def gettheiDLevelConcatoid(myjsonDictionary, key)
for item in myjsonDictionary:
if (item == key):
#not sure what to do
I am so lost on how to implement a generic method or approach for this.
With recursive traversal for specific keys:
def get_team_idlvel_oid_pair(d, search_key):
for k, v in d.items():
if k.startswith('Team'):
if k == search_key:
return '{}{}.{}'.format(d['#oid'] + '.' if '#oid' in d else '',
v['#oid'], v['dataRequestList']['state']['#oid'])
elif any(k.startswith('Team') for k_ in v):
return get_team_idlvel_oid_pair(v, search_key)
print(get_team_idlvel_oid_pair(myjsonDictionary['Teams'], 'TeamA'))
print(get_team_idlvel_oid_pair(myjsonDictionary['Teams'], 'TeamSub'))
Sample output:
123.0.0.1.2
123.0.0.1.3.2
I am new to pyspark . My requirement is to get/extract the attribute names from a nested json file . I tried using json_normalize imported from pandas package. It works for direct attributes but never fetches the attributes within json array attributes. My json doesn't have a static structure. It varies for each document that we receive. Could someone please help me with explanation for the small example provided below,
{
"id":"1",
"name":"a",
"salaries":[
{
"salary":"1000"
},
{
"salary":"5000"
}
],
"states":{
"state":"Karnataka",
"cities":[
{
"city":"Bangalore"
},
{
"city":"Mysore"
}
],
"state":"Tamil Nadu",
"cities":[
{
"city":"Chennai"
},
{
"city":"Coimbatore"
}
]
}
}
Especially for the json array elements..
Expected output :
id
name
salaries.salary
states.state
states.cities.city``
Here is the another solution for extracting all nested attributes from json
import json
result_set = set([])
def parse_json_array(json_obj, parent_path):
array_obj = list(json_obj)
for i in range(0, len(array_obj)):
json_ob = array_obj[i]
if type(json_obj) == type(json_obj):
parse_json(json_ob, parent_path)
return None
def parse_json(json_obj, parent_path):
for key in json_obj.keys():
key_value = json_obj.get(key)
# if isinstance(a, dict):
if type(key_value) == type(json_obj):
parse_json(key_value, str(key) if parent_path == "" else parent_path + "." + str(key))
elif type(key_value) == type(list(json_obj)):
parse_json_array(key_value, str(key) if parent_path == "" else parent_path + "." + str(key))
result_set.add((parent_path + "." + key).encode('ascii', 'ignore'))
return None
file_name = "C:/input/sample.json"
file_data = open(file_name, "r")
json_data = json.load(file_data)
print json_data
parse_json(json_data, "")
print list(result_set)
Output:
{u'states': {u'state': u'Tamil Nadu', u'cities': [{u'city': u'Chennai'}, {u'city': u'Coimbatore'}]}, u'id': u'1', u'salaries': [{u'salary': u'1000'}, {u'salary': u'5000'}], u'name': u'a'}
['states.cities.city', 'states.cities', '.id', 'states.state', 'salaries.salary', '.salaries', '.states', '.name']
Note:
My Python version: 2.7
you can do in this way also.
data = { "id":"1", "name":"a", "salaries":[ { "salary":"1000" }, { "salary":"5000" } ], "states":{ "state":"Karnataka", "cities":[ { "city":"Bangalore" }, { "city":"Mysore" } ], "state":"Tamil Nadu", "cities":[ { "city":"Chennai" }, { "city":"Coimbatore" } ] } }
def dict_ittr(lin,data):
for k, v in data.items():
if type(v)is list:
for l in v:
dict_ittr(lin+"."+k,l)
elif type(v)is dict:
dict_ittr(lin+"."+k,v)
pass
else:
print lin+"."+k
dict_ittr("",data)
output
.states.state
.states.cities.city
.states.cities.city
.id
.salaries.salary
.salaries.salary
.name
If you treat the json like a python dictionary, this should work.
I just wrote a simple recursive program.
Script
import json
def js_r(filename):
with open(filename) as f_in:
return(json.load(f_in))
g = js_r("city.json")
answer_d = {}
def base_line(g, answer_d):
for key in g.keys():
answer_d[key] = {}
return answer_d
answer_d = base_line(g, answer_d)
def recurser_func(g, answer_d):
for k in g.keys():
if type(g[k]) == type([]): #If the value is a list
answer_d[k] = {list(g[k][0].keys())[0]:{}}
if type(g[k]) == type({}): #If the value is a dictionary
answer_d[k] = {list(g[k].keys())[0]: {}} #set key equal to
answer_d[k] = recurser_func(g[k], answer_d[k])
return answer_d
recurser_func(g,answer_d)
def printer_func(answer_d, list_to_print, parent):
for k in answer_d.keys():
if len(answer_d[k].keys()) == 1:
list_to_print.append(parent)
list_to_print[-1] += k
list_to_print[-1] += "." + str(list(answer_d[k].keys())[0])
if len(answer_d[k].keys()) == 0:
list_to_print.append(parent)
list_to_print[-1] += k
if len(answer_d[k].keys()) > 1:
printer_func(answer_d[k], list_to_print, k + ".")
return list_to_print
l = printer_func(answer_d, [], "")
final = " ".join(l)
print(final)
Explanation
base_line makes a dictionary of all your base keys.
recursur_func checks if the key's value is a list or dict then adds to the answer dictionary as is necessary until answer_d looks like: {'id': {}, 'name': {}, 'salaries': {'salary': {}}, 'states': {'state': {}, 'cities': {'city': {}}}}
After these 2 functions are called you have a dictionary of keys in a sense. Then printer_func is a recursive function to print it as you desired.
NOTE:
Your question is similar to this one: Get all keys of a nested dictionary but since you have a nested list/dictionary instead of just a nested dictionary, their answers won't work for you, but there is more discussion on the topic on that question if you like more info
EDIT 1
my python version is 3.7.1
I have added a json file opener to the top. I assume that the json is named city.json and is in the same directory
EDIT 2: More thorough explanation
The main difficulty that I found with dealing with your data is the fact that you can have infinitely nested lists and dictionaries. This makes it complicated. Since it was infinite possible nesting, I new this was a recursion problem.
So, I build a dictionary of dictionaries representing the key structure that you are looking for. Firstly I start with the baseline.
base_line makes {'id': {}, 'name': {}, 'salaries': {}, 'states': {}} This is a dictionary of empty dictionaries. I know that when you print. Every key structure (like states.state) starts with one of these words.
recursion
Then I add all the child keys using recursur_func.
When given a dictionary g this function for loop through all the keys in that dictionary and (assuming answer_d has each key that g has) for each key will add that keys child to answer_d.
If the child is a dictionary. Then I recurse with the given dictionary g now being the sub-part of the dictionary that pertains to the children, and answer_d being the sub_part of answer_d that pertains to the child.
I have the following JSON data the I need to read in Flask:
{
"NewList":[
{
"key" : "myvalue1",
"value" : "value1"
},
{
"key" : "myvalue2",
"value" : "value2"
},
{
"key" : "myvalu3",
"value" : "value4"
}
]
}
And I'm having trouble doing so. The code I currently have is as follows:
#app.route('/dataread', methods=['GET', 'POST'])
def dataread():
if(request.json):
myvalue1 = request.json['NewList']['myvalue1']
return str(myvalue1)
else:
return 'nothing'
But it isn't working. I'm getting the following error:
KeyError: 'NewList'
I know my syntax must be wrong but I can't figure how to fix it. I'm sorry for such a newb question. Please can you help.
Thanks.
There's a few things going in your example that aren't correct.
The json is actually interpreted as :
{ "NewList":[
{
"key1" : "value1",
"key2" : "value2"
},
{
"key1" : "value1",
"key2" : "value2"
}
]}
So in your example myvalue1 is a value not a key.
The main error however is that in flask request.json() only returns the whole json input, you can't select certain elements from the json. Also request.json is deprecated so it now should be request.get_json()
So the final solution given your input data would be something like
{
"NewList":[
{
"key" : "myvalue1",
"value" : "value1"
},
{
"key" : "myvalue2",
"value" : "value2"
},
{
"key" : "myvalu3",
"value" : "value4"
}
]
}
#app.route('/dataread', methods=['GET', 'POST'])
def dataread():
if(request.data):
jdata = request.get_json()
return str(jdata['Newlist'][0]['key'])
else:
return 'nothing'
The [0] being the first object in the array that's the value of Newlist.
Unless you know it's always going to be array element 0 you'll get dud data, plus you're returning the value of the "key". Given that input data I suspect you probably want something more like.
if(request.data):
jdata = request.get_json()
for j in jdata['Newlist']:
if j['key'] == 'myvalue1':
return str(j['value'])
return 'not found'
else:
return 'nothing'
I wasn't sure which part of this you were having an issue with so I did it long hand. It's obviously not ideal but should be easily understandable.But I would echo the comment above that you should start by printing exactly what you got in. May not be a complete match.
The dictionary is a set of lists of dictionaries so you end up walking the set for each one.
dict = {
"NewList":[
{
"key" : "myvalue1",
"value" : "value1"
},
{
"key" : "myvalue2",
"value" : "value2"
},
{
"key" : "myvalu3",
"value" : "value4"
}
]
}
for firstkey, big_list in dict.items():
print('print dict: ' + str(firstkey))
for pair in big_list:
print('print sets in dict: ' + str(pair))
nextdict = pair
for nextkey, small_list in nextdict.items():
print('print each: ' + str(nextkey)+ '->' + str(small_list))
#address each one
print('pull just data: ' + str(nextdict[nextkey]))
"""
results
print dict: NewList
print sets in dict: {'key': 'myvalue1', 'value': 'value1'}
print each: key->myvalue1
pull just data: myvalue1
print each: value->value1
pull just data: value1
print sets in dict: {'key': 'myvalue2', 'value': 'value2'}
print each: key->myvalue2
pull just data: myvalue2
print each: value->value2
pull just data: value2
print sets in dict: {'key': 'myvalu3', 'value': 'value4'}
print each: key->myvalu3
pull just data: myvalu3
print each: value->value4
pull just data: value4
"""
The value of newList is a list. You have to access an element of this list first before reading off "key" and "value".
# print first item in the list
print(request.json['NewList'][0]['value']
# print them all
for item in request.json['NewList']
print(item['key'])
print(item['value'])