I have a dictionary of dictionaries, like given below:
{
"dev": {
"project_id": "dev_project_id",
"secret_id": "dev_secret_id",
"secret_project_id": "dev_secret_project_id",
"service_account_email": "dev_service_account_email#gmail.com",
"email_list": ["dev_email#gmail.com"],
"core_func_path":"dev/core_func.py",
"secret_id_email": "dev_secret_id_email"
},
"prod": {
"project_id": "prod_project_id",
"secret_id": "prod_secret_id",
"secret_project_id": "prod_secret_project_id",
"service_account_email": "prod_service_account_email#gmail.com",
"email_list": ["prod_email_list#gmail.com"],
"core_func_path":"prod/core_func.py",
"secret_id_email": "prod_secret_id_email"
}
}
And I need to extract key when a specific project_id is provided.
Till now, I have this code, which can get values from a dictionary, however, it is failing for a dictionary of dictionaries.
check_project_id='dev_project_id'
curr_dir = Path(os.path.dirname(os.path.abspath(__file__)))
default_config_dir = os.fspath(Path(curr_dir.parent.parent, 'config').resolve())
constants_path = str(default_config_dir)+'/config.json'
with open(constants_path, 'r') as f:
std_config = json.load(f)
for val in std_config.values():
if(val['project_id']==check_project_id):
print(list(std_config.keys())[list(std_config.values()).index(check_project_id)])
Is there any way I can implement this?
So if I understand correctly, std_config is your dictionary of dictionaries. Just use the items call on the dictionary to be able to extract the key that matches your criteria.
for k, v in std_config.items():
if v["project_id"] == check_project_id:
print(k)
> dev
Related
I'm trying to dynamically create a database without hard coding all of the API response values for columns and rows so I want to learn how to automatically parse through JSON and put all the keys/values in a variable so I can sort through and put them in the db.
Lets say I had some JSON like this:
(modified snippet from destiny 2 API response indented by 5)
{
"Response": {
"profile": {
"data": {
"userInfo": {
"crossSaveOverride": 1,
"applicableMembershipTypes": [
3,
1
],
"isPublic": true,
"membershipType": 1,
"membershipId": "123",
"displayName": "Name1",
"bungieGlobalDisplayName": "name again",
"bungieGlobalDisplayNameCode": 0000
},
"dateLastPlayed": "2021-6-18T02:33:01Z",
"versionsOwned": 127,
"characterIds": [
"id1",
"id2",
"id3"
],
"seasonHashes": [
3243434344,
3423434244,
3432342443,
3434243244
],
"currentSeasonHash": 3434243244,
"currentSeasonRewardPowerCap": 1330
},
"privacy": 1
}
},
"ErrorCode": 1,
"ThrottleSeconds": 0,
"ErrorStatus": "Success",
"Message": "Ok",
"MessageData": {}
}
I want to automatically parse through and get all of the key/values minus the error code area so everything under "Response". It would all go into the database for example:
displayName
isPublic
Name1
True
Name2
False
I know how to parse normally or loop through but only to grab values like:
displayName = Response["Response"]["profile"]["data"]["userInfo"]["displayName"]
How does one grab all keys and values and then automatically store in a variable from the lowest level? Also, how would one exclude keys if they are not needed?
EDIT: Adding Clarification
I learned that this JSON response type is a dict and I can use Response.keys() and Response.values() to get the keys and values.
What I am asking is, from Response["Response"], how to get all of the keys and values down to the bottom level and exclude the ones I don't need.
For example if I do:
r = Response["Response"]
for key in r.keys():
print(key)
I'm only going to get profile which is what I don't need.
I would have to do:
r = Response["Response"]["profile"]["data"]["userInfo"]
for key in r.keys():
print(key)
which would return
crossSaveOverride
applicableMembershipTypes
isPublic
membershipType
membershipId
displayName
bungieGlobalDisplayName
bungieGlobalDisplayNameCode
Which is what I need but I do not want to manually define
["Response"]["profile"]["data"]["userInfo"]or similar for every response. I want to grab everything automatically while excluding the items I don't need.
Dmitry Torba over at Get all keys of a nested dictionary posted a function that does what you want.
def recursive_items(dictionary):
for key, value in dictionary.items():
if type(value) is dict:
yield from recursive_items(value)
else:
yield (key, value)
a = {'a': {1: {1: 2, 3: 4}, 2: {5: 6}}}
for key, value in recursive_items(a):
print(key, value)
I have tried this approach, what if there is list type in given dictionary!
CallPretty.py
def JsonPrint(response, response_Keys):
print()
print(response_Keys)
for i in range(len(response_Keys)):
if type(response[response_Keys[i]]) == list:
print()
for j in range(len(response[response_Keys[i]])):
if type(response[response_Keys[i]][j]) == dict:
JsonPrint(response[response_Keys[i]][j], list(response[response_Keys[i]][j].keys()))
elif type(response[response_Keys[i]]) == dict:
JsonPrint(response[response_Keys[i]], list(response[response_Keys[i]].keys()))
else:
print(response_Keys[i], ':')
for j in range(len(response[response_Keys[i]].split('\n'))):
print('\t', response[response_Keys[i]].split('\n')[j])
This is the only an example but for the general idea :
i have Json as dict created in different places in the application
and in the end i like to merge them as 1 JSON .
json_1 ={}
formated_db_name = "formatedname"
json_1[formated_db_name] = {"data_source_name": formated_db_name}
json_1[formated_db_name] = {"db_servers_entry_list": {}}
json_2 = {}
formated_db_name2 = "formatedname2"
json_2[formated_db_name2] = {"data_source_name2": formated_db_name2}
json_2[formated_db_name2] = {"db_servers_entry_list2": {}}
it creates 2 jsons :
{
"formatedname2": {
"db_servers_entry_list2": {}
}
}
and
{
"formatedname": {
"db_servers_entry_list": {}
}
}
now i like to combine them to look like this :
{
"formatedname2": {
"db_servers_entry_list2": {}
},
"formatedname1": {
"db_servers_entry_list1": {}
}
}
didn't found any json.dumps method to combine both . ( there can be more then 2 such dict that i need to combine )
You can just iterate through the sequence of the dictionaries and call update and pass the individual dictionaries to a dictionary variable:
out = {}
for each in (json_1, json_2):
out.update(each)
# out
{'formatedname': {'db_servers_entry_list': {}}, 'formatedname2': {'db_servers_entry_list2': {}}}
You can later call json.dumps and pass out once you are done merging the dictionaries.
Since the dictionary is being update via the call to update method, it will maintain the the dictionary/json property (there is never going to be multiple key value pair for a single key, it will just update the dictionary with the latest value being passed for a key.)
You can join the dicts beforehand and then you can use the dumps method!
a = dict(a="Test")
b = dict(b=True)
c = dict()
c.update(a)
c.update(b)
print(c)
{
"a": "Test",
"b": True
}
I am trying to get all keys from a json file in Python.
How to get nested second level(x,y) and third level keys(a,b).
For example, Keys: results,x,y,a,b
Code:
#open data
import json
with open('list.json') as f:
my_dict = json.load(f)
#1
#find keys
for key in my_dict.keys():
print("Keys : {}".format(key))
Json:
{
"results":[
{
"x":5
},
{
"x":5,
"y":[
1,
2,
3
]
},
{
"x":5,
"y":{
"a":2,
"b":67
}
}
]
}
Output:
Keys : results
You need to get the keys which are a part of the value of the JSON.
You therefore need to iterate over the values of my_dict not the keys.
Use recursive function to return all nested keys. Here is the reference stackoverflow page.
import json
def recursive_items(dictionary):
for key, value in dictionary.items():
if type(value) is list:
for i in value:
if type(i) is dict:
yield from recursive_items(i)
else:
yield key
with open('list.json') as f:
my_dict = json.load(f)
#find keys
for key in recursive_items(my_dict):
print("Keys : {}".format(key))
Giving data organized in JSON format (code example bellow) how can we get the path of keys and sub-keys associated with a given value?
i.e.
Giving an input "23314" we need to return a list with:
Fanerozoico, Cenozoico, Quaternario, Pleistocenico, Superior.
Since data is a json file, using python and json lib we had decoded it:
import json
def decode_crono(crono_file):
with open(crono_file) as json_file:
data = json.load(json_file)
Now on we do not know how to treat it in a way to get what we need.
We can access keys like this:
k = data["Fanerozoico"]["Cenozoico"]["Quaternario "]["Pleistocenico "].keys()
or values like this:
v= data["Fanerozoico"]["Cenozoico"]["Quaternario "]["Pleistocenico "]["Superior"].values()
but this is still far from what we need.
{
"Fanerozoico": {
"id": "20000",
"Cenozoico": {
"id": "23000",
"Quaternario": {
"id": "23300",
"Pleistocenico": {
"id": "23310",
"Superior": {
"id": "23314"
},
"Medio": {
"id": "23313"
},
"Calabriano": {
"id": "23312"
},
"Gelasiano": {
"id": "23311"
}
}
}
}
}
}
It's a little hard to understand exactly what you are after here, but it seems like for some reason you have a bunch of nested json and you want to search it for an id and return a list that represents the path down the json nesting. If so, the quick and easy path is to recurse on the dictionary (that you got from json.load) and collect the keys as you go. When you find an 'id' key that matches the id you are searching for you are done. Here is some code that does that:
def all_keys(search_dict, key_id):
def _all_keys(search_dict, key_id, keys=None):
if not keys:
keys = []
for i in search_dict:
if search_dict[i] == key_id:
return keys + [i]
if isinstance(search_dict[i], dict):
potential_keys = _all_keys(search_dict[i], key_id, keys + [i])
if 'id' in potential_keys:
keys = potential_keys
break
return keys
return _all_keys(search_dict, key_id)[:-1]
The reason for the nested function is to strip off the 'id' key that would otherwise be on the end of the list.
This is really just to give you an idea of what a solution might look like. Beware the python recursion limit!
Based on the assumption that you need the full dictionary path until a key named id has a particular value, here's a recursive solution that iterates the whole dict. Bear in mind that:
The code is not optimized at all
For huge json objects it might yield StackOverflow :)
It will stop at first encountered value found (in theory there shouldn't be more than 1 if the json is semantically correct)
The code:
import json
from types import DictType
SEARCH_KEY_NAME = "id"
FOUND_FLAG = ()
CRONO_FILE = "a.jsn"
def decode_crono(crono_file):
with open(crono_file) as json_file:
return json.load(json_file)
def traverse_dict(dict_obj, value):
for key in dict_obj:
key_obj = dict_obj[key]
if key == SEARCH_KEY_NAME and key_obj == value:
return FOUND_FLAG
elif isinstance(key_obj, DictType):
inner = traverse_dict(key_obj, value)
if inner is not None:
return (key,) + inner
return None
if __name__ == "__main__":
value = "23314"
json_dict = decode_crono(CRONO_FILE)
result = traverse_dict(json_dict, value)
print result
i am parsing a log file which is in json format,
and contains data in the form of key : value pair.
i was stuck at place where key itself is variable. please look at the attached code
in this code i am able to access keys like username,event_type,ip etc.
problem for me is to access the values inside the "submission" key where
i4x-IITB-CS101-problem-33e4aac93dc84f368c93b1d08fa984fc_2_1 is a variable key which will change for different users,
how can i access it as a variable ?
{
"username": "batista",
"event_type": "problem_check",
"ip": "127.0.0.1",
"event": {
"submission": {
"i4x-IITB-CS101-problem-33e4aac93dc84f368c93b1d08fa984fc_2_1": {
"input_type": "choicegroup",
"question": "",
"response_type": "multiplechoiceresponse",
"answer": "MenuInflater.inflate()",
"variant": "",
"correct": true
}
},
"success": "correct",
"grade": 1,
"correct_map": {
"i4x-IITB-CS101-problem-33e4aac93dc84f368c93b1d08fa984fc_2_1": {
"hint": "",
"hintmode": null,
"correctness": "correct",
"npoints": null,
"msg": "",
"queuestate": null
}
}
this is my code how i am solving it :
import json
import pprint
with open("log.log") as infile:
# Loop until we have parsed all the lines.
for line in infile:
# Read lines until we find a complete object
while (True):
try:
json_data = json.loads(line)
username = json_data['username']
print "username :- " + username
except ValueError:
line += next(infile)
how can i access i4x-IITB-CS101-problem-33e4aac93dc84f368c93b1d08fa984fc_2_1 key and
data inside this key ??
You don't need to know the key in advance, you can simply iterate over the dictionary:
for k,v in obj['event']['submission'].iteritems():
print(k,v)
Suppose you have a dictionary of type d = {"a":"b"} then d.popitem() would give you a tuple ("a","b") which is (key,value). So using this you can access key-value pairs without knowing the key.
In you case if j is the main dictionary then j["event"]["submission"].popitem() would give you tuple
("i4x-IITB-CS101-problem-33e4aac93dc84f368c93b1d08fa984fc_2_1": {
"input_type": "choicegroup",
"question": "",
"response_type": "multiplechoiceresponse",
"answer": "MenuInflater.inflate()",
"variant": "",
"correct": true
})
Hope this is what you were asking.
using python json module you'll end up with a dictionary of parsed values from the above JSON data
import json
parsed = json.loads(this_sample_data_in_question)
# parsed is a dictionary, so are "correct_map" and "submission" dictionary keys within "event" key
So you could iterate over the key, values of the data as a normal dictionary, say like this:
for k, v in parsed.items():
print k, v
Now you could find the (possible different values) of "i4x-IITB-CS101-problem-33e4aac93dc84f368c93b1d08fa984fc_2_1" key in a quick way like this:
import json
parsed = json.loads(the_data_in_question_as_string)
event = parsed['event']
for key, val in event.items():
if key in ('correct_map', 'submission'):
section = event[key]
for possible_variable_key, its_value in section.items():
print possible_variable_key, its_value
Of course there might be better way of iterating over the dictionary, but that one you could choose based on your coding taste, or performance if you have a fairly larger kind of data than the one posted in here.