I have a dictionary that contains the information to make a POST request to site and fetch data.
Below is my dictionary
request_dict = {
"scope_id": "user1",
"tool_id": "appdynamics",
"api_id": "get metrics names",
"input_params": {"user": "abcd", "pwd": "1234", "acc_id": "ab12", "app_id": "TestApp",
"metric-path": "ars",
"time-range-type": "BEFORE_NOW", "duration-in-mins": 10},
"output_filters": null
}
I have another dictionary which contains the updated value for some of the keys mentioned in the above dictionary.Below is the dictionary
new_metric_dict = {'tool_id': 'appdynamics', 'pwd': '5678', 'user': 'efgh', 'metric-path': 'ars', 'app_id': 'TestApp', 'acc_id': 'cd34'}
As you can see I got new values for user, pwd and acc_id.But I am unable to put these values into the POST request dictionary since they are inside input_params.
I tried doing something like this
for i in request_dict:
if type(request_dict[i]) == dict:
for j in request_dict:
if j in new_metric_dict:
request_dict[j] = new_metric_dict[j]
if i in new_metric_dict:
request_dict[i] = new_metric_dict[i]
But it doesn't work.What is an efficient way to achieve this?
This is one way, which creates a new dictionary with the desired updates.
def update_dict(d, new):
out = {}
for m, val in d.items():
if isinstance(d[m], dict):
out[m] = {k: new.get(k, d[m][k]) for k, v in d[m].items()}
else:
out[m] = new.get(m, val)
return out
new = update_dict(request_dict, new_metric_dict)
# {'api_id': 'get metrics names',
# 'input_params': {'acc_id': 'cd34',
# 'app_id': 'TestApp',
# 'duration-in-mins': 10,
# 'metric-path': 'ars',
# 'pwd': '5678',
# 'time-range-type': 'BEFORE_NOW',
# 'user': 'efgh'},
# 'output_filters': 'null',
# 'scope_id': 'user1',
# 'tool_id': 'appdynamics'}
Related
I wrote a code that takes 9 keys from API.
The authors, isbn_one, isbn_two, thumbinail, page_count fields may not always be retrievable, and if any of them are missing, I would like it to be None. Unfortunately, if, or even nested, doesn't work. Because that leads to a lot of loops. I also tried try and except KeyError etc. because each key has a different error and it is not known which to assign none to. Here is an example of logic when a photo is missing:
th = result['volumeInfo'].get('imageLinks')
if th is not None:
book_exists_thumbinail = {
'thumbinail': result['volumeInfo']['imageLinks']['thumbnail']
}
dnew = {**book_data, **book_exists_thumbinail}
book_import.append(dnew)
else:
book_exists_thumbinail_n = {
'thumbinail': None
}
dnew_none = {**book_data, **book_exists_thumbinail_n}
book_import.append(dnew_none)
When I use logic, you know when one condition is met, e.g. for thumbinail, the rest is not even checked.
When I use try and except, it's similar. There's also an ISBN in the keys, but there's a list in the dictionary over there, and I need to use something like this:
isbn_zer = result['volumeInfo']['industryIdentifiers']
dic = collections.defaultdict(list)
for d in isbn_zer:
for k, v in d.items():
dic[k].append(v)
Output data: [{'type': 'ISBN_10', 'identifier': '8320717507'}, {'type': 'ISBN_13', 'identifier': '9788320717501'}]
I don't know what to use anymore to check each key separately and in the case of its absence or lack of one ISBN (identifier) assign the value None. I have already tried many ideas.
The rest of the code:
book_import = []
if request.method == 'POST':
filter_ch = BookFilterForm(request.POST)
if filter_ch.is_valid():
cd = filter_ch.cleaned_data
filter_choice = cd['choose_v']
filter_search = cd['search']
search_url = "https://www.googleapis.com/books/v1/volumes?"
params = {
'q': '{}{}'.format(filter_choice, filter_search),
'key': settings.BOOK_DATA_API_KEY,
'maxResults': 2,
'printType': 'books'
}
r = requests.get(search_url, params=params)
results = r.json()['items']
for result in results:
book_data = {
'title': result['volumeInfo']['title'],
'authors': result['volumeInfo']['authors'][0],
'publish_date': result['volumeInfo']['publishedDate'],
'isbn_one': result['volumeInfo']['industryIdentifiers'][0]['identifier'],
'isbn_two': result['volumeInfo']['industryIdentifiers'][1]['identifier'],
'page_count': result['volumeInfo']['pageCount'],
'thumbnail': result['volumeInfo']['imageLinks']['thumbnail'],
'country': result['saleInfo']['country']
}
book_import.append(book_data)
else:
filter_ch = BookFilterForm()
return render(request, "BookApp/book_import.html", {'book_import': book_import,
'filter_ch': filter_ch})```
I have ['key','value'] format list, which also contain sub-list. How can I convert nested list to JSON format in python
[[' key ', ' 1542633482511430199'
],
['value=>>>BasicData',
[['isConfirmAndOrder', '0'],['brmRequestId', 'BR-2018-0000124'],
['requestType','batch'],['projectId', 'PRJ-2018-0000477'],
['createdOn', 'Mon Nov 19 18:48:02 IST 2018']]
],
['createdBy=>>>BasicData',
[['userId', '999996279'], ['email', 'ITEST275#ITS.JNJ.com'],
['firstName', 'Iris'], ['lastName', 'TEST275'],
['ntId', 'itest275'], ['region', 'NA'],
[' LastAccessTime ', ' 1542639905785 ']]
]
]
Excepted format is
{
"key": "1542633482511430199",
"value=>>>BasicData": {
"isConfirmAndOrder": "0",
"brmRequestId": "BR-2018-0000124"
.
},
"createdBy=>>>BasicData": {
"userId": "999996279",
"email": "ITEST275#ITS.JNJ.com"
.
}
.
}
Actually format of large data is:
[
[
['key11','value11']
['key12',['key13','value13']]
['key14',['key15','value15']]
]
[
['key21','value21']
['key22',['key23','value23']]
['key24',['key25','value25']]
]
]
You can write a simple recursive function for this:
def to_dict_recursive(x):
d = {}
for key, value in x:
if isinstance(value, list):
value = to_dict_recursive(value)
else:
value = value.strip() # get rid of unnecessary whitespace
d[key.strip()] = value
return d
to_dict_recursive(x)
# {'createdBy=>>>BasicData': {'displayName': 'Iris TEST275',
# 'email': 'ITEST275#ITS.JNJ.com',
# 'firstName': 'Iris',
# 'lastName': 'TEST275',
# 'ntId': 'itest275',
# 'region': 'NA',
# 'roles': '[0]CG510_DHF_AP_Role',
# 'userId': '999996279'},
# 'formulaDetails=>>>BasicData': {'CreationTime': '1542633482512',
# 'LastAccessTime': '1542639905785',
# 'batchSizeUnits': 'kg<<<<<<',
# 'hitCount': '1',
# 'version': '1'},
# 'key': '1542633482511430199',
# 'value=>>>BasicData': {'brmRequestId': 'BR-2018-0000124',
# 'createdMonth': 'Nov',
# 'createdOn': 'Mon Nov 19 18:48:02 IST 2018',
# 'department': 'Global Packaging',
# 'gxp': '1',
# 'id': '1542633482511430199',
# 'isConfirmAndOrder': '0',
# 'isFilling': 'false',
# 'projectId': 'PRJ-2018-0000477',
# 'projectName': 'Automation_Product_By_Admin',
# 'requestType': 'batch',
# 'status': 'New',
# 'statusDescription': 'Batch request created',
# 'updatedOn': 'Mon Nov 19 18:48:02 IST 2018'}}
(I ran this in Python 3.6 so the order of the keys in the dictionary representation is different than insertion order. In Python 3.7+ this would be different.)
You can even make this into a dict comprehension:
def to_dict_recursive(x):
return {key.strip(): to_dict_recursive(value) if isinstance(value, list)
else value.strip
for key, value in x}
Since apparently some elements in your object are not a two-element list of key and value, you can add a simple guard against that:
def to_dict_recursive(x):
d = {}
try:
for key, value in x:
if isinstance(value, list):
value = to_dict_recursive(value)
else:
value = value.strip()
d[key.strip()] = value
except ValueError:
return x
return d
x = [[' key ', ' 1542633482511430199'],
["test", ["a", "b", "c"]]
]
to_dict_recursive(x)
# {'key': '1542633482511430199', 'test': ['a', 'b', 'c']}
Note that if mylist is a key-value pair list, then dict(mylist) simply returns a dictionary version of it. The tricky part is traversing deep into those nested lists to replace them with dictionaries. Here's a recursive function that does that:
# Where <kv> is your giant list-of-lists.
def kv_to_dict(kv):
if isinstance(kv, list):
kv = dict(kv)
for k in kv:
if isinstance(kv[k], list):
kv[k] = kv_to_dict(kv[k])
return kv
newdict = kv_to_dict(kvpairs)
Once you have things converted to a dictionary, you can just use json.dumps() to format it as JSON:
import json
as_json = json.dumps(newdict, indent=4)
print(as_json)
I see though that you've tried something similar and got an error. Are you sure that all of the lists in your data are really key-value pairs, and not for example a list of 3 strings?
I am trying to create a nested dictionary, whereby the key to each nested dictionary is named from the value from a variable. My end result should look something like this:
data_dict = {
'jane': {'name': 'jane', 'email': 'jane#example.com'},
'jim': {'name': 'jim', 'email': 'jim#example.com'}
}
Here is what I am trying:
data_dict = {}
s = "jane"
data_dict[s][name] = 'jane'
To my surprise, this does not work. Is this possible?
You want something like:
data_dict = {}
s = "jane"
data_dict[s] = {}
data_dict[s]['name'] = s
That should work, though I would recommend instead of a nested dictionary that you use a dictionary of names to either namedtuples or instances of a class.
Try this:
data_dict = {}
s = ["jane", "jim"]
for name in s:
data_dict[name] = {}
data_dict[name]['name'] = name
data_dict[name]['email'] = name + '#example.com'
as #Milad in the comment mentioned, you first need to initialize s as empty dictionary first
data={}
data['Tom']={}
data['Tom']['name'] = 'Tom Marvolo Riddle'
data['Tom']['email'] = 'iamlordvoldermort.com'
For existing dictionaries you can do dict[key] = value although if there is no dict that would raise an error. I think this is the code you want to have:
data_dict = {}
s = "jane"
data_dict[s] = {"name": s, "email": f"{s}#example.com"}
print(data_dict)
I just realized when I got a notification about this question:
data_dict = defaultdict(dict)
data_dict["jane"]["name"] = "jane"
Would be a better answer I think.
I have a data set which follows the structure of the following example:
exampleset = {
'body' : {
'abdomen' : [{
'arms' : {
'value' : 2,
}
},{
'legs': {
'value' : 2,
}
}],
'hands' : {
'fingers' : {
'value' : 5,
}
},
}
}
I am trying to reverse this so I get something like:
{'value': {'value1': {5: {'fingers': {'hands': {'body': {}}}}},
'value2': {2: {'legs': {'abdomen': {'body': {}}}}},
'value3': {2: {'arms': {'abdomen': {'body': {}}}}}},
}
(I hope I got the bracket matching right, but you get the idea.)
I am using a couple of recursion functions to do this, like so:
def recurse_find(data, values, count):
global conf
for key in data:
for v in conf['value_names']:
if key == v:
values[v+str(count)] = {}
values[v+str(count)][data[key]] = {}
count += 1
# originally just using this line:
# values[data[key]] = {}
if type(data[key]) is list:
for i in data[key]:
if type(i) is dict:
values = recurse_find(i, values, count)
values = add_new_level(values, key)
elif type(data[key]) is dict:
values = recurse_find(data[key], values, count)
values = add_new_level(values, key)
return values
def add_new_level(data, new_key):
for key in data:
if data[key] == {}:
data[key][new_key] = {}
else:
data[key] = add_new_level(data[key], new_key)
return data
conf = { "value_names": ["value"] }
for value in conf['value_names']:
values[value] = recurse_find(exampleset, {}, 1)
print(values)
At the moment I only get one value returned correctly, obviously I would like them all. Originally I didn't label the values (value1, value2 etc), but when doing this example set I realised that of course if the values are the same I'll only get one! If I remove the value name keys it finds all the values (unless duplicate) but still doesn't return the correct levels as it includes some of the others while it loops round. I don't care about the order of the values, just that they are labelled differently so I don't miss out any.
Current result:
{'value': {'value1': {5: {'fingers': {'hands': {'body': {}}}}}}}
I think that the solution is the inclusion of a pretty simple step, but I can't see it at the moment and I've already spent too long looking at this.
Any help appreciated.
EDIT:
I've gotten a little further by changing my recursive function to make count a global variable and having count=1 outside the function which has sorted out the getting all the values problem.
I have narrowed down the addition of extra keys to the add_new_level function, but haven't yet figured out how to change it.
Output:
{'value': {'value1': {2: {'arms': {'abdomen': {'legs': {'abdomen': {'fingers': {'hands': {'body': {}}}}}}}}},
'value2': {2: {'legs': {'abdomen': {'fingers': {'hands': {'body': {}}}}}}},
'value3': {5: {'fingers': {'hands': {'body': {}}}}}}}
I have adjusted your output type slightly to make the dictionary containing 'value1' 'value2' etc... to an array. I believe this is better because the order of these will be lost anyway unless an OrderedDict (from collections package) is used and in any case an array will translate quite easily from index 0,1,2,3.. to val1, val2, val3, etc...
res = {'value': []}
def revnest(inp, keys=[]):
res2 = res['value']
if type(inp) == list:
inp = {i:j[i] for j in inp for i in j}
for x in inp:
if x == 'value':
res2.append({inp[x]:{}})
res2 = res2[-1][inp[x]]
for y in keys[::-1]:
res2[y] = {}
res2 = res2[y]
else:
revnest(inp[x], keys+[x])
revnest(exampleset)
print res
which given your exampleset, prints:
{'value': [{2: {'legs': {'abdomen': {'body': {}}}}}, {2: {'arms': {'abdomen': {'body': {}}}}}, {5: {'fingers': {'hands': {'body': {}}}}}]}
I have a bit of a complex question that I can't seem to get to the bottom of. I have a list of keys corresponding to a position in a Python dictionary. I would like to be able to dynamically change the value at the position (found by the keys in the list).
For example:
listOfKeys = ['car', 'ford', 'mustang']
I also have a dictionary:
DictOfVehiclePrices = {'car':
{'ford':
{'mustang': 'expensive',
'other': 'cheap'},
'toyota':
{'big': 'moderate',
'small': 'cheap'}
},
'truck':
{'big': 'expensive',
'small': 'moderate'}
}
Via my list, how could I dynamically change the value of DictOfVehiclePrices['car']['ford']['mustang']?
In my actual problem, I need to follow the list of keys through the dictionary and change the value at the end position. How can this be done dynamically (with loops, etc.)?
Thank you for your help! :)
Use reduce and operator.getitem:
>>> from operator import getitem
>>> lis = ['car', 'ford', 'mustang']
Update value:
>>> reduce(getitem, lis[:-1], DictOfVehiclePrices)[lis[-1]] = 'cheap'
Fetch value:
>>> reduce(getitem, lis, DictOfVehiclePrices)
'cheap'
Note that in Python 3 reduce has been moved to functools module.
A very simple approach would be:
DictOfVehiclePrices[listOfKeys[0]][listOfKeys[1]][listOfKeys[2]] = 'new value'
print reduce(lambda x, y: x[y], listOfKeys, dictOfVehiclePrices)
Output
expensive
In order to change the values,
result = dictOfVehiclePrices
for key in listOfKeys[:-1]:
result = result[key]
result[listOfKeys[-1]] = "cheap"
print dictOfVehiclePrices
Output
{'car': {'toyota': {'small': 'cheap', 'big': 'moderate'},
'ford': {'mustang': 'cheap', 'other': 'cheap'}},
'truck': {'small': 'moderate', 'big': 'expensive'}}
You have a great solution here by #Joel Cornett.
based on Joel method you can use it like this:
def set_value(dict_nested, address_list):
cur = dict_nested
for path_item in address_list[:-2]:
try:
cur = cur[path_item]
except KeyError:
cur = cur[path_item] = {}
cur[address_list[-2]] = address_list[-1]
DictOfVehiclePrices = {'car':
{'ford':
{'mustang': 'expensive',
'other': 'cheap'},
'toyota':
{'big': 'moderate',
'small': 'cheap'}
},
'truck':
{'big': 'expensive',
'small': 'moderate'}
}
set_value(DictOfVehiclePrices,['car', 'ford', 'mustang', 'a'])
print DictOfVehiclePrices
STDOUT:
{'car': {'toyota': {'small': 'cheap', 'big': 'moderate'}, 'ford':
{'mustang': 'a', 'other': 'cheap'}}, 'truck': {'small': 'moderate',
'big': 'expensive'}}
def update_dict(parent, data, value):
'''
To update the value in the data if the data
is a nested dictionary
:param parent: list of parents
:param data: data dict in which value to be updated
:param value: Value to be updated in data dict
:return:
'''
if parent:
if isinstance(data[parent[0]], dict):
update_dict(parent[1:], data[parent[0]], value)
else:
data[parent[0]] = value
parent = ["test", "address", "area", "street", "locality", "country"]
data = {
"first_name": "ttcLoReSaa",
"test": {
"address": {
"area": {
"street": {
"locality": {
"country": "india"
}
}
}
}
}
}
update_dict(parent, data, "IN")
Here is a recursive function to update a nested dict based on a list of keys:
1.Trigger the update dict function with the required params
2.The function will iterate the list of keys, and retrieves the value from the dict.
3.If the retrieved value is dict, it pops the key from the list and also it updates the dict with the value of the key.
4.Sends the updated dict and list of keys to the same function recursively.
5.When the list gets empty, it means that we have reached the desired the key, where we need to apply our replacement. So if the list is empty, the funtion replaces the dict[key] with the value