I'm looking for a concise way of taking two dicts that have a common key/value, and copying a key and value into one of the dicts. Example:
d1 = [{'name': 'john', 'uid': 'ax01', 'phone': '555-555-5555'},
{'name': 'jane', 'uid': 'ax02', 'phone': '555-555-5555'},
{'name': 'jimmy', 'uid': 'ax03', 'phone': '555-555-5555'}]
d2 = [{'uid': 'ax01', 'orderid': '9999', 'note': 'testing this'},
{'uid': 'ax02', 'orderid': '6666', 'note': 'testing this'},
{'uid': 'ax03', 'orderid': '7777', 'note': 'testing this'}]
Here uid is the key that I want to use to then copy the orderid key and the value for that matching data point. In the end I'd get something like:
output = [
{'name': 'john', 'uid': 'ax01', 'phone': '555-555-5555', 'orderid': '9999'},
{'name': 'jane', 'uid': 'ax02', 'phone': '555-555-5555', 'orderid': '6666'},
{'name': 'jimmy', 'uid': 'ax03', 'phone': '555-555-5555', 'orderid': '7777'}
]
Where the orderid is pulled into d1. I'm looking for the pythonic way if at all possible.
You can use dict() to copy one dictionary and pass in the extra key. You need to create a mapping from uid to orderid first though:
uid_to_orderid = {d['uid']: d['orderid'] for d in d2}
output = [dict(d, orderid=uid_to_orderid[d['uid']]) for d in d1]
This assumes that you want to leave the dictionaries in d1 untouched otherwise. Other assumptions made are that uid values are unique, and that all uid values in d1 are present in d2.
Demo:
>>> d1 = [{'name': 'john', 'uid': 'ax01', 'phone': '555-555-5555'},
... {'name': 'jane', 'uid': 'ax02', 'phone': '555-555-5555'},
... {'name': 'jimmy', 'uid': 'ax03', 'phone': '555-555-5555'}]
>>> d2 = [{'uid': 'ax01', 'orderid': '9999', 'note': 'testing this'},
... {'uid': 'ax02', 'orderid': '6666', 'note': 'testing this'},
... {'uid': 'ax03', 'orderid': '7777', 'note': 'testing this'}]
>>> uid_to_orderid = {d['uid']: d['orderid'] for d in d2}
>>> [dict(d, orderid=uid_to_orderid[d['uid']]) for d in d1]
[{'orderid': '9999', 'phone': '555-555-5555', 'name': 'john', 'uid': 'ax01'}, {'orderid': '6666', 'phone': '555-555-5555', 'name': 'jane', 'uid': 'ax02'}, {'orderid': '7777', 'phone': '555-555-5555', 'name': 'jimmy', 'uid': 'ax03'}]
Great answer #Martijn Pieters
I can't think of a clever way to protect against missing data other something brutish like this:
d1 = [{'name': 'john', 'uid': 'ax01', 'phone': '555-555-5555'},
{'name': 'jane', 'uid': 'ax02', 'phone': '555-555-5555'},
{'name': 'jimmy', 'uid': 'ax03', 'phone': '555-555-5555'},
{'name': 'jack', 'uid': 'ax04', 'phone': '555-555-5555'},
{'name': 'joan', 'uid': 'ax05', 'phone': '555-555-5555'}]
d2 = [{'uid': 'ax01', 'orderid': '9999', 'note': 'testing this'},
{'uid': 'ax02', 'orderid': '6666', 'note': 'testing this'},
{'uid': 'ax03', 'orderid': '6666', 'note': 'testing this'},
{'uid': 'ax05', 'orderid-not-here': '7777', 'note': 'testing this'}]
def get_orderid(search):
match = [x for x in d2 if x['uid']==search]
if len(match) == 0:
return 'None'
else:
return match[0].get('orderid','None')
[dict(d, orderid=get_orderid(d['uid'])) for d in d1]
[ {'name': 'john', 'orderid': '9999', 'phone': '555-555-5555', 'uid': 'ax01'},
{'name': 'jane', 'orderid': '6666', 'phone': '555-555-5555', 'uid': 'ax02'},
{'name': 'jimmy', 'orderid': '6666', 'phone': '555-555-5555', 'uid': 'ax03'},
{'name': 'jack', 'orderid': 'None', 'phone': '555-555-5555', 'uid': 'ax04'},
{'name': 'joan', 'orderid': 'None', 'phone': '555-555-5555', 'uid': 'ax05'}]
`
Related
I have a list of dictionaries that looks like this:
test_dict = [{'id': 0, 'name': ['Jim'], 'lastname': ['kkk']}, {'id': 1, 'name': ['John'], 'lastname': ['lll']}]
test_dict
[{'id': 0, 'name': ['Jim'], 'lastname': ['kkk']},
{'id': 1, 'name': ['John'], 'lastname': ['lll']}]
I would like to create another dictionary that will have as keys, the ids of the test_dict.
The output I am looking for, looks like this:
test_dict_f = {'0': {'name': ['Jim'], 'lastname': ['kkk']},
'1': {'name': ['John'], 'lastname': ['lll']}}
test_dict_f
{'0': {'name': ['Jim'], 'lastname': ['kkk']},
'1': {'name': ['John'], 'lastname': ['lll']}}
Any ideas how I could achieve this ?
Try this in one line:
result = {str(i["id"]): {"name": i["name"], "lastname": i["lastname"]} for i in test_dict}
the result will be:
{'0': {'name': ['Jim'], 'lastname': ['kkk']},
'1': {'name': ['John'], 'lastname': ['lll']}}
Here's another way to do it that doesn't rely on knowledge of any keys other than 'id'. Note that this is destructive:
test_dict = [{'id': 0, 'name': ['Jim'], 'lastname': ['kkk']}, {'id': 1, 'name': ['John'], 'lastname': ['lll']}]
test_dict_f = {}
for d in test_dict:
id_ = d.pop('id')
test_dict_f[str(id_)] = d
print(test_dict_f)
Output:
{'0': {'name': ['Jim'], 'lastname': ['kkk']}, '1': {'name': ['John'], 'lastname': ['lll']}}
test_dict = [{'id': 0, 'name': ['Jim'], 'lastname': ['kkk']}, {'id': 1, 'name': ['John'], 'lastname': ['lll']}]
pp({i["id"]: {k:v for k,v in i.items() if k!="id"} for i in test_dict})
or event much funnier:
pp({i["id"]: i|{"id":i["id"]} for i in test_dict})
{0: {'name': ['Jim'], 'lastname': ['kkk']},
1: {'name': ['John'], 'lastname': ['lll']}}
I have a problem. I have a list and this in turn contains dicts. The problem is that the dicts can have different sizes. That some elements are missing in some dicts. Is there an option to fill the dicts with the missing attributes? It is only a matter of filling up the keys and the values can simply be filled up with None.
The following example
[
{'_id': 'orders/213123',
'contactEditor': {'name': 'Max Power',
'phone': '1234567'},
'isCompleteDelivery': False,
},
{'_id': 'orders/2173823',
'contactEditor': {'name': 'Michael'},
},
]
It turns out that the first element has the most keys and the second element is missing the following keys phone and isCompleteDelivery.
this is filled in and it looks like this
[
{'_id': 'orders/213123',
'contactEditor': {'name': 'Max Power',
'phone': '1234567'},
'isCompleteDelivery': False,
},
{'_id': 'orders/2173823',
'contactEditor': {'name': 'Michael',
'phone': None},
'isCompleteDelivery': None,
},
]
This gives me which dict has the most elements and the number.
%%time
highest_element = 0
elements = 0
for i, element in enumerate(myList):
counted = count(myList[i])
if(counted > elements):
elements = counted
highest_element = i
print(str(elements) + " " + str(highest_element))
myList = [
{'_id': 'orders/213123',
'contactEditor': {'name': 'Max Power',
'phone': '1234567',
'email': 'max#power.com'},
'contactSoldToParty': {'name': 'Max Not',
'phone': '123456789',
'email': 'maxnot#power.com'},
'isCompleteDelivery': False,
'metaData': {'dataOriginSystem': 'Goods',
'dataOriginWasCreatedTime': '10:12:12',},
'orderDate': '2021-02-22',
'orderDateBuyer': '2021-02-22',
},
{'_id': 'orders/12323',
'contactEditor': {'name': 'Max Power2',
'phone': '1234567',
'email': 'max#power.com'},
'contactSoldToParty': {'name': 'Max Not',
'phone': '123456789',
'email': 'maxnot#power.com'},
'isCompleteDelivery': False,
'metaData': {'dataOriginSystem': 'Goods',
'dataOriginWasCreatedTime': '10:12:12',},
},
{'_id': 'orders/12323',
'contactEditor': {'name': 'Max Power2',
'phone': '1234567',
'email': 'max#power.com'},
'contactSoldToParty': {'name': 'Max Not',
'phone': '123456789',
'email': 'maxnot#power.com'},
'orderDate': '2021-02-22',
'orderDateBuyer': '2021-02-22',
},
{'_id': 'orders/12323',
'contactEditor': {'name': 'Max Power2',
'phone': '1234567',
'email': 'max#power.com'},
'contactSoldToParty': {'name': 'Max Not',},
'isCompleteDelivery': False,
},
]
If I understand the problem correctly, it seems that you need to construct a superset of all possible keys. Once you have that you can iterate over the list and set default values in each dictionary. Now that the "top level" dictionaries have been adjusted you need to step down a level and do the same thing for the nested dictionaries. This code is limited to the data structure as shown in the question.
list_ = [
{'_id': 'orders/213123',
'contactEditor': {'name': 'Max Power',
'phone': '1234567'},
'isCompleteDelivery': False,
},
{'_id': 'orders/2173823',
'contactEditor': {'name': 'Michael'},
}
]
def modkv(L, K):
superset_ = set(key for dict_ in list_ for key in dict_[K])
for dict_ in L:
for key in superset_:
dict_[K].setdefault(key, None)
superset = set(key for dict_ in list_ for key in dict_)
for dict_ in list_:
for key in superset:
dict_.setdefault(key, None)
for dict_ in list_:
for k, v in dict_.items():
if isinstance(v, dict):
modkv(list_, k)
print(list_)
Output:
[{'_id': 'orders/213123', 'contactEditor': {'name': 'Max Power', 'phone': '1234567'}, 'isCompleteDelivery': False}, {'_id': 'orders/2173823', 'contactEditor': {'name': 'Michael', 'phone': None}, 'isCompleteDelivery': None}]
This code make a key if not present in dictionary with the value None.
lst = [
{'_id': 'orders/213123',
'contactEditor': {'name': 'Max Power',
'phone': '1234567'},
'isCompleteDelivery': False,
},
{'_id': 'orders/2173823',
'contactEditor': {'name': 'Michael'},
},
]
complete_dict = {'_id': None,
'contactEditor': {'name': None,
'phone': None},
'isCompleteDelivery': None,
}
# here complete dictionary means which is contain all possible keys.
all_keys = complete_dict.keys()
for i,value in enumerate(lst):
for a in all_keys:
try:
value[a]
except KeyError:
value[a]=None
else:
v = value[a]
if isinstance(v,dict):
for a2 in complete_dict[a].keys():
try:
v[a2]
except KeyError:
v[a2]=None
value[a]=v
del lst[i]
lst.insert(i,value)
print(lst)
OUTPUT
[{'_id': 'orders/213123',
'contactEditor': {'name': 'Max Power', 'phone': '1234567'},
'isCompleteDelivery': False},
{'_id': 'orders/2173823',
'contactEditor': {'name': 'Michael', 'phone': None},
'isCompleteDelivery': None}]
I have o first list A=
[{'name': 'PASSWORD', 'id': '5f2496e5-dc40-418a-92e0-098e4642a92e'},
{'name': 'PERSON_NAME', 'id': '3a255440-e2aa-4c4d-993f-4cdef3237920'},
{'name': 'PERU_DNI_NUMBER', 'id': '41f41303-4a71-4732-a8a4-0eecea464562'},
{'name': 'PHONE_NUMBER', 'id': 'ac24413b-bb8f-4adc-ada5-a984f145a70b'},
{'name': 'POLAND_NATIONAL_ID_NUMBER',
'id': '32c49d92-6d5f-408e-b41e-dfec76ceae6a'}]
and I have a second list B :
[{'name': 'PHONE_NUMBER', 'count': '96'}]
I want to filter the first list based on the second in order to have the following list :
[{'name': 'PHONE_NUMBER', 'count': '96','id': 'ac24413b-bb8f-4adc-ada5-a984f145a70b'}.
I have used the following code but I dont get the right ouptut:
filtered = []
for x,i in DLP_job[i]['name']:
if x,i in ids[i]['name']:
filtered.append(x)
print(filtered)
Here is the solution
A = [{'name': 'PASSWORD', 'id': '5f2496e5-dc40-418a-92e0-098e4642a92e'},
{'name': 'PERSON_NAME', 'id': '3a255440-e2aa-4c4d-993f-4cdef3237920'},
{'name': 'PERU_DNI_NUMBER', 'id': '41f41303-4a71-4732-a8a4-0eecea464562'},
{'name': 'PHONE_NUMBER', 'id': 'ac24413b-bb8f-4adc-ada5-a984f145a70b'},
{'name': 'POLAND_NATIONAL_ID_NUMBER',
'id': '32c49d92-6d5f-408e-b41e-dfec76ceae6a'}]
B = [{'name': 'PHONE_NUMBER', 'count': '96'}]
print([{**x, **y} for x in A for y in B if y['name'] == x['name']])
One way is to walk both lists, and wherever you have matching name keys, use the merger of the 2 dicts:
l1 = [{'name': 'PASSWORD', 'id': '5f2496e5-dc40-418a-92e0-098e4642a92e'},
{'name': 'PERSON_NAME', 'id': '3a255440-e2aa-4c4d-993f-4cdef3237920'},
{'name': 'PERU_DNI_NUMBER', 'id': '41f41303-4a71-4732-a8a4-0eecea464562'},
{'name': 'PHONE_NUMBER', 'id': 'ac24413b-bb8f-4adc-ada5-a984f145a70b'},
{'name': 'POLAND_NATIONAL_ID_NUMBER',
'id': '32c49d92-6d5f-408e-b41e-dfec76ceae6a'}]
l2 = [{'name': 'PHONE_NUMBER', 'count': '96'}, {'name': 'PERSON_NAME', 'count': '100'}]
result = []
for d2 in l2:
for d1 in l1:
if d1['name'] == d2['name']:
result.append({**d1, **d2})
print(result)
[{'name': 'PHONE_NUMBER', 'id': 'ac24413b-bb8f-4adc-ada5-a984f145a70b', 'count': '96'},
{'name': 'PERSON_NAME', 'id': '3a255440-e2aa-4c4d-993f-4cdef3237920', 'count': '100'}]
I have a scenario where I have three dictionaries which I want merge into one but the condition is while I compare the three dictionaries with key name if there are duplicates need to remove them.
Here is what I have tried :
dict1= {'d1': [{'name': 'app1', 'id': 7134}, {'name': 'app2', 'id': 242}, {'name': 'yest app', 'id': 67},{'name': 'abc jam app', 'id': 6098}]}
dict2= {'d2': [{'name': 'app1 ', 'id': 30}, {'name': 'app2', 'id': 82}, {'name': 'yest app', 'id': 17}]}
dict3= {'d3': [{'name': 'app1', 'id': 70}, {'name': 'app2', 'id': 2582},{'name': 'availabla2z', 'id': 6667}]}
dict2 = {i:j for i,j in dict2.items() if i not in dict1}
dict3 = {i:j for i,j in dict3.items() if i not in dict2}
But the same do not give results also I am not sure how to compare three dicts for that matter.
and since if you look at the data dict1 is having an element 'name': 'app1' where as the same element is there in dict2 like this 'name': 'app1 ' (with a space) not sure how to format this as well and get a final dict like below as result.
{'final': [{'name': 'app1 ', 'id': 30}, {'name': 'app2', 'id': 82}, {'name': 'yest app', 'id': 17},{'name': 'abc jam app', 'id': 6098},{'name': 'availabla2z', 'id': 6667}]}
Here's a solution, taking advantage of this other SO answer (useful for python-2.x alternatives) that will remove duplicates without any particular rule:
final_dict = dict()
final_dict["final"] = dict1["d1"] + dict2["d2"] + dict3["d3"]
final_dict["final"] = list(
{v['name'].strip():v for v in final_dict["final"]}.values()
) # see usage of .strip() to handle space problems you mention
print(final_dict)
Result:
{'final': [
{'name': 'app1', 'id': 70},
{'name': 'app2', 'id': 2582},
{'name': 'yest app', 'id': 17},
{'name': 'abc jam app', 'id': 6098},
{'name': 'availabla2z', 'id': 6667}]
}
Here is a working Updater code:
dict1= {'d1': [{'name': 'app1', 'id': 7134}, {'name': 'app2', 'id': 242}, {'name': 'yest app', 'id': 67},{'name': 'abc jam app', 'id': 6098}]}
dict2= {'d2': [{'name': 'app1 ', 'id': 30}, {'name': 'app2', 'id': 82}, {'name': 'yest app', 'id': 17}]}
dict3= {'d3': [{'name': 'app1', 'id': 70}, {'name': 'app2', 'id': 2582},{'name': 'availabla2z', 'id': 6667}]}
final = {'final':[]}
for i in dict1['d1']:
final['final'].append(i)
for k,l in zip(dict3['d3'],range(len(dict1['d1']))):
if k['name'] == final['final'][l]['name']:
final['final'][l].update(k)
else:
final['final'].append(k)
for j,l in zip(dict2['d2'],range(len(dict1['d1']))):
if j['name'].strip() == final['final'][l]['name'].strip():
final['final'][l].update(j)
else:
final['final'].append(j)
This gives:
{'final': [{'name': 'app1 ', 'id': 30}, {'name': 'app2', 'id': 82}, {'name': 'yest app', 'id': 17}, {'name': 'abc jam app', 'id': 6098}, {'name': 'availabla2z', 'id': 6667}]}
You could group all dictionaries together by name using defaultdict:
from collections import defaultdict
d = defaultdict(list)
for lst in (dict1.values(), dict2.values(), dict3.values()):
for sublst in lst:
for dic in sublst:
d[dic["name"].strip()].append(dic)
Then choose the dictionaries with the smallest id value using min(). This still works for the requirements since it still chooses one dictionary and matches the output requested.
from operator import itemgetter
result = {'field': [min(x, key=itemgetter('id')) for x in d.values()]}
print(result)
Output:
{'field': [{'name': 'app1', 'id': 30}, {'name': 'app2', 'id': 82}, {'name': 'yest app', 'id': 17}, {'name': 'abc jam app', 'id': 6098}, {'name': 'availabla2z', 'id': 6667}]}
I have 2 lists with the following data structure:
array1:
[{'student': {'name': 'abc'}, 'address': 'add_abc'},
{'student': {'name': 'xyz'}, 'address': 'add_xyz'}]
array2:
[{'student': {'name': 'abc'}, 'address': 'add_abc'},
{'student': {'name': 'rst'}, 'address': 'add_rst'}]
I want to have an array3 with union of the above 2 lists
array3:
[{'student': {'name': 'abc'}, 'address': 'add_abc'},
{'student': {'name': 'rst'}, 'address': 'add_rst'},
{'student': {'name': 'xyz'}, 'address': 'add_xyz'}]
How can I do that in Python?
They are lists, not arrays, but here is a solution:
a1 = [{'student': {'name': 'abc'}, 'address': 'add_abc'},
{'student': {'name': 'xyz'}, 'address': 'add_xyz'}]
a2 = [{'student': {'name': 'abc'}, 'address': 'add_abc'},
{'student': {'name': 'rst'}, 'address': 'add_rst'}]
a3 = a1 + [i for i in a2 if i not in a1]
the value of a3 will be
[{'student': {'name': 'abc'}, 'address': 'add_abc'},
{'student': {'name': 'xyz'}, 'address': 'add_xyz'},
{'student': {'name': 'rst'}, 'address': 'add_rst'}]
If you want your resulting list to be sorted (by student name, for example), simply use sort method:
a3.sort(key=lambda x: x["student"]["name"])
and the resulting a3 would be:
[{'student': {'name': 'abc'}, 'address': 'add_abc'},
{'student': {'name': 'rst'}, 'address': 'add_rst'},
{'student': {'name': 'xyz'}, 'address': 'add_xyz'}]
Normally you would simply use set(a1 + a2) but the problem here is that you have dicts inside your lists and they are unhashable, therefore they cannot be set items.