Develop Reference

Comprehension list of nested dictionary to get values but got keys

Fairly new to list comprehension and have the_list that I want to extract the keys of nested dictionary but I got the values instead. What am I missing or what am I doing wrong?
the_list = [{'size': 0, 'values': [], 'start': 0}, {'size': 2, 'values': [{'user': {'name': 'anna', 'id': 10, 'displayName': 'Anna'}, 'category': 'Secretary'}, {'user': {'name': 'bob', 'id': 11, 'displayName': 'Bobby'}, 'category': 'Manager'}], 'start': 0}, {'size': 1, 'values': [{'user': {'name': 'claire', 'id': 13, 'displayName': 'Clarissa Claire'}, 'category': 'Secretary'}], 'start': 0}]
list_comprehension = []
list_comprehension = [x for x in the_list for x in the_list[1]['values'][0]]
print(list_comprehension)
>> ['user', 'category', 'user', 'category', 'user', 'category']
Want
list_comprehension = [[anna, Secretary], [bob, manager], [claire, Secretary]]

You could use this. I personnally try to avoid nested list comprehension as they are hard to read and debug.
[[x['category'], x['user']['displayName']] for nest_list in the_list for x in nest_list["values"] ]
Output:
[['Secretary', 'Anna'], ['Manager', 'Bobby'], ['Secretary', 'Clarissa Claire']]
EDIT:
A version that doesn't have a nested comprehension list. When doing it I realised that there was one more level than I realised that makes this version a bit long. So in the end I'm not sure which one I would use in prod.
result = []
dict_list = [nest_list["values"] for nest_list in the_list]
for elt in dict_list:
for d in elt:
result.append([d['category'], d['user']['displayName']])

I've come up with this solution, but it is not very readable ...
comprehensionList = [[user['user']['name'], user['category']] for x in the_list for user in x['values']]
# Output
[['anna', 'Secretary'], ['bob', 'Manager'], ['claire', 'Secretary']]

Adding key and value to dictionary in python based on other dictionaries

I am using for loop in python and every loop creates a dictionary. I have the below set of dictionaries created.
{'name': 'xxxx'}
{'name': 'yyyy','age':'28'}
{'name': 'zzzz','age':'27','sex':'F'}
My requirement is to compare all the dictionaries created and find out the missing key values and add the key to missing dictionaries and order every dictionary based on key. Below is the expected output
Expected output:
{'age':'','name': 'xxxx','sex':''}
{'age':'28','name': 'yyyy','sex':''}
{'age':'27','name': 'zzzz','sex':'F'}
How to achieve this in python.

If you want to modify the dicts in-place, dict.setdefault would be easy enough.
my_dicts = [
{'name': 'xxxx'},
{'name': 'yyyy','age':'28'},
{'name': 'zzzz','age':'27','sex':'F'},
]
desired_keys = ['name', 'age', 'sex']
for d in my_dicts:
for key in desired_keys:
d.setdefault(key, "")
print(my_dicts)
prints out
[
{'name': 'xxxx', 'age': '', 'sex': ''},
{'name': 'yyyy', 'age': '28', 'sex': ''},
{'name': 'zzzz', 'age': '27', 'sex': 'F'},
]
If you don't want to hard-code the desired_keys list, you can make it a set and gather it from the dicts before the loop above.
desired_keys = set()
for d in my_dicts:
desired_keys.update(set(d)) # update with keys from `d`
Another option, if you want new dicts instead of modifying them in place, is
desired_keys = ... # whichever method you like
empty_dict = dict.fromkeys(desired_keys, "")
new_dicts = [{**empty_dict, **d} for d in my_dicts]
EDIT based on comments:
This doesn't remove keys that are not there in desired keys.
This will leave only the desired keys:
desired_keys = ... # Must be a set
for d in my_dicts:
for key in desired_keys:
d.setdefault(key, "")
for key in set(d) - desired_keys:
d.pop(key)
However, at that point it might be easier to just create new dicts:
new_dicts = [
{key: d.get(value, "") for key in desired_keys}
for d in my_dicts
]

data = [{'name': 'xxxx'},
{'name': 'yyyy','age':'28'},
{'name': 'zzzz','age':'27','sex':'F'}]
First get the maximum, to get all the keys.
Then use dict.get to get default value as empty string for each of the keys, and sort the dictionary on key, you can combine List-comprehension and dict-comprehension:
allKD = max(data, key=len)
[dict(sorted({k:d.get(k, '') for k in allKD}.items(), key=lambda x:x[0])) for d in data]
OUTPUT:
[{'age': '', 'name': 'xxxx', 'sex': ''},
{'age': '28', 'name': 'yyyy', 'sex': ''},
{'age': '27', 'name': 'zzzz', 'sex': 'F'}]

One approach:
from operator import or_
from functools import reduce
lst = [{'name': 'xxxx'},
{'name': 'yyyy', 'age': '28'},
{'name': 'zzzz', 'age': '27', 'sex': 'F'}]
# find all the keys
keys = reduce(or_, map(dict.keys, lst))
# update each dictionary with the complement of the keys
for d in lst:
d.update(dict.fromkeys(keys - d.keys(), ""))
print(lst)
Output
[{'name': 'xxxx', 'age': '', 'sex': ''}, {'name': 'yyyy', 'age': '28', 'sex': ''}, {'name': 'zzzz', 'age': '27', 'sex': 'F'}]

Replace dictionaries keys with values from list

keys = ['id', 'name', 'address']
list = [{'Value': 1}, {'Value': 'Example name'}, {'VarCharValue': 'GA'}]
Looking for the most pythonic way to replace key dicts keys. I tried with for loop and list indexes but it was ugly. Excepted result:
list = [{'id': 1}, {'name': 'Example name'}, {'address': 'GA'}]

You can use a list comprehension with zip. To extract the only value in a dictionary d, you can use next(iter(d.values())) or list(d.values())[0].
K = ['id', 'name', 'address']
L = [{'Value': 1}, {'Value': 'Example name'}, {'VarCharValue': 'GA'}]
res = [{k: next(iter(v.values()))} for k, v in zip(K, L)]
[{'id': 1}, {'name': 'Example name'}, {'address': 'GA'}]

If you don't want to use iter(), you can use list(), which looks almost the same as jpp's solution
res = [{k: list(v.values())[0]} for k,v in zip(K,L)]
In this, you simply convert the dict_values object to a list, and get the first item, instead of getting the iterator and calling next on it.

Merging dictionaries of dictionaries

I'd like to know what be the best/most pythonic way to combine two dictionaries of dictionaries.
I have two dictionaries that look like so:
original = {'user1': {'COL': 'green', 'ID': '234235', 'NAME': 'andy'},
'user2': {'COL': 'blue', 'ID': '234234', 'NAME': 'john'}}
update = {'user1': {'COL': 'black', 'ID': '234235', 'NAME': 'andy'},
'user2': {'COL': 'purple', 'SPEC': 'na'}}
I would like to merge them together so that values found in the update dictionary would update the respective values in the original dictionary.
result = {'user1': {'COL': 'black', 'ID': '234235', 'NAME': 'andy'},
'user2': {'COL': 'purple', 'ID': '234234', 'NAME': 'john', 'SPEC': 'na'}}
I have tried using collections and copy, however, I end up replacing the entire value for the key.

That is just a simple for-loop:
for k in original:
original[k].update(update.get(k, {}))

for k in update:
if k in original:
original[k].update(update[k])
else:
original[k] = update[k]
This is assuming only one level of nesting. A deeper solution promises to be more complex. This also mutates the dictionary original, which may not be the behavior you want.

You could use double iteration if there are two levels each time:
for user, values in update.iteritems():
for k, v in values.iteritems():
original[user][k] = v

You may create a nested dict comprehension expression to achieve it:
{k1: {k2: update[k1].get(k2, v2) for k2, v2 in v1.items()} for k1, v1 in original.items()}
which will return value:
{'user2': {'ID': '234234', 'NAME': 'john', 'COL': 'purple'}, 'user1': {'ID': '234235', 'NAME': 'andy', 'COL': 'black'}}

How should I remove all dicts from a list that have None as one of their values?

Suppose I have a list like so:
[{'name': 'Blah1', 'age': x}, {'name': 'Blah2', 'age': y}, {'name': None, 'age': None}]
It is guaranteed that both 'name' and 'age' values will either be filled or empty.
I tried this:
for person_dict in list:
if person_dict['name'] == None:
list.remove(person_dict)
But obviously that does not work because the for loop skips over an index sometimes and ignores some blank people.
I am relatively new to Python, and I am wondering if there is a list method that can target dicts with a certain value associated with a key.
EDIT: Fixed tuple notation to list as comments pointed out

Just test for the presence of None in the dict's values to test ALL dict keys for the None value:
>>> ToD=({'name': 'Blah1', 'age': 'x'}, {'name': 'Blah2', 'age': 'y'}, {'name': None, 'age': None})
>>> [e for e in ToD if None not in e.values()]
[{'age': 'x', 'name': 'Blah1'}, {'age': 'y', 'name': 'Blah2'}]
Or, use filter:
>>> filter(lambda d: None not in d.values(), ToD)
({'age': 'x', 'name': 'Blah1'}, {'age': 'y', 'name': 'Blah2'})
Or, if it is a limited test to 'name':
>>> filter(lambda d: d['name'], ToD)
({'age': 'x', 'name': 'Blah1'}, {'age': 'y', 'name': 'Blah2'})

You can use list comprehension as a filter like this
[c_dict for c_dict in dict_lst if all(c_dict[key] is not None for key in c_dict)]
This will make sure that you get only the dictionaries where all the values are not None.

for index,person_dict in enumerate(lis):
if person_dict['name'] == None:
del lis[index]
you can also try
lis=[person_dict for person_dict in lis if person_dict['name'] != None]
never use List as variable

You can create new list with accepted data. If you have tuple then you have to create new list.
List comprehension could be faster but this version is more readable for beginners.
data = ({'name': 'Blah1', 'age': 'x'}, {'name': 'Blah2', 'age': 'y'}, {'name': None, 'age': None})
new_data = []
for x in data:
if x['name']: # if x['name'] is not None and x['name'] != ''
new_data.append(x)
print new_data