Develop Reference

Adding key and value to dictionary in python based on other dictionaries

I am using for loop in python and every loop creates a dictionary. I have the below set of dictionaries created.
{'name': 'xxxx'}
{'name': 'yyyy','age':'28'}
{'name': 'zzzz','age':'27','sex':'F'}
My requirement is to compare all the dictionaries created and find out the missing key values and add the key to missing dictionaries and order every dictionary based on key. Below is the expected output
Expected output:
{'age':'','name': 'xxxx','sex':''}
{'age':'28','name': 'yyyy','sex':''}
{'age':'27','name': 'zzzz','sex':'F'}
How to achieve this in python.

If you want to modify the dicts in-place, dict.setdefault would be easy enough.
my_dicts = [
{'name': 'xxxx'},
{'name': 'yyyy','age':'28'},
{'name': 'zzzz','age':'27','sex':'F'},
]
desired_keys = ['name', 'age', 'sex']
for d in my_dicts:
for key in desired_keys:
d.setdefault(key, "")
print(my_dicts)
prints out
[
{'name': 'xxxx', 'age': '', 'sex': ''},
{'name': 'yyyy', 'age': '28', 'sex': ''},
{'name': 'zzzz', 'age': '27', 'sex': 'F'},
]
If you don't want to hard-code the desired_keys list, you can make it a set and gather it from the dicts before the loop above.
desired_keys = set()
for d in my_dicts:
desired_keys.update(set(d)) # update with keys from `d`
Another option, if you want new dicts instead of modifying them in place, is
desired_keys = ... # whichever method you like
empty_dict = dict.fromkeys(desired_keys, "")
new_dicts = [{**empty_dict, **d} for d in my_dicts]
EDIT based on comments:
This doesn't remove keys that are not there in desired keys.
This will leave only the desired keys:
desired_keys = ... # Must be a set
for d in my_dicts:
for key in desired_keys:
d.setdefault(key, "")
for key in set(d) - desired_keys:
d.pop(key)
However, at that point it might be easier to just create new dicts:
new_dicts = [
{key: d.get(value, "") for key in desired_keys}
for d in my_dicts
]

data = [{'name': 'xxxx'},
{'name': 'yyyy','age':'28'},
{'name': 'zzzz','age':'27','sex':'F'}]
First get the maximum, to get all the keys.
Then use dict.get to get default value as empty string for each of the keys, and sort the dictionary on key, you can combine List-comprehension and dict-comprehension:
allKD = max(data, key=len)
[dict(sorted({k:d.get(k, '') for k in allKD}.items(), key=lambda x:x[0])) for d in data]
OUTPUT:
[{'age': '', 'name': 'xxxx', 'sex': ''},
{'age': '28', 'name': 'yyyy', 'sex': ''},
{'age': '27', 'name': 'zzzz', 'sex': 'F'}]

One approach:
from operator import or_
from functools import reduce
lst = [{'name': 'xxxx'},
{'name': 'yyyy', 'age': '28'},
{'name': 'zzzz', 'age': '27', 'sex': 'F'}]
# find all the keys
keys = reduce(or_, map(dict.keys, lst))
# update each dictionary with the complement of the keys
for d in lst:
d.update(dict.fromkeys(keys - d.keys(), ""))
print(lst)
Output
[{'name': 'xxxx', 'age': '', 'sex': ''}, {'name': 'yyyy', 'age': '28', 'sex': ''}, {'name': 'zzzz', 'age': '27', 'sex': 'F'}]

unflatten a python dictionary in an efficient way

assume you have a list of dictionaries like this:
{'id':1,
'station': 'LYO',
'country': 'France',
'classes': 'EU',
'label': 'LYO CS',
'color': 'orange',
'population':100000,
'university':yes}
I would like to create:
{'data':{'id':1,
'station': 'LYO',
'country': 'France'},
'classes': 'EU',
'label': 'LYO CS',
'color': 'orange',
'population':100000,
'university':yes}
basically include in data subdictionary the key-values of data_items= ['id','station','country']
the obvious long way to do that is:
data_keys = ['id','station']
data_sub_dict = {'data':{el:node[el] for el in data_keys}}
rest_sub_dict = {el:node[el] for el in node.keys() if el not in data_keys}
dict_result ={}
dict_result.update(data_sub_dict)
dict_result.update(rest_sub_dict)
This is ok, but it smells to me as not very pythonic.
Would you do it otherwise? (more compacted?, other ways to do it?)
thanks

Why updates?
It's possible to write all in one instruction using dict constructor:
data = {'id':1,
'station': 'LYO',
'country': 'France',
'classes': 'EU',
'label': 'LYO CS',
'color': 'orange',
'population':100000,
'university':'yes'}
dict(
{'data': {k:v for k,v in data.items() if k in ('id', 'station','country')}},
**{k:v for k,v in data.items() if k in ('classes', 'color', 'population', 'university')})
Result:
{'data': {'id': 1, 'station': 'LYO', 'country': 'France'},
'classes': 'EU',
'color': 'orange',
'population': 100000,
'university': 'yes'}

I would suggest to write a function that splits a dictionary into two, while providing a list of keys:
def filterByKeys(d,keys):
a={}; b={};
for k in d:
if k in keys: a[k] = d[k]
else : b[k] = d[k]
return (a,b)
d1, d2 = filterByKeys(d,['id','station'])
{'data':d1,**d2}
While this approach does not use list comprehension for iterating the dictionary, it becomes very easy to understand.

Replace dictionaries keys with values from list

keys = ['id', 'name', 'address']
list = [{'Value': 1}, {'Value': 'Example name'}, {'VarCharValue': 'GA'}]
Looking for the most pythonic way to replace key dicts keys. I tried with for loop and list indexes but it was ugly. Excepted result:
list = [{'id': 1}, {'name': 'Example name'}, {'address': 'GA'}]

You can use a list comprehension with zip. To extract the only value in a dictionary d, you can use next(iter(d.values())) or list(d.values())[0].
K = ['id', 'name', 'address']
L = [{'Value': 1}, {'Value': 'Example name'}, {'VarCharValue': 'GA'}]
res = [{k: next(iter(v.values()))} for k, v in zip(K, L)]
[{'id': 1}, {'name': 'Example name'}, {'address': 'GA'}]

If you don't want to use iter(), you can use list(), which looks almost the same as jpp's solution
res = [{k: list(v.values())[0]} for k,v in zip(K,L)]
In this, you simply convert the dict_values object to a list, and get the first item, instead of getting the iterator and calling next on it.

How should I remove all dicts from a list that have None as one of their values?

Suppose I have a list like so:
[{'name': 'Blah1', 'age': x}, {'name': 'Blah2', 'age': y}, {'name': None, 'age': None}]
It is guaranteed that both 'name' and 'age' values will either be filled or empty.
I tried this:
for person_dict in list:
if person_dict['name'] == None:
list.remove(person_dict)
But obviously that does not work because the for loop skips over an index sometimes and ignores some blank people.
I am relatively new to Python, and I am wondering if there is a list method that can target dicts with a certain value associated with a key.
EDIT: Fixed tuple notation to list as comments pointed out

Just test for the presence of None in the dict's values to test ALL dict keys for the None value:
>>> ToD=({'name': 'Blah1', 'age': 'x'}, {'name': 'Blah2', 'age': 'y'}, {'name': None, 'age': None})
>>> [e for e in ToD if None not in e.values()]
[{'age': 'x', 'name': 'Blah1'}, {'age': 'y', 'name': 'Blah2'}]
Or, use filter:
>>> filter(lambda d: None not in d.values(), ToD)
({'age': 'x', 'name': 'Blah1'}, {'age': 'y', 'name': 'Blah2'})
Or, if it is a limited test to 'name':
>>> filter(lambda d: d['name'], ToD)
({'age': 'x', 'name': 'Blah1'}, {'age': 'y', 'name': 'Blah2'})

You can use list comprehension as a filter like this
[c_dict for c_dict in dict_lst if all(c_dict[key] is not None for key in c_dict)]
This will make sure that you get only the dictionaries where all the values are not None.

for index,person_dict in enumerate(lis):
if person_dict['name'] == None:
del lis[index]
you can also try
lis=[person_dict for person_dict in lis if person_dict['name'] != None]
never use List as variable

You can create new list with accepted data. If you have tuple then you have to create new list.
List comprehension could be faster but this version is more readable for beginners.
data = ({'name': 'Blah1', 'age': 'x'}, {'name': 'Blah2', 'age': 'y'}, {'name': None, 'age': None})
new_data = []
for x in data:
if x['name']: # if x['name'] is not None and x['name'] != ''
new_data.append(x)
print new_data

How to slice a string in Python using a dictionary containing character positions?

I have a dictionary containing the character positions of different fields in a string. I'd like to use that information to slice the string. I'm not really sure how to best explain this, but the example should make it clear:
input:
mappings = {'name': (0,4), 'job': (4,11), 'color': (11, 15)}
data = "JohnChemistBlue"
desired output:
{'name': 'John', 'job': 'Chemist', 'color': 'Blue'}
Please disregard the fact that jobs, colors and names obviously vary in character length. I'm parsing fixed-length fields but simplified it here for illustrative purposes.

>>> dict((f, data[slice(*p)]) for f, p in mappings.iteritems())
{'color': 'Blue', 'job': 'Chemist', 'name': 'John'}

dict([(name, data[range[0]:range[1]]) for name, range in mappings.iteritems()])

>>> dict([(k, data[ mappings[k][0]:mappings[k][1] ]) for k in mappings])
{'color': 'Blue', 'job': 'Chemist', 'name': 'John'}
or with a generator instead of a list (probably more efficient):
>>> dict(((k, data[ mappings[k][0]:mappings[k][1] ]) for k in mappings))
{'color': 'Blue', 'job': 'Chemist', 'name': 'John'}