Develop Reference

Create a new dictionary with the key-value pair from values in a list of dictionaries based on matches from a separate list

I am trying to get a new dictionary with the k: v pair from values in a list of dictionary based on matches from a separate list.
The casing is different.
My data looks like this:
list_of_dicts = [
{'fieldname': 'Id', 'source': 'microsoft', 'nullable': True, 'type': 'int'},
{'fieldname': 'FirstName', 'source': 'microsoft', 'nullable': True, 'type': 'string'},
{'fieldname': 'LastName', 'source': 'microsoft', 'nullable': False, 'type': 'string'},
{'fieldname': 'Address1', 'source': 'microsoft', 'nullable': False, 'type': 'string'}
]
fieldname_list = ['FIRSTNAME', 'LASTNAME']
From this I would like to create a new dictionary as follows:
new_dict = {'FirstName': 'string', 'LastName': 'string'}
I think this should be possible with a dictionary comprehension, but I can't work it out - can anyone help?

Do you want to do?
list_of_dicts = [
{'fieldname': 'Id', 'source': 'microsoft', 'nullable': True, 'type': 'int'},
{'fieldname': 'FirstName', 'source': 'microsoft', 'nullable': True, 'type': 'string'},
{'fieldname': 'LastName', 'source': 'microsoft', 'nullable': False, 'type': 'string'},
{'fieldname': 'Address1', 'source': 'microsoft', 'nullable': False, 'type': 'string'}
]
fieldname_list = ['FIRSTNAME', 'LASTNAME']
new_dict = {x['fieldname']: x['type'] for x in list_of_dicts if x['fieldname'].upper() in fieldname_list}
print(new_dict)
# output: {'FirstName': 'string', 'LastName': 'string'}

How to rename keys in a dictionary and make a dataframe of it?

I have a complex situation which I hope to solve and which might profit us all. I collected data from my API, added a pagination and inserted the complete data package in a tuple named q1 and finally I have made a dictionary named dict_1of that tuple which looks like this:
dict_1 = {100: {'ID': 100, 'DKSTGFase': None, 'DK': False, 'KM': None,
'Country: {'Name': GE', 'City': {'Name': 'Berlin'}},
'Type': {'Name': '219'}, 'DKObject': {'Name': '8555', 'Object': {'Name': 'Car'}},
'Order': {'OrderId': 101, 'CreatedOn': '2018-07-06T16:54:36.783+02:00',
'ModifiedOn': '2018-07-06T16:54:36.783+02:00',
'Name': Audi, 'Client': {‘1’ }}, 'DKComponent': {'Name': ‘John’}},
{200: {'ID': 200, 'DKSTGFase': None, 'DK': False, ' KM ': None,
'Country: {'Name': ES', 'City': {'Name': 'Madrid'}}, 'Type': {'Name': '220'},
'DKObject': {'Name': '8556', 'Object': {'Name': 'Car'}},
'Order': {'OrderId': 102, 'CreatedOn': '2018-07-06T16:54:36.783+02:00',
'ModifiedOn': '2018-07-06T16:54:36.783+02:00',
'Name': Mercedes, 'Client': {‘2’ }}, 'DKComponent': {'Name': ‘Sergio’}},
Please note that in the above dictionary I have just stated 2 records. The actual dictionary has 1400 records till it reaches ID 1500.
Now I want to 2 things:
I want to change some keys for all the records. key DK has to become DK1. Key Name in Country has to become Name1 and Name in Object has to become 'Name2'
The second thing I want is to make a dataFrame of the whole bunch of data. My expected outcome is:
This is my code:
q1 = response_2.json()
next_link = q1['#odata.nextLink']
q1 = [tuple(q1.values())]
while next_link:
new_response = requests.get(next_link, headers=headers, proxies=proxies)
new_data = new_response.json()
q1.append(tuple(new_data.values()))
next_link = new_data.get('#odata.nextLink', None)
dict_1 = {
record['ID']: record
for tup in q1
for record in tup[2]
}
#print(dict_1)
for x in dict_1.values():
x['DK1'] = x['DK']
x['Country']['Name1'] = x['Country']['Name']
x['Object']['Name2'] = x['Object']['Name']
df = pd.DataFrame(dict_1)
When i run this I receive the following Error:
Traceback (most recent call last):
File "c:\data\FF\Desktop\Python\PythongMySQL\Talky.py", line 57, in <module>
x['Country']['Name1'] = x['Country']['Name']
TypeError: 'NoneType' object is not subscriptable

working code
lists=[]
alldict=[{100: {'ID': 100, 'DKSTGFase': None, 'DK': False, 'KM': None,
'Country': {'Name': 'GE', 'City': {'Name': 'Berlin'}},
'Type': {'Name': '219'}, 'DKObject': {'Name': '8555', 'Object': {'Name': 'Car'}},
'Order': {'OrderId': 101, 'CreatedOn': '2018-07-06T16:54:36.783+02:00',
'ModifiedOn': '2018-07-06T16:54:36.783+02:00',
'Name': 'Audi', 'Client': {'1' }}, 'DKComponent': {'Name': 'John'}}}]
for eachdict in alldict:
key=list(eachdict.keys())[0]
eachdict[key]['DK1']=eachdict[key]['DK']
del eachdict[key]['DK']
eachdict[key]['Country']['Name1']=eachdict[key]['Country']['Name']
del eachdict[key]['Country']['Name']
eachdict[key]['DKObject']['Object']['Name2']=eachdict[key]['DKObject']['Object']['Name']
del eachdict[key]['DKObject']['Object']['Name']
lists.append([key, eachdict[key]['DK1'], eachdict[key]['KM'], eachdict[key]['Country']['Name1'],
eachdict[key]['Country']['City']['Name'], eachdict[key]['DKObject']['Object']['Name2'], eachdict[key]['Order']['Client']])
pd.DataFrame(lists, columns=[<columnNamesHere>])
Output:
{100: {'ID': 100,
'DKSTGFase': None,
'KM': None,
'Country': {'City': {'Name': 'Berlin'}, 'Name1': 'GE'},
'Type': {'Name': '219'},
'DKObject': {'Name': '8555', 'Object': {'Name2': 'Car'}},
'Order': {'OrderId': 101,
'CreatedOn': '2018-07-06T16:54:36.783+02:00',
'ModifiedOn': '2018-07-06T16:54:36.783+02:00',
'Name': 'Audi',
'Client': {'1'}},
'DKComponent': {'Name': 'John'},
'DK1': False}}

python - dictionary - update text values of keys - setting an priority (max principle)

I have the following strings as values for a dictionary key:
["low", "middle", "high", "very high"]
These are the options for the dicionary item key 'priority', a sample dict element is:
{'name': 'service', 'priority': value}
My task is to collect a list of dictionaries with the keys, all differ in the key value 'priority'.
my_list = [{'name': 'service', 'priority': 'low'}, {'name': 'service', 'priority': 'high'}]
In the end a final dictionary item should exist, that has the highest priority value. It should work like the maximum principle. In this case {'name': 'service', 'priority': 'high'} would be the result.
The problem is that the value is a string, not an integer.
Thanks for all ideas to get it work.

Here is the approach with itertools module usage:
# Step 0: prepare data
score = ["low", "middle", "high", "very high"]
my_list = [{'name': 'service', 'priority': 'low', 'label1':'text'}, {'name': 'service', 'priority': 'middle', 'label2':'text'}, {'name': 'service_b', 'priority': 'middle'}, {'name': 'service_b', 'priority': 'very high'}]
my_list # to just show source data in list
Out[1]:
[{'name': 'service', 'priority': 'low', 'label1': 'text'},
{'name': 'service', 'priority': 'middle', 'label2': 'text'},
{'name': 'service_b', 'priority': 'middle'},
{'name': 'service_b', 'priority': 'very high'}]
# Step 0.5: convert bytes-string (if it is) to string
# my_list = [{k:(lambda x: (x.decode() if type(x) == bytes else x))(v) for k,v in i.items()} for i in my_list ]
# Step 1: reorganize "score"-list on most useful way - to dict
score_dic = {i[0]:i[1] for i in list(zip(score, range(len(score))))}
score_dic
Out[2]:
{'low': 0, 'middle': 1, 'high': 2, 'very high': 3}
# Step 2: get result
import itertools
[max(list(g), key = lambda b: score_dic[b['priority']]) for k,g in itertools.groupby(my_list, lambda x:x['name'])]
Out[3]:
[{'name': 'service', 'priority': 'middle', 'label2': 'text'},
{'name': 'service_b', 'priority': 'very high'}]

Is this what you want?
priorities = ["low", "middle", "high", "very high"]
items = [{'name': 'service', 'priority': 'high'}, {'name': 'service2', 'priority': 'high'}, {'name': 'service', 'priority': 'very high'}, {'name': 'service2', 'priority': 'very high'}]
max_priority = max(items, key=lambda item: priorities.index(item['priority']))['priority']
max_items = [item for item in items if item['priority'] == max_priority]
print(max_items)
Output:
[{'name': 'service', 'priority': 'very high'}, {'name': 'service2', 'priority': 'very high'}]

How to extract common elements from list of lists of dictionaries

I am trying to build dictionaries (one contains all common elements and the other one contains the different elements) out of a list of dictionaries.
Now I've managed to get it working for a list of 2 dictionaries by converting to a set of tuples and then getting the unique keys as well as the differences with the intersection and difference methods but I don't know how to go about a list of varying length (sometimes I'll have 3 or 4 dictionaries in my list).
I'm sure I need to use map or reduce/lambda function but I can't figure it out.
This is my input:
all_maps = [
[{'key': 'target', 'value': 'true'},
{'key': 'region_name', 'value': 'europe'},
{'field': 'AccessToken', 'key': 'token','path': 'test/path'}],
[{'key': 'target', 'value': 'true'},
{'key': 'region_name', 'value': 'usa'},
{'field': 'AccessToken', 'key': 'token', 'path': 'test/path'}],
[{'key': 'target', 'value': 'true'},
{'key': 'region_name', 'value': 'japan'},
{'field': 'AccessToken', 'key': 'token', 'path': 'test/path'}]
]
What I want is to get 4 dictionaries as such:
intersection = {'key': 'target', 'value': 'true'},
{'field': 'AccessToken', 'key': 'token', 'path': 'test/path'}
diff1 = {'key': 'region_name', 'value': 'europe'}
diff2 = {'key': 'region_name', 'value': 'usa'}
diff3 = {'key': 'region_name', 'value': 'japan'}

A simple answer would be to flatten the all_maps list and separate each items based on its list.count() value:
def flatten(map_groups):
items = []
for group in map_groups:
items.extend(group)
return items
def intersection(map_groups):
unique = []
items = flatten(map_groups)
for item in items:
if item not in unique and items.count(item) > 1:
unique.append(item)
return unique
def difference(map_groups):
unique = []
items = flatten(map_groups)
for item in items:
if item not in unique and items.count(item) == 1:
unique.append(item)
return unique
Here's the output using these functions:
>>> intersection(all_maps)
[{'key': 'target', 'value': 'true'},
{'field': 'AccessToken', 'key': 'token', 'path': 'test/path'}]
>>> difference(all_maps)
[{'key': 'region_name', 'value': 'europe'},
{'key': 'region_name', 'value': 'usa'},
{'key': 'region_name', 'value': 'japan'}]
For a more advanced implementation, you can look into set().

How do I remove text from a list containing dictionary objects?

I have a list
x = [{'key': 'ACT', 'value': 'Active'},
{'key': 'EXP', 'value': 'Expired'},
{'key': 'RCAL', 'value': 'Recall'}]
I want to convert it to
[{'ACT': 'Active', 'EXP': 'Expired', 'RCAL': 'Recall'}]
How do I get rid of 'key': and 'value' through out list x??

Try this:
>>> y = {a['key']: a['value'] for a in x}
>>> y
{'ACT': 'Active', 'EXP': 'Expired', 'RCAL': 'Recall'}
Read: Dict Comprehensions in python.

I would do list comprehension where you are creating a dictionary from each element in your list x:
x = [{'key': 'ACT', 'value': 'Active'},
{'key': 'EXP', 'value': 'Expired'},
{'key': 'RCAL', 'value': 'Recall'}]
x = [{e['key']:e['value']} for e in x]
print(x)
This results in:
[{'ACT': 'Active'}, {'EXP': 'Expired'}, {'RCAL': 'Recall'}]