Develop Reference

Iterate through list of dictionaries in Python and return value

I have a list of dictionaries in this format:
user_input =[{
'user':'guest',
'record':
'{"message":"User Request","context":{"session_id":"YWOZe"},"level":100}'
},
{
'user':'guest',
'record':
'{"message":"User Inquiry","context":{"session_id":"YWOZf"},"level":100}'
},
]
What I want is, I want to use eval in eval(user_input[0]['record']) and eval(user_input[1]['record']) by iterating over two dictionaries, so far I could do for one dictionary as
def record_to_objects(data:dict):
new_dict = {}
for key, value in data.items():
if data[key] == data['record']:
new_dict['record'] = eval(data['record'])
r = data
del r['record']
return {**r, **new_dict}
But for two dictionaries in a list, I got only one result(as the last item) using "double for loop", maybe I am not returning in a proper way.
Expected output:
output = [
{'user': 'guest',
'record': {'message': 'User Request',
'context': {'session_id': 'YWOZe'},
'level': 100}},
{'user': 'guest',
'record': {'message': 'User Inquiry',
'context': {'session_id': 'YWOZf'},
'level': 100}}
]
Can somebody help me? I want a list of dictionaries and both should have been processed by the eval method.

That looks like JSON. You should not use eval() for this. Instead, use json.loads() with a for loop:
import json
for item in user_input:
item["record"] = json.loads(item["record"])
This outputs:
[
{
'user': 'guest',
'record': {'message': 'User Request', 'context': {'session_id': 'YWOZe'}, 'level': 100}
},
{
'user': 'guest',
'record': {'message': 'User Inquiry', 'context': {'session_id': 'YWOZf'}, 'level': 100}
}
]

Remove item in JSON if key has value

I have tried everything I can possible come up with, but the value wont go away.
I have a JSON user and if user['permissions'] have key permission = "DELETE PAGE" remove that index of del user['permissions'][1] (in this example)
I want to have a list of possible values as "DELETE PAGE" and so on. If value in key, then delete that index.
Then return the users json without those items found.
I have tried del user['permission][x] and .pop() and so on but it is still there.
{
'id': 123,
'name': 'My name',
'description': 'This is who I am',
'permissions': [{
'id': 18814,
'holder': {
'type': 'role',
'parameter': '321',
'projectRole': {
'name': 'Admin',
'id': 1,
}
},
'permission': 'VIEW PAGE'
}, {
'id': 18815,
'holder': {
'type': 'role',
'parameter': '123',
'projectRole': {
'name': 'Moderator',
'id': 2
}
},
'permission': 'DELETE PAGE'
}]
}

Here's the code:
perm = a['permissions']
for p in perm:
if p['permission'] == 'DELETE PAGE':
perm.remove(p)
print(a)
Output:
{'id': 123, 'name': 'My name', 'description': 'This is who I am', 'permissions': [{'id': 18814, 'holder': {'type': 'role', 'parameter': '321', 'projectRole': {'name': 'Admin', 'id': 1}}, 'permission': 'VIEW PAGE'}]}

Mandrill API to send email with recipients(cc), but It also send mail to recipients(cc) as 'TO' using Django==1.11.10

Code:
def send_mandrill_subscribe_wac(link,user_email, from_user, from_email='examplecc#abc.com'):
mandrill_client = mandrill.Mandrill(MANDRILL_API_KEY)
#template_name = "Invited-user-email-for-someone-without-an-account"
template_name = "invited-user-email-for-someone-without-an-account"
template_content = [{'content': 'example content', 'name': 'example name'}]
message = {
'from_email': 'examplecc#abc.com',
'to': [
{
'email': user_email,
'type': 'to'
},
{
'email': from_email,
'type': 'cc'
},
{
'email': 'examplecc2#abc.com',
'type': 'cc'
}
],
'global_merge_vars': [
{'content': user_email},
{'name': "redirect_link", 'content': link},
{'name': 'from_user', 'content': from_user}
],
}
return mandrill_client.messages.send_template(template_name=template_name, template_content=template_content, message=message)
In official Documentation they mentioned preserve_recipients set to True to send recipients, But don't know where to add this parameter.
Need help please

preserve_recipients is a property of message from what I see at https://mailchimp.com/developer/transactional/api/messages/send-new-message/
message = {
'from_email': 'examplecc#abc.com',
'preserve_recipients': true,

How to print this nested dictionary containing several list and dictionaries?

[[{'text': '\n ', 'category': 'cooking', 'title': {'text': 'Everyday
Italian', 'lang': 'en'}, 'author': {'text': 'Giada De Laurentiis'}, 'year':
{'text': '2005'}, 'price': {'text': '30.00'}},
{'text': '\n ', 'category': 'children', 'title': {'text': 'Harry Potter',
'lang': 'en'}, 'author': {'text': 'J K. Rowling'}, 'year': {'text':
'2005'}, 'price': {'text': '29.99'}}, {'text': '\n ', 'category':
'web', 'title': {'text': 'XQuery Kick Start', 'lang': 'en'}, 'author':
[{'text': 'James McGovern'}, {'text': 'Per Bothner'}, {'text': 'Kurt
Cagle'}, {'text': 'James Linn'}, {'text': 'Vaidyanathan Nagarajan'}],
'year': {'text': '2003'}, 'price': {'text': '49.99'}}, {'text': '\n ',
'category': 'web', 'cover': 'paperback', 'title': {'text': 'Learning XML',
'lang': 'en'}, 'author': {'text': 'Erik T. Ray'}, 'year': {'text': '2003'},
'price': {'text': '39.95'}}]]
output format:
category : cooking,
title : ['Everyday Italian', 'lang': 'en'],
author : Giada De Laurentiis,
year : '2005',
price : '30.00'
category : children,
title : ['Harry Potter', 'lang': 'en'],
author : 'J K. Rowling',
year : '2005',
price : '29.99'
category : web,
title : [ 'XQuery Kick Start''lang': 'en'],
author :[ 'James McGovern' , 'Per Bothner','Kurt Cagle','James Linn', 'Vaidyanathan Nagarajan'],
year : '2003',
price : '49.99'
category : web,
cover : paperback,
title : [ 'Learning XML','lang': 'en'],
author : 'Erik T. Ray',
year : '2003',
price : '39.95'

A simple loop like the following should get the output you require.
for entry in data[0]:
for k, v in entry.items():
print(k, ':', v)

def printBook(d):
del d['text']
for i in d:
if type(d[i]) == dict:
if len(d[i])==1:
d[i] = list(d[i].values())[0]
else:
d[i] = [('' if j=='text' else (j+':')) + d[i][j] for j in d[i]]
s = '\n'
for i,j in d.items():
s += f' {i} : {j} ,\n'
print(s)
try these it prints individual dictionary into your described format

Take a look at the pprint module which provides a nice way of printing data structures without the need for writing your own formatter.

Thanks for the coding excercise. Man, that output format was specific! :D Tricky, as the strings are not quoted if being standalone, quoted if coming from text attribute. Also tricky, that stuff must be thrown into [] if not just text. Also it is a little bit underspecified, because hey, what if there is no text at all, yet other keys?
Output format disclaimer:
I think there is a missing , after 'XQuery Kick Start'
I think Giada De Laurentiis should have been quoted as it is coming from 'text'
import copy
def transform_value(stuff):
if type(stuff) is str:
return stuff
if type(stuff) is dict:
elements = []
text = stuff.pop("text", "")
if text:
elements.append(f"'{text}'") # we quote only if there was really a text entry
if not stuff: # dict just got empty
return elements[0] # no more attributes, so no [] needed around text
elements.extend(f"'{key}': '{transform_value(value)}'" for key, value in stuff.items())
if type(stuff) is list:
elements = [transform_value(e) for e in stuff]
# this will obviously raise an exception if stuff is not one of str, dict or list
return f"[{', '.join(elements)}]"
def transform_pub(d: dict):
d = copy.deepcopy(d) # we are gonna delete keys, so we don't mess with the outer data
tail = d.pop("text", "")
result = ",\n".join(f'{key} : {transform_value(value)}' for key, value in d.items())
return result + tail
if __name__ == "__main__":
for sublist in data:
for pub in sublist:
print(transform_pub(pub))
I first wanted to use somehow the same mechanism for the publications themselves via some recursion. But then the code become too complicated as the text field is appended for publications, while it is coming first for attributes.
Once I let go of the fully structured solution, I started out with a test for the publication printer:
import pytest
from printing import transform_value
#pytest.mark.parametrize('input,output', [
("cooking", "cooking"),
({"text": "J K. Rowling"}, "'J K. Rowling'"),
({"lang": "en", "text": "Everyday Italian"},
"['Everyday Italian', 'lang': 'en']"),
([{"text": "James McGovern"},
{"text": "Per Bothner"},
{"text": "Kurt Cagle"},
{"text": "James Linn"},
{"text": "Vaidyanathan Nagarajan"}],
"['James McGovern', 'Per Bothner', 'Kurt Cagle', 'James Linn', 'Vaidyanathan Nagarajan']"
),
([{"text": "Joe"}], "['Joe']"),
({"a": "1"}, "['a': '1']"),
])
def test_transform_value(input, output):
assert transform_value(input) == output

concat string values in multiple dicts

suppose I have a list of dicts (where each dict has the same keys) like this:
list_of_dicts = [
{'Id': 4726, 'Body': 'Hello from John', 'Title': None, 'Comments': 'Dallas. '},
{'Id': 4726, 'Body': 'Hello from Mary', 'Title': None, 'Comments': "Austin"},
{'Id': 4726, 'Body': 'Hello from Dylan', 'Title': None, 'Comments': "Boston"},
]
I need to concat only the Body, Title and Comments part and return a single dict, like this:
{'Id': 4726, 'Body': 'Hello from John Hello from Mary Hello from Dylan', 'Title': None, 'Comments': 'Dallas. Austin Boston'}
Please note, Title is None. So, we have to be careful there. This is what I have done so far...but, failing somewhere...I cannot see where...
keys = set().union(*list_of_dicts)
print(keys)
k_value = list_of_dicts[0]['Id']
d_dict = {k: " ".join(str(dic.get(k, '')) for dic in list_of_dicts) for k in keys if k != 'Id'}
merged_dict = {'Id': k_value}
merged_dict.update(d_dict)
But, the above returns this ...which I do not like:
Final Merged Dict: {'Id': 4726, 'Body': 'Hello from John Hello from Mary Hello from Dylan', 'Title': 'None None None', 'Comments': 'Dallas. Austin Boston'}

First, I'd remove Id from keys to avoid having to skip it in the dictionary comprehension, and use a simple assignment rather than .update() at the end.
In the argument to join, filter out when dic[k] is None. And if the join results in an empty string (because all the values are None), convert that to None in the final result.
keys = set().union(*list_of_dicts)
keys.remove('Id')
print(keys)
k_value = list_of_dicts[0]['Id']
d_dict = {k: (" ".join(str(dic[k]) for dic in list_of_dicts if k in dic and dic[k] is not None) or None) for k in keys}
d_dict['Id'] = k_value
print(d_dict)
DEMO

As you parse your list of dictionaries, you can store the intermediate results in defaultdict objects to hold a list of the string values. Once all the dictionaries have been parsed, you can then join together the strings.
from collections import defaultdict
dd_body = defaultdict(list)
dd_comments = defaultdict(list)
dd_titles = defaultdict(list)
for row in list_of_dicts:
dd_body[row['Id']].append(row['Body'])
dd_comments[row['Id']].append(row['Comments'])
dd_titles[row['Id']].append(row['Title'] or '') # Effectively removes `None`.
result = []
for id_ in dd_body: # All three dictionaries have the same keys.
body = ' '.join(dd_body[id_]).strip()
comments = ' '.join(dd_comments[id_]).strip()
titles = ' '.join(dd_titles[id_]).strip() or None
result.append({'Id': id_, 'Body': body, 'Title': titles, 'Comments': comments})
>>> result
[{'Id': 4726,
'Body': 'Hello from John Hello from Mary Hello from Dylan',
'Title': None,
'Comments': 'Dallas. Austin Boston'}]

Less Pythonic then other answers but I like to think that it is easy to understand.
body, title, comments = "", "", ""
list_of_dicts=[
{'Id': 4726, 'Body': 'Hello from John', 'Title': None, 'Comments': 'Dallas. '},
{'Id': 4726, 'Body': 'Hello from Mary', 'Title': None, 'Comments': "Austin"},
{'Id': 4726, 'Body': 'Hello from Dylan', 'Title': None, 'Comments': "Boston"},
]
id = list_of_dicts[0]['Id']
for dict in list_of_dicts:
if dict['Body'] is not None:
body=body + dict['Body']
if dict['Title'] is not None:
title=title + dict['Title']
if dict ['Comments'] is not None:
comments=comments + dict['Comments']
if title == "":
title = None
if body == "":
body = None
if comments == "":
comments = None
record = {'Id': id, 'Body': body, 'Title': title, 'Comments': comments}
If only Title field has a option of being None then it can be shortened by removing the checks on the other fields.
body, title, comments = "", "", ""
list_of_dicts=[
{'Id': 4726, 'Body': 'Hello from John', 'Title': None, 'Comments': 'Dallas. '},
{'Id': 4726, 'Body': 'Hello from Mary', 'Title': None, 'Comments': "Austin"},
{'Id': 4726, 'Body': 'Hello from Dylan', 'Title': None, 'Comments': "Boston"}]
id = list_of_dicts[0]['Id']
for dict in list_of_dicts:
body=body + dict['Body']
comments=comments + dict['Comments']
if dict['Title'] is not None:
title=title + dict['Title']
if title == "":
title = None
record = {'Id': id, 'Body': body, 'Title': title, 'Comments': comments}

For this type of data manipulations pandas is your friend.
import pandas as pd
# Your list of dictionaries.
list_of_dicts = [
{'Id': 4726, 'Body': 'Hello from John', 'Title': None, 'Comments': 'Dallas. '},
{'Id': 4726, 'Body': 'Hello from Mary', 'Title': None, 'Comments': "Austin"},
{'Id': 4726, 'Body': 'Hello from Dylan', 'Title': None, 'Comments': "Boston"},
]
# Can be read into a pandas dataframe
df = pd.DataFrame(list_of_dicts)
# Do a database style groupby() and apply the function that you want to each group
group_transformed_df = df.groupby('Id').agg(lambda x: ' '.join(x)).reset_index() # I do reset_index to get a normal DataFrame back.
# DataFrame() -> dict
output_dict = group_transformed_df.to_dict('records')
There are many types of dicts you can get from a DataFrame. You want the records option.