Develop Reference

creating a dictionary by partitioning a dictionary with new keys in python

I have the a dictionary like this:
{"Topic":"text","title":"texttitle","abstract":"textabs","year":"textyear","authors":"authors"}
I want to create another list as follows:
[{"label":{"Topic":"text","title":"texttitle","abstract":"textabs","year":"textyear","authors":"authors"},"value":
{"Topic":"text","title":"texttitle","abstract":"textabs","year":"textyear","authors":"authors"}}]
I have tried some methods with .items() but none of them gives the desired result.

Is that what you want?
dict_ = {"Topic":"text","title":"texttitle","abstract":"textabs","year":"textyear","authors":"authors"}
output = [{"label": dict_ , "value": dict_ }]
print(output)
[{"label":{"Topic":"text","title":"texttitle","abstract":"textabs","year":"textyear","authors":"authors"},"value":
{"Topic":"text","title":"texttitle","abstract":"textabs","year":"textyear","authors":"authors"}}] == [{"label": dict_ , "value": dict_ }]
Gives True

Following my comment, below is the code I would go through assuming key and output:
# Could be the keys would get from somewhere
vals = ["1","2","3","4"]
# Probably same coming from external sources
example_op =
{"Topic":"text","title":"texttitle","abstract":"textabs","year":"textyear","authors":"authors"}
#Global list
item_list = []
temp_dict = {}
for key in vals:
temp_dict[key] = example_op
item_list.append(temp_dict)
Final output of the list would be as:
Out[9]:
[{'1': {'Topic': 'text',
'title': 'texttitle',
'abstract': 'textabs',
'year': 'textyear',
'authors': 'authors'},
'2': {'Topic': 'text',
'title': 'texttitle',
'abstract': 'textabs',
'year': 'textyear',
'authors': 'authors'},
'3': {'Topic': 'text',
'title': 'texttitle',
'abstract': 'textabs',
'year': 'textyear',
'authors': 'authors'},
'4': {'Topic': 'text',
'title': 'texttitle',
'abstract': 'textabs',
'year': 'textyear',
'authors': 'authors'}}]

Storing List of Dict in a DynamoDB Table

I want to store a list of Tags of an Elasticsearch domain in a DynamoDB and i'm facing some errors.
I'm getting the list of tags using list_tags() function :
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/es.html#ElasticsearchService.Client.list_tags
response = client.list_tags(
ARN='string'
)
It returns that :
{
'TagList': [
{
'Key': 'string',
'Value': 'string'
},
]
}
Here's what they say in the doc :
Response Structure
(dict) --
The result of a ListTags operation. Contains tags for all requested Elasticsearch domains.
TagList (list) --
List of Tag for the requested Elasticsearch domain.
(dict) --
Specifies a key value pair for a resource tag.
Now i tried to insert the list in DynamoDB using various ways but i'm always getting errors :
':TagList': {
'M': response_list_tags['TagList']
},
Invalid type for parameter ExpressionAttributeValues.:TagList.M, value: [{'Key': 'Automation', 'Value': 'None'}, {'Key': 'Owner', 'Value': 'owner'}, {'Key': 'BU', 'Value': 'DS'}, {'Key': 'Support', 'Value': 'teamA'}, {'Key': 'Note', 'Value': ''}, {'Key': 'Environment', 'Value': 'dev'}, {'Key': 'Creator', 'Value': ''}, {'Key': 'SubProject', 'Value': ''}, {'Key': 'DateTimeTag', 'Value': 'nodef'}, {'Key': 'ApplicationCode', 'Value': ''}, {'Key': 'Criticity', 'Value': '3'}, {'Key': 'Name', 'Value': 'dev'}], type: , valid types: : ParamValidationError
Tried with L instead of M and got this :
Unknown parameter in ExpressionAttributeValues.:TagList.L[11]: "Value", must be one of: S, N, B, SS, NS, BS, M, L, NULL, BOOL: ParamValidationError

The specific error you are getting is because you are using the native DynamoDB document item JSON format which requires that any attribute value (including key-values in a map, nested in a list) to be fully qualified with a type as a key-value.
There are two ways you can do that and from your question I'm not sure if you wanted to store those key-value tag objects as a list, or you wanted to store that as an actual map in Dynamo.
Either way, I recommend you JSON encode you list and just store it in DynamoDB as a string value. There's no really good reason why you would want to go through the trouble of storing that as a map or list.
However, if you really wanted to you could do the conversion to the DynamoDB native JSON and store as a map. You will end up with something like this:
':TagList': {
'M': {
'Automation': { 'S': 'None' },
'Owner': {'S': 'owner'},
'BU': {'S': 'DS'},
'Support': {'S': 'teamA'}
...
}
}
Another possibility would be using a list of maps:
':TagList': {
'L': [
'M': {'Key': {'S': 'Automation'}, 'Value': { 'S': 'None' }},
'M': {'Key': {'S': 'Owner'}, 'Value' : {'S': 'owner'}},
'M': {'Key': {'S': 'BU'}, 'Value': {'S': 'DS'}},
'M': {'Key': {'S': 'Support'}, 'Value': {'S': 'teamA'}}
...
]
}
But in my experience I have never gotten any real value out of storing data like this in Dynamo. Instead, storing those tags as a JSON string is both easier and less error prone. You end up with this:
':TagList': {
'S': '{\'Key\': \'Automation\', \'Value\': \'None\'}, {\'Key\': \'Owner\', \'Value\': \'owner\'}, {\'Key\': \'BU\', \'Value\': \'DS\'}, {\'Key\': \'Support\', \'Value\': \'teamA\'}, ... }'
}
And all you have to do is writhe the equivalent of:
':TagList': {
'S': json.dumps(response_list_tags['TagList'])
}

Thank you Mike, i eneded up with a similar solution. I stored the Tag List as String like that :
':TagList': {
'S': str(response_list_tags['TagList'])
}
Then to convert the string to a list for a later use i did this :
import ast
...
TagList= ast.literal_eval(db_result['Item']['TagList']['S'])

Convert Dict to dict with children and "unlimited" depth

I'm trying to convert dict for current view to format that I can use it in AngularJS object:
data = "{'root': {'host': {'hostname1': {'10.0.0.1': {}}, 'hostname2': {'10.0.0.2': {}}}, 'monitor': {'bandwidth': {'hostname1': {'10.0.0.1': {'hostname1': {'10.0.0.1': {'10': {}}}, 'hostname2': {'10.0.0.2': {'10': {}}}}}, 'hostname2': {'10.0.0.2': {'hostname1': {'10.0.0.1': {'10': {}}}, 'hostname2': {'10.0.0.2': {'10': {}}}}}}}}}"
to format with names and children values, like:
[{
name: "Node 1",
children: [{
name: "Node 1.1",
children:[{name:"Node 1.1.1"},{name: "Node 1.1.2"}]
}]},{
name: "Node 2",
children: [{name: "Node 2.1"},{name: "Node 2.2"}]
}]
I tried few different approaches, but always received partial results. For example I tried to use recursion, it went till the depth value and then ignored all other tree.
def modifydict2(data):
for key, value in data.items():
return [{'name': key, 'children':modifydict2(value)}]
As a result I received only part of my dict back. I understood that my loop never worked cause I returned value before next iteration, but not sure how to fix that:
[{'name': 'root', 'children': [{'name': 'host', 'children': [{'name': 'ctest1.prod01.weave.local', 'children': [{'name': '10.32.62.1', 'children': None}]}]}]}]

You need to append those individual values that you're currently returning to a list and then return that list. Or, using a list comprehension:
def modify_dict(d):
return [{'name': key, 'children': modify_dict(value)}
for key, value in d.items()]

Combine Dictionarys in a list based on a value

I can't get my head around this problem:
I have a List:
[{'name' : 'Bob', 'Salary2014' : 2000}, {'name': 'Alice', 'Salary2014' : 1000}, {'name':'Bob', 'Salary2013' : 1500}]
I want to join the dictionarys on base of the Name (which is unique)
[{'name' : 'Bob', 'Salary2014' : 2000, 'Salary2013' : 1500}, {'name': 'Alice', 'Salary2014' : 1000}]
I know the solution has to be simple and may involve the .update method, but I just don't get it.

Use a new dictionary to track your dictionaries based on name:
combined = {}
for d in yourlist:
combined.setdefault(d['name'], {}).update(d)
output = combined.values()
The combined.setdefault() method here sets {} as a default value if d['name'] is not present, then updates the dictionary with the current iteration.
If you are using Python 3, use list(combined.values()).
Demo:
>>> yourlist = [{'name' : 'Bob', 'Salary2014' : 2000}, {'name': 'Alice', 'Salary2014' : 1000}, {'name':'Bob', 'Salary2013' : 1500}]
>>> combined = {}
>>> for d in yourlist:
... combined.setdefault(d['name'], {}).update(d)
...
>>> combined.values()
[{'Salary2013': 1500, 'name': 'Bob', 'Salary2014': 2000}, {'name': 'Alice', 'Salary2014': 1000}]

The other pretty similar way is to use defaultdict:
from collections import defaultdict
inp = [{'name': 'Bob', 'Salary2014': 2000},
{'name': 'Alice', 'Salary2014': 1000},
{'name': 'Bob', 'Salary2013': 1500}]
out = defaultdict(dict)
for rec in inp:
out[rec['name']].update(rec)
print out.values()

Appending/Merging in Python

I have the following structure:
[
{
u'123456': {'name': "Bill"},
u'234567': {'name': "Dave"},
u'345678': {'name': "Tom"}
},
]
During a for loop new items are added to the list using the extend function. Unfortunately this results in the following structure:
[
{
u'123456': {'name': "Bill"},
u'234567': {'name': "Dave"},
u'345678': {'name': "Tom"}
},
{
u'555555': {'name': "Steve"},
u'666666': {'name': "Michael"},
u'777777': {'name': "George"}
}
]
The intended result is actually a flat structure such in the following:
[
{
u'123456': {'name': "Bill"},
u'234567': {'name': "Dave"},
u'345678': {'name': "Tom"},
u'555555': {'name': "Steve"},
u'666666': {'name': "Michael"},
u'777777': {'name': "George"}
}
]
Is it possible to append to the list so that the structure gets built in a flat way.
or
Is it possible to flatten after the loop has finished?

If your list is named l you could use l[0].update(new_dict).
Example:
l = [{u'123456': {'name': "Bill"}}]
l[0].update({u'234567': {'name': "Dave"}})
print(l)
Nice formatted output is:
[
{
u'123456': {'name': 'Bill'},
u'234567': {'name': 'Dave'}
}
]

Where you currently have something like this:
mylist.extend(newdict)
You should use this:
mylist[0].update(newdict)

You can add the items of both dictionaries together:
>>> mylist = [
{
u'123456': {'name': "Bill"},
u'234567': {'name': "Dave"},
u'345678': {'name': "Tom"}
},
]
>>> mydict = {
u'555555': {'name': "Steve"},
u'666666': {'name': "Michael"},
u'777777': {'name': "George"}
}
>>> [dict(mylist[0].items() + mydict.items())]
[{u'123456': {'name': 'Bill'}, u'555555': {'name': 'Steve'}, u'777777': {'name': 'George'}, u'666666': {'name': 'Michael'}, u'345678': {'name': 'Tom'}, u'234567': {'name': 'Dave'}}]
Although it's more clean to just do .update():
>>> mylist[0].update(mydict)

You can use .update(), however this will overwrite values if you'll have duplicated keys.
def flatten(results):
newresult = {}
for subdict : results:
newresult.update(subdict)
return [newresult]