Develop Reference

Add to a list of maps or update value of previously added map aws-dynamodb

I have a attribute of type 'List of maps' in my dynamodb. e.g.
List = [{key1: value1},{key2: value2},{key3:value3},...]
I want to update the list (from my python script) with current_key to add a new key or replace the value if the current_key is already present.

To illustrate the code below better, I've chosen to change your {key1:value1} to strings. Functionally, when you are calling the add_to_datalist function, there should be no difference.
def add_to_datalist(datalist, key, value):
check = 0
#check is needed to ensure the if condition after the for loop can trigger
for datadict in datalist:
if key in datadict.keys():
datadict[key] = value
check = 1 #turn off check if a key matching is found
if check == 0:
datalist.append({key:value})
return datalist
datalist = [{'key1':'value1'},{'key2':'value2'},{'key3':'value3'}]
key1 = 'key333'
value1 = 'value999'
datalist = add_to_datalist(datalist, key1, value1)
key2 = 'key2'
value2 = 'valueNEW2'
datalist = add_to_datalist(datalist, key2, value2)
print (datalist)
#[{'key1': 'value1'}, {'key2': 'valueNEW2'}, {'key3': 'value3'}, {'key333': 'value999'}]

Edit values within dictionary

I have a dictionary
TestDict = {'Group1': 'Crew 1', 'Group2': 'Crew 2', 'Group3': 'Crew 3'}
And I'd like to edit the value in the dictionary to have this desired output
{'Group1': 'Group1_Crew_1', 'Group2': 'Group2_Crew_2', 'Group3': 'Group3_Crew_3'}
So in the above, the key is now a part of the value and the space have been replaced with _
So far I've only tried to tackle appending the key to the value with
for key, value in TestDict.items():
newDict = {key: key + "_" + value for key in TestDict}
Problematically, this outputs in the proper dictionary, however, all of the values are Group3_Crew 3. I'm not sure how to navigate this issue.

You have two options:
Create a new dictionary and update it content:
newDict = {}
for key, value in TestDict.items():
newDict[key] = key + "_" + value.replace(" ", "_")
or use a dict comprehension:
newdDict = {key: key + "_" + value.replace(" ", "_") for key in TestDict}
You were like mixing both concepts.

You have two problems.
Firstly, you're iterating over TestDict twice, where you only need to do so once. You're recreating an entire new version of newDict each time through the loop and, after you're done, you're only seeing the last one created.
Use either
newDict = {}
for key, value in TestDict.items():
newDict[key] = key + "_" + value
or
newDict = {key: key + "_" + value for key, value in TestDict.items()}
but don't combine both.
(The results you're seeing are because you're getting a new key in each inner iteration, but only a single value from the outer iteration.)
Also, you're not replacing spaces with underscores, which you would do by replacing value with value.replace(' ', '_') in either version.

You can use:
{k: k+'_'+v.replace(' ', '_') for k, v in TestDict.items()}

Python: List as a key in dictionary -

I am trying to read multiple files with very similar data. Each line of this data has a accessor_key and a value assosciated with it. I am trying to create a dictionary with the accessor_key as the dictionary key and as the dictionary value - a list of all the values read so far.
My code looks like this:
with open(ind_file, "r") as r:
for line in r:
nline = line.strip()
spl = nline.split(",")
if agg_d.has_key(spl[0]):
key = spl[0]
val = spl[1]
dummy = agg_d[key]
dummy.append(val)
agg_d[key] = dummy
print key, agg_d[key]
else:
print "Something is wrong"
print agg_d
print spl[0]
print spl[1]
As you can see I want the value to get bigger every time, (the list increases in size by 1 every iteration) and store it back to the dictionary.
However when I run this program, all keys in the dictionary take on the value of the list.
So for example in the beginning of the program the dictionary is :
agg_d = {'some_key': [], 'another_key': []}
After running it once it becomes:
agg_d = {'some_key': ['1'], 'another_key': ['1']}
When it should be just:
agg_d = {'some_key': ['1'], 'another_key': []}
EDIT: I found the work around I was looking for. I simply did:
with open(ind_file, "r") as r:
for line in r:
nline = line.strip()
spl = nline.split(",")
if agg_d.has_key(spl[0]):
key = spl[0]
val = spl[1]
dummy = agg_d[key]
ad = dummy[:]
ad.append(val)
agg_d[key] = ad
print key, agg_d[key]
else:
print "Something is wrong"
print agg_d
print spl[0]
print spl[1]
But I would still like to know why this is happening at all. Is 'dummy' referenced to all the values of the dictionary? I am running this with Python 2.7.

Is 'dummy' referenced to all the values of the dictionary? I am running this with Python 2.7.
Yes. You've added a reference to the list, and there can be multiple references to that same list as you have observed. To illustrate this simply, try this:
dummy = [1,2,3] # creates a list object and assigns reference to the name 'dummy'
d = dict()
d['some key'] = dummy # creates the key 'some key' in the dictionary and assigns its value as the reference to the name 'dummy'
dummy.append(4) # mutates the list referred to by name 'dummy'
# at this point, all references to that object have mutated similarly
print d['some key']
You will observe the following output:
>>> [1,2,3,4]
Your workaround is OK, but you could improve:
with open(ind_file, "r") as r:
for line in r:
spl = line.strip().split(",")
key, val = spl[0], spl[1]
if key in agg_d:
agg_d[key] = agg_d[key][:].append(val)
print key, agg_d[key]
else:
print "Something is wrong"
print agg_d
print spl[0]
print spl[1]
agg_d[key] = agg_d[key][:].append(val)
This does not mutate your dummy list in place, and reassigns the value to the dictionary. Also avoids some unnecessary variables like nline and ad and dummy.

It looks like agg_d is already initialised with your expected keys. You don't show how this is done, but I'm guessing that all of the initial values are in fact the same list - to which you append values in the code above.
If you initialise agg_d with a new list per key, then the problem should go away. You may be able to do this with a dictionary comprehension:
>>> keys = ["a", "b", "c"]
>>> agg_d = {k:[] for k in keys}
>>> agg_d["a"].append(1)
>>> agg_d
{'a': [1], 'c': [], 'b': []}
Alternatively, depending on your needs, you could initialise each entry on demand as you encounter each key when reading the file.
Your workaround works because it replaces the original list with a new list and removes the shared reference.

The issue is that by default Python just adds a reference to the list as the dict value, not the list itself. So dict values are actually the bunch of pointers to the same object. You need to explicitly copy the list using either dummy[:] as you suggest in comment, or copy.deepcopy() to be more explicit.

Adding multiple dictionaries to a key in python dictionary

I am trying to add multiple dictionaries to a key.
e.g.
value = { column1 : {entry1 : val1}
{entry2 : val2}
column2 : {entry3 : val3}
{entry4 : val4}
}
What exactly I am trying to do with this code is:
There is a file.txt which has columns and valid entries for that header. I am trying to make a dictionary with columns as key and for each column another dictionary for each valid entry.
So I am parsing the text file line by line to find the pattern for column and entries and storing it in a variable, check if the column(which is a key) already exists in the dictionary, if exists then add another dictionary to the column, if not create a new entry. I Hope this makes sense.
Sample contents of file.txt
blah blah Column1 blah blah
entry1 val1
entry2 val2
blah blah Column2 blah blah
entry3 val3
entry4 val4
My code:
from __future__ import unicode_literals
import os, re, string, gzip, fnmatch, io
from array import *
header = re.compile(...) #some regex
valid_entries = re.compile(---) #some regex
matches=[]
entries=[]
value = {'MONTH OF INTERVIEW' : {'01': 'MIN VALUE'}}
counter = 0
name = ''
f =open(r'C:/file.txt')
def exists(data, name):
for key in data.keys():
if key == name :
print "existing key : " + name
return True
else :
return False
for line in f:
col = ''
ent = ''
line = re.sub(ur'\u2013', '-', line)
line = re.sub(ur'\u2026', '_', line)
m = header.match(line)
v = valid_entries.match(line)
if m:
name= ''
matches.append(m.groups())
_,_, name,_,_= m.groups()
#print "name : " + name
if v:
entries.append(v.groups())
ent,col= v.groups()
#print v.groups()
#print "col :" + col
#print "ent :" + ent
if (name is not None) and (ent is not None) and (col is not None):
print value
if exists(value, name):
print 'inside existing loop'
value[name].update({ent:col})
else:
value.update({name:{ent:col}})
print value
problem with this code is , it is replacing the values of the sub dictionary and also it is not adding all the values to the dictionary.
I am new to python, so this could be a naive approach to handle this kind of situation. If you think there is a better way of getting what I want, I would really appreciate if you tell me.

Dictionaries have only one value per key. The trick is to make that value a container too, like a list:
value = {
'column1': [{entry1 : val1}, {entry2 : val2}]
'column2': [{entry3 : val3}, {entry4 : val4}]
}
Use dict.setdefault() to insert a list value when there is no value yet:
if name is not None and ent is not None and col is not None:
value.setdefault(name, []).append({ent: col})
You could just make the values one dictionary with multiple (ent, col) key-value pairs here:
if name is not None and ent is not None and col is not None:
value.setdefault(name, {})[ent] = col
Your exists() function was overcomplicating a task dictionaries excel at; testing for a key is done using in instead:
if name in value:
would have sufficed.

I would keep the keys as a list of dictionaries, so you can extend or append
>>> d = {}
>>> d[1] = [{'a': 1}]
>>> d[1].append({'b':2})
>>> d
{1: [{'a': 1}, {'b': 2}]}

You can use defaultdict and regex for this (demo here):
with open('/path/to/file.txt', 'rU') as f: # read the contents from the file
lines = f.readlines()
import re
from collections import defaultdict
d = defaultdict(list) # dict with default value: []
lastKey = None
for line in lines:
m = re.search('Column\d',line) # search the current line for a key
if m: lastKey = m.group()
else:
m = re.search('(?<=entry\d ).*',line) # search the current line for a value
if m: d[lastKey].append(m.group()) # append the value
Output:
[('Column1', ['val1', 'val2']), ('Column2', ['val3', 'val4'])]
Note: Of course, the above code assumes your file.txt was formatted as in your example. For your real file.txt data you might have to adjust the regex.

Appending values to dictionary in Python

I have a dictionary to which I want to append to each drug, a list of numbers. Like this:
append(0), append(1234), append(123), etc.
def make_drug_dictionary(data):
drug_dictionary={'MORPHINE':[],
'OXYCODONE':[],
'OXYMORPHONE':[],
'METHADONE':[],
'BUPRENORPHINE':[],
'HYDROMORPHONE':[],
'CODEINE':[],
'HYDROCODONE':[]}
prev = None
for row in data:
if prev is None or prev==row[11]:
drug_dictionary.append[row[11][]
return drug_dictionary
I later want to be able to access the entirr set of entries in, for example, 'MORPHINE'.
How do I append a number into the drug_dictionary?
How do I later traverse through each entry?

Just use append:
list1 = [1, 2, 3, 4, 5]
list2 = [123, 234, 456]
d = {'a': [], 'b': []}
d['a'].append(list1)
d['a'].append(list2)
print d['a']

You should use append to add to the list. But also here are few code tips:
I would use dict.setdefault or defaultdict to avoid having to specify the empty list in the dictionary definition.
If you use prev to to filter out duplicated values you can simplfy the code using groupby from itertools
Your code with the amendments looks as follows:
import itertools
def make_drug_dictionary(data):
drug_dictionary = {}
for key, row in itertools.groupby(data, lambda x: x[11]):
drug_dictionary.setdefault(key,[]).append(row[?])
return drug_dictionary
If you don't know how groupby works just check this example:
>>> list(key for key, val in itertools.groupby('aaabbccddeefaa'))
['a', 'b', 'c', 'd', 'e', 'f', 'a']

It sounds as if you are trying to setup a list of lists as each value in the dictionary. Your initial value for each drug in the dict is []. So assuming that you have list1 that you want to append to the list for 'MORPHINE' you should do:
drug_dictionary['MORPHINE'].append(list1)
You can then access the various lists in the way that you want as drug_dictionary['MORPHINE'][0] etc.
To traverse the lists stored against key you would do:
for listx in drug_dictionary['MORPHINE'] :
do stuff on listx

To append entries to the table:
for row in data:
name = ??? # figure out the name of the drug
number = ??? # figure out the number you want to append
drug_dictionary[name].append(number)
To loop through the data:
for name, numbers in drug_dictionary.items():
print name, numbers

If you want to append to the lists of each key inside a dictionary, you can append new values to them using + operator (tested in Python 3.7):
mydict = {'a':[], 'b':[]}
print(mydict)
mydict['a'] += [1,3]
mydict['b'] += [4,6]
print(mydict)
mydict['a'] += [2,8]
print(mydict)
and the output:
{'a': [], 'b': []}
{'a': [1, 3], 'b': [4, 6]}
{'a': [1, 3, 2, 8], 'b': [4, 6]}
mydict['a'].extend([1,3]) will do the job same as + without creating a new list (efficient way).

You can use the update() method as well
d = {"a": 2}
d.update{"b": 4}
print(d) # {"a": 2, "b": 4}

how do i append a number into the drug_dictionary?
Do you wish to add "a number" or a set of values?
I use dictionaries to build associative arrays and lookup tables quite a bit.
Since python is so good at handling strings,
I often use a string and add the values into a dict as a comma separated string
drug_dictionary = {}
drug_dictionary={'MORPHINE':'',
'OXYCODONE':'',
'OXYMORPHONE':'',
'METHADONE':'',
'BUPRENORPHINE':'',
'HYDROMORPHONE':'',
'CODEINE':'',
'HYDROCODONE':''}
drug_to_update = 'MORPHINE'
try:
oldvalue = drug_dictionary[drug_to_update]
except:
oldvalue = ''
# to increment a value
try:
newval = int(oldval)
newval += 1
except:
newval = 1
drug_dictionary[drug_to_update] = "%s" % newval
# to append a value
try:
newval = int(oldval)
newval += 1
except:
newval = 1
drug_dictionary[drug_to_update] = "%s,%s" % (oldval,newval)
The Append method allows for storing a list of values but leaves you will a trailing comma
which you can remove with
drug_dictionary[drug_to_update][:-1]
the result of the appending the values as a string means that you can append lists of values as you need too and
print "'%s':'%s'" % ( drug_to_update, drug_dictionary[drug_to_update])
can return
'MORPHINE':'10,5,7,42,12,'

vowels = ("a","e","i","o","u") #create a list of vowels
my_str = ("this is my dog and a cat") # sample string to get the vowel count
count = {}.fromkeys(vowels,0) #create dict initializing the count to each vowel to 0
for char in my_str :
if char in count:
count[char] += 1
print(count)