Cumulatively add values to a dictionary Python - python

I have a list of 2 different dictionaries which I update in a function, however I want to run this in a for loop where for loop two I want to iteratively add to these 2 dictionaries that I already have.
def dictionaries_test(items):
a = dict()
for column in items:
#print("group:",column)
[s, i] = column
combined = '%s,%s' % (s, i)
if combined in a:
a[combined] += 1
else:
a[combined] = 1
return a
def isolation_forest(df, n_trees=2, subspace=256):
X = df.values
forest = []
n = subspace
node_paths = []
c = []
max_depth = np.ceil(np.log(subspace))
for i in range(n_trees):
dataFrame = df.sample(subspace)
tree = isolation_tree(dataFrame, max_depth=max_depth)
# Save tree to forest
forest.append(tree)
for i in range(df.shape[0]):
instance = df.iloc[[i]]
paths = []
d = {}
j = 0
node_paths = []
for tree in forest:
j = j+1
paths.append(pathLength(instance,tree))
node_paths.append(paths[j-1][1])
d.append(dictionaries_test(node_paths))
node_paths.clear()
c.append(d)
print("c:",c)
The problem is that I want to iteratively append new values to c in the already existing 2 dictionaries, I know this is the wrong structure but I don't know how to solve it. Currently c updates and add 2 new dictionaries for each loop. I want to add the keys to the 2 dictionaries, not append 2 more.
c: [[{'Avg_Order_Insert_Size,19.818156222977095': 1,
'Avg_Order_Insert_Size,5.363540682822778': 1,
'Ask_Avg_Volume,9.323600245263918': 1},
{'Ask_Avg_Volume,197.50444935762542': 1,
'Avg_Order_Insert_Size,42.21451340936669': 1,
'Avg_Order_Insert_Size,3.3998163712677645': 1}]]
Second iteration:
c: [{'Avg_Order_Insert_Size,335.1846225162': 1,'Avg_Order_Insert_Size,40.96866540084905': 1,
'Ask_Avg_Volume,13.323205588601743': 1,
'Avg_Order_Insert_Size,659.3733901302467': 1,
'Ask_Avg_Volume,4.417019295299902': 1,
'Avg_Order_Insert_Size,54.943278300874425': 1},
{'Avg_Order_Insert_Size,335.1846225162': 1,
'Avg_Order_Insert_Size,40.96866540084905': 1,
'Ask_Avg_Volume,13.323205588601743': 1,
'Avg_Order_Insert_Size,659.3733901302467': 1,
'Ask_Avg_Volume,4.417019295299902': 1,
'Avg_Order_Insert_Size,54.943278300874425': 1},
{'Avg_Order_Insert_Size,335.1846225162': 1,
'Avg_Order_Insert_Size,40.96866540084905': 1,
'Ask_Avg_Volume,13.323205588601743': 1,
'Avg_Order_Insert_Size,659.3733901302467': 1,
'Ask_Avg_Volume,4.417019295299902': 1,
'Avg_Order_Insert_Size,4.935231863487447': 1}]
So in the variable c I want to add the next loop into the specific dictionaries, and not append them to create 2 more.

Related

Can someone explain how this loops through a dictionary?

I have two dictionaries, and I want to compare them and see what is different between the two. Where I am getting confused is the dict. Is there a name for this?
Everything is working fine, I just don't really understand why it works or what it is doing.
x = {"#04": 0, "#05": 0, "#07": 0, "#08": 1, "#09": 0, "#10": 0, "#11": 1, "#12": 1, "#14": 1, "#15": 1, "#17": 0, "#18": 1, "#19": 1, "#20": 1}
y = {"#04": 1, "#05": 0, "#07": 0, "#08": 1, "#09": 0, "#10": 0, "#11": 1, "#12": 1, "#14": 1, "#15": 0, "#17": 1, "#18": 1, "#19": 0, "#20": 1}
dict = {k: x[k] for k in x if y[k] != x[k]}
list = []
for k, v in dict.items()
if v==0:
difference = k + ' became ' + '0'
list.append(difference)
else:
difference = k + ' became ' + '1'
list.append(difference)
print(list)
It should print ['#04 became 0', '#15 became 1', '#17 became 0', '#19 became 1'] but I don't understand how the dict works to loop through the x and y dictionaries.
The procedure implemented is comparing two dictionaries assuming that both have the same keys (potentially y could have more entries).
To make this comparison quick, and facilitate the next code block, they decided to generate a dictionary that only contains the keys that have different values.
To generate such dictionary, they use a "dictionary comprehension", which is very efficient.
Now, this construct:
d = {k: x[k] for k in x if y[k] != x[k]}
can be rewritten as:
d = {}
for k,v in x: # for each key->value pairs in dictionary x
if y[k] != x[k]: # if the corresponding elements are different
d[k] = x[k] # store the key->value pair in the new dictionary
You could replace x[k] with v above.

Storing randomly generated dictionaries in Python

I have a series of functions that end up giving a list, with the first item containing a number, derived from the dictionaries, and the second and third items are dictionaries.
These dictionaries have been previously randomly generated.
The function I am using generates a given number of these dictionaries, trying to get the highest number possible as the first item. (It's designed to optimise dice rolls).
This all works fine, and I can print the value of the highest first item from all iterations. However, when I try and print the two dictionaries associated with this first number (bearing in mind they're all in a list together), it just seemingly randomly generates the two other dictionaries.
def repeat(type, times):
best = 0
for i in range(0, times):
x = rollForCharacter(type)
if x[0] > best:
print("BEST:", x)
best = x[0]
print("The highest average success is", best)
return best
This works great. The last thing shown is:
BEST: (3.58, [{'strength': 4, 'intelligence': 1, 'charisma': 1, 'stamina': 4, 'willpower': 2, 'dexterity': 2, 'wits': 5, 'luck': 2}, {'agility': 1, 'brawl': 2, 'investigation': 3, 'larceny': 0, 'melee': 1, 'survival': 0, 'alchemy': 3, 'archery': 0, 'crafting': 0, 'drive': 1, 'magic': 0, 'medicine': 0, 'commercial': 0, 'esteem': 5, 'instruction': 2, 'intimidation': 2, 'persuasion': 0, 'seduction': 0}])
The highest average success is 3.58
But if I try something to store the list which gave this number:
def repeat(type, times):
best = 0
bestChar = []
for i in range(0, times):
x = rollForCharacter(type)
if x[0] > best:
print("BEST:", x)
best = x[0]
bestChar = x
print("The highest average success is", best)
print("Therefore the best character is", bestChar)
return best, bestChar
I get this as the last result, which is fine:
BEST: (4.15, [{'strength': 2, 'intelligence': 3, 'charisma': 4, 'stamina': 4, 'willpower': 1, 'dexterity': 2, 'wits': 4, 'luck': 1}, {'agility': 1, 'brawl': 0, 'investigation': 5, 'larceny': 0, 'melee': 0, 'survival': 0, 'alchemy': 7, 'archery': 0, 'crafting': 0, 'drive': 0, 'magic': 0, 'medicine': 0, 'commercial': 1, 'esteem': 0, 'instruction': 3, 'intimidation': 0, 'persuasion': 0, 'seduction': 0}])
The highest average success is 4.15
but the last line is
Therefore the best character is (4.15, [{'strength': 1, 'intelligence': 3, 'charisma': 4, 'stamina': 4, 'willpower': 1, 'dexterity': 2, 'wits': 2, 'luck': 3}, {'agility': 1, 'brawl': 0, 'investigation': 1, 'larceny': 4, 'melee': 2, 'survival': 0, 'alchemy': 2, 'archery': 4, 'crafting': 0, 'drive': 0, 'magic': 0, 'medicine': 0, 'commercial': 1, 'esteem': 0, 'instruction': 0, 'intimidation': 2, 'persuasion': 1, 'seduction': 0}])
As you can see this doesn't match with what I want, and what is printed literally right above it.
Through a little bit of checking, I realised what it gives out as the "Best Character" is just the last one generated, which is not the best, just the most recent. However, it isn't that simple, because the first element IS the highest result that was recorded, just not from the character in the rest of the list. This is really confusing because it means the list is somehow being edited but at no point can I see where that would happen.
Am I doing something stupid whereby the character is randomly generated every time? I wouldn't think so since x[0] gives the correct result and is stored fine, so what changes when it's the whole list?
From the function rollForCharacter() it returns rollResult, character which is just the number and then the two dictionaries.
I would greatly appreciate it if anyone could figure out and explain where I'm going wrong and why it can print the correct answer to the console yet not store it correctly a line below!
EDIT:
Dictionary 1 Code:
attributes = {}
def assignRow(row, p): # p is the number of points you have to assign to each row
rowValues = {}
for i in range(0, len(row)-1):
val = randint(0, p)
rowValues[row[i]] = val + 1
p -= val
rowValues[row[-1]] = p + 1
return attributes.update(rowValues)
def getPoints():
points = [7, 5, 3]
shuffle(points)
row1 = ['strength', 'intelligence', 'charisma']
row2 = ['stamina', 'willpower']
row3 = ['dexterity', 'wits', 'luck']
for i in range(0, len(points)):
row = eval("row" + str(i+1))
assignRow(row, points[i])
Dictionary 2 Code:
skills = {}
def assignRow(row, p): # p is the number of points you have to assign to each row
rowValues = {}
for i in range(0, len(row) - 1):
val = randint(0, p)
rowValues[row[i]] = val
p -= val
rowValues[row[-1]] = p
return skills.update(rowValues)
def getPoints():
points = [11, 7, 4]
shuffle(points)
row1 = ['agility', 'brawl', 'investigation', 'larceny', 'melee', 'survival']
row2 = ['alchemy', 'archery', 'crafting', 'drive', 'magic', 'medicine']
row3 = ['commercial', 'esteem', 'instruction', 'intimidation', 'persuasion', 'seduction']
for i in range(0, len(points)):
row = eval("row" + str(i + 1))
assignRow(row, points[i])
It does look like the dictionary is being re-generated, which could easily happen if the function rollForCharacter returns either a generator or alternatively is overwriting a global variable which is being overwritten by a subsequent cycle of the loop.
A simple-but-hacky way to solve the problem would be to take a deep copy of the dictionary at the time of storing, so that you're sure you're keeping the values at that point:
def repeat(type, times):
best = 0
bestChar = []
for i in range(0, times):
x = rollForCharacter(type)
if x[0] > best:
print("BEST:", x)
best = x[0]
# Create a brand new tuple, containing a copy of the current dict
bestChar = (x[0], x[1].copy())
The correct answer would be however to pass a unique dictionary variable that is not affected by later code.
See this SO answer with a bit more context about how passing a reference to a dictionary can be risky as it's still a mutable object.

How to check check a key's value within an if statement

I hope you are all well.
This is how my data looks:
dictionary1 = {2876: 1, 9212: 1, 953997: 1, 9205: 1, 9206: 1, 9207: 1, 9208: 1, 9209: 1, 9210: 1, 9211: 1, 6908: 1, 1532: 1, 945237: 1, 6532: 2, 6432: 4}
data1 = [[2876, 5423],[2312, 4532],[953997, 5643]...]
I am trying to run a statement that looks like this:
for y in data1:
if y[0] in dictionary1 and dictionary1[y[0]] == 1:
dictionary1[y[1]] = 2
Presumably this would create a new dataset looking like this:
dictionary1 = {5423: 2, 953997: 2, 2876: 1, 9212: 1, 953997: 1, 9205: 1, 9206: 1, 9207: 1, 9208: 1, 9209: 1, 9210: 1, 9211: 1, 6908: 1, 1532: 1, 945237: 1, 6532: 2, 6432: 4}
What am I doing wrong? Is dictionary1[y[0]] == 1 the correct way to check a key's value?
Thank you everyone.
Dictionary comprehension converts the list of lists to a dictionary:
dict1 = {t[0]:t[1:] for t in dictionary1}
Then it should be easy to do what you want:
for y in data1:
if y in dict1 and dict1[y] ==1:
dictionary1[y] = 2
You can use dict.get(key, default) to avoid an exception for missing values, and provide a safe default. This reduces your loop to a single condition:
#!python3
dictionary1 = {2876: 1, 9212: 1, 953997: 1, 9205: 1, 9206: 1, 9207: 1, 9208: 1, 9209: 1, 9210: 1, 9211: 1, 6908: 1, 1532: 1, 945237: 1, 6532: 2, 6432: 4}
data1 = [[2876, 5423],[2312, 4532],[953997, 5643]]
for x,y in data1:
if dictionary1.get(x, 0) == 1:
dictionary1[y] = 2
print(dictionary1)
You could use dict.update(other) to bulk-overwrite the values in dictionary1 with a one-liner dict comprehension:
dictcompr = {b:2 for a,b in data1 if dictionary1.get(a,0) == 1}
dictionary1.update(dictcompr)
And then you can combine them into one single, unholy, unmaintainable, barely-readable mess:
dictionary1.update({b:2 for a,b in data1 if dictionary1.get(a,0) == 1})
Update:
To delete all keys having a value of 1, you have some choices:
for k,v in dictionary1.items():
if v == 1:
del dictionary1[k]
# Versus:
d2 = dict(filter(lambda item: item[1] != 1, dictionary1.items()))
dictionary1 = d2
# or
dictionary1.clear()
dictionary1.update(d2)
Frankly, for your purposes the for loop is probably better. The filter approach can take the lambda as a parameter, to configure what gets filtered. Using clear()/update() is a win if you expect multiple references to the dictionary. That is, A = B = dictionary1. In this case, clear/update would keep the same underlying object, so the linkage still holds. (This is also true of the for loop - the benefit is solely for the filter which requires a temporary.)
please try this,
for y in data1:
if y[0] in dictionary1.keys() and dictionary1.keys() == y[0]:
dictionary1[y[1]] = 2
u can simply use
for y in data1:
if dictionary1.has_key(y[0]):
dictionary1[y[1]] = 2
Hope this is what u r looking for .

Algorithm to offset a list of data

Given a list of data as follows:
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
I would like to create an algorithm that is able to offset the list of certain number of steps. For example, if the offset = -1:
def offsetFunc(inputList, offsetList):
#make something
return output
where:
output = [0,0,0,0,1,1,5,5,5,5,5,5,3,3,3,2,2]
Important Note: The elements of the list are float numbers and they are not in any progression. So I actually need to shift them, I cannot use any work-around for getting the result.
So basically, the algorithm should replace the first set of values (the 4 "1", basically) with the 0 and then it should:
Detect the lenght of the next range of values
Create a parallel output vectors with the values delayed by one set
The way I have roughly described the algorithm above is how I would do it. However I'm a newbie to Python (and even beginner in general programming) and I have figured out time by time that Python has a lot of built-in functions that could make the algorithm less heavy and iterating. Does anyone have any suggestion to better develop a script to make this kind of job? This is the code I have written so far (assuming a static offset at -1):
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
output = []
PrevVal = 0
NextVal = input[0]
i = 0
while input[i] == NextVal:
output.append(PrevVal)
i += 1
while i < len(input):
PrevVal = NextVal
NextVal = input[i]
while input[i] == NextVal:
output.append(PrevVal)
i += 1
if i >= len(input):
break
print output
Thanks in advance for any help!
BETTER DESCRIPTION
My list will always be composed of "sets" of values. They are usually float numbers, and they take values such as this short example below:
Sample = [1.236,1.236,1.236,1.236,1.863,1.863,1.863,1.863,1.863,1.863]
In this example, the first set (the one with value "1.236") is long 4 while the second one is long 6. What I would like to get as an output, when the offset = -1, is:
The value "0.000" in the first 4 elements;
The value "1.236" in the second 6 elements.
So basically, this "offset" function is creating the list with the same "structure" (ranges of lengths) but with the values delayed by "offset" times.
I hope it's clear now, unfortunately the problem itself is still a bit silly to me (plus I don't even speak good English :) )
Please don't hesitate to ask any additional info to complete the question and make it clearer.
How about this:
def generateOutput(input, value=0, offset=-1):
values = []
for i in range(len(input)):
if i < 1 or input[i] == input[i-1]:
yield value
else: # value change in input detected
values.append(input[i-1])
if len(values) >= -offset:
value = values.pop(0)
yield value
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
print list(generateOutput(input))
It will print this:
[0, 0, 0, 0, 1, 1, 5, 5, 5, 5, 5, 5, 3, 3, 3, 2, 2]
And in case you just want to iterate, you do not even need to build the list. Just use for i in generateOutput(input): … then.
For other offsets, use this:
print list(generateOutput(input, 0, -2))
prints:
[0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 5, 5, 5, 3, 3]
Using deque as the queue, and using maxlen to define the shift length. Only holding unique values. pushing inn new values at the end, pushes out old values at the start of the queue, when the shift length has been reached.
from collections import deque
def shift(it, shift=1):
q = deque(maxlen=shift+1)
q.append(0)
for i in it:
if q[-1] != i:
q.append(i)
yield q[0]
Sample = [1.236,1.236,1.236,1.236,1.863,1.863,1.863,1.863,1.863,1.863]
print list(shift(Sample))
#[0, 0, 0, 0, 1.236, 1.236, 1.236, 1.236, 1.236, 1.236]
My try:
#Input
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
shift = -1
#Build service structures: for each 'set of data' store its length and its value
set_lengths = []
set_values = []
prev_value = None
set_length = 0
for value in input:
if prev_value is not None and value != prev_value:
set_lengths.append(set_length)
set_values.append(prev_value)
set_length = 0
set_length += 1
prev_value = value
else:
set_lengths.append(set_length)
set_values.append(prev_value)
#Output the result, shifting the values
output = []
for i, l in enumerate(set_lengths):
j = i + shift
if j < 0:
output += [0] * l
else:
output += [set_values[j]] * l
print input
print output
gives:
[1, 1, 1, 1, 5, 5, 3, 3, 3, 3, 3, 3, 2, 2, 2, 5, 5]
[0, 0, 0, 0, 1, 1, 5, 5, 5, 5, 5, 5, 3, 3, 3, 2, 2]
def x(list, offset):
return [el + offset for el in list]
A completely different approach than my first answer is this:
import itertools
First analyze the input:
values, amounts = zip(*((n, len(list(g))) for n, g in itertools.groupby(input)))
We now have (1, 5, 3, 2, 5) and (4, 2, 6, 3, 2). Now apply the offset:
values = (0,) * (-offset) + values # nevermind that it is longer now.
And synthesize it again:
output = sum([ [v] * a for v, a in zip(values, amounts) ], [])
This is way more elegant, way less understandable and probably way more expensive than my other answer, but I didn't want to hide it from you.

Updating integer values of a dictionary in python

I have a dictionary with 3 values associated to each key,
I want to know how to increment the values of noTaken & totalBranch as i pass through the data file, as my current method doesn't change the values, the output of lookup gives me (1,1,0) or (0,1,0) - i need the first two values to increase
for line in datafile.readlines():
items = line.split(' ')
instruction = items[1]
if lookup.has_key(instruction):
if (taken == 1):
lookup[instruction] = (noTaken + 1, totalBranch + 1, prediction)
else:
lookup[instruction] = (noTaken, totalBranch + 1, prediction)
else:
if (taken == 1):
lookup[instruction] = (1, 1, prediction)
else:
lookup[instruction] = (0, 1, prediction)
(noTaken, prediction & totalBranch are all initialised as 0 above this)
Thanks in advance!
A cleaner way to initialize is to use defaultdict , then you can directly refer to elements in dict values e.g.
from collections import defaultdict
lookup = defaultdict(lambda: [0,0,0])
lookup['a'][0] += 1
lookup['b'][1] += 1
lookup['a'][0] += 1
print lookup
output:
{'a': [2, 0, 0], 'b': [0, 1, 0]}
Also note that I am defaulting value to a list instead of tuple so that we can modify values in place, tuple being immutable can't be modified

Categories

Resources