Related
I have a list of 2 different dictionaries which I update in a function, however I want to run this in a for loop where for loop two I want to iteratively add to these 2 dictionaries that I already have.
def dictionaries_test(items):
a = dict()
for column in items:
#print("group:",column)
[s, i] = column
combined = '%s,%s' % (s, i)
if combined in a:
a[combined] += 1
else:
a[combined] = 1
return a
def isolation_forest(df, n_trees=2, subspace=256):
X = df.values
forest = []
n = subspace
node_paths = []
c = []
max_depth = np.ceil(np.log(subspace))
for i in range(n_trees):
dataFrame = df.sample(subspace)
tree = isolation_tree(dataFrame, max_depth=max_depth)
# Save tree to forest
forest.append(tree)
for i in range(df.shape[0]):
instance = df.iloc[[i]]
paths = []
d = {}
j = 0
node_paths = []
for tree in forest:
j = j+1
paths.append(pathLength(instance,tree))
node_paths.append(paths[j-1][1])
d.append(dictionaries_test(node_paths))
node_paths.clear()
c.append(d)
print("c:",c)
The problem is that I want to iteratively append new values to c in the already existing 2 dictionaries, I know this is the wrong structure but I don't know how to solve it. Currently c updates and add 2 new dictionaries for each loop. I want to add the keys to the 2 dictionaries, not append 2 more.
c: [[{'Avg_Order_Insert_Size,19.818156222977095': 1,
'Avg_Order_Insert_Size,5.363540682822778': 1,
'Ask_Avg_Volume,9.323600245263918': 1},
{'Ask_Avg_Volume,197.50444935762542': 1,
'Avg_Order_Insert_Size,42.21451340936669': 1,
'Avg_Order_Insert_Size,3.3998163712677645': 1}]]
Second iteration:
c: [{'Avg_Order_Insert_Size,335.1846225162': 1,'Avg_Order_Insert_Size,40.96866540084905': 1,
'Ask_Avg_Volume,13.323205588601743': 1,
'Avg_Order_Insert_Size,659.3733901302467': 1,
'Ask_Avg_Volume,4.417019295299902': 1,
'Avg_Order_Insert_Size,54.943278300874425': 1},
{'Avg_Order_Insert_Size,335.1846225162': 1,
'Avg_Order_Insert_Size,40.96866540084905': 1,
'Ask_Avg_Volume,13.323205588601743': 1,
'Avg_Order_Insert_Size,659.3733901302467': 1,
'Ask_Avg_Volume,4.417019295299902': 1,
'Avg_Order_Insert_Size,54.943278300874425': 1},
{'Avg_Order_Insert_Size,335.1846225162': 1,
'Avg_Order_Insert_Size,40.96866540084905': 1,
'Ask_Avg_Volume,13.323205588601743': 1,
'Avg_Order_Insert_Size,659.3733901302467': 1,
'Ask_Avg_Volume,4.417019295299902': 1,
'Avg_Order_Insert_Size,4.935231863487447': 1}]
So in the variable c I want to add the next loop into the specific dictionaries, and not append them to create 2 more.
I have two dictionaries, and I want to compare them and see what is different between the two. Where I am getting confused is the dict. Is there a name for this?
Everything is working fine, I just don't really understand why it works or what it is doing.
x = {"#04": 0, "#05": 0, "#07": 0, "#08": 1, "#09": 0, "#10": 0, "#11": 1, "#12": 1, "#14": 1, "#15": 1, "#17": 0, "#18": 1, "#19": 1, "#20": 1}
y = {"#04": 1, "#05": 0, "#07": 0, "#08": 1, "#09": 0, "#10": 0, "#11": 1, "#12": 1, "#14": 1, "#15": 0, "#17": 1, "#18": 1, "#19": 0, "#20": 1}
dict = {k: x[k] for k in x if y[k] != x[k]}
list = []
for k, v in dict.items()
if v==0:
difference = k + ' became ' + '0'
list.append(difference)
else:
difference = k + ' became ' + '1'
list.append(difference)
print(list)
It should print ['#04 became 0', '#15 became 1', '#17 became 0', '#19 became 1'] but I don't understand how the dict works to loop through the x and y dictionaries.
The procedure implemented is comparing two dictionaries assuming that both have the same keys (potentially y could have more entries).
To make this comparison quick, and facilitate the next code block, they decided to generate a dictionary that only contains the keys that have different values.
To generate such dictionary, they use a "dictionary comprehension", which is very efficient.
Now, this construct:
d = {k: x[k] for k in x if y[k] != x[k]}
can be rewritten as:
d = {}
for k,v in x: # for each key->value pairs in dictionary x
if y[k] != x[k]: # if the corresponding elements are different
d[k] = x[k] # store the key->value pair in the new dictionary
You could replace x[k] with v above.
I have a series of functions that end up giving a list, with the first item containing a number, derived from the dictionaries, and the second and third items are dictionaries.
These dictionaries have been previously randomly generated.
The function I am using generates a given number of these dictionaries, trying to get the highest number possible as the first item. (It's designed to optimise dice rolls).
This all works fine, and I can print the value of the highest first item from all iterations. However, when I try and print the two dictionaries associated with this first number (bearing in mind they're all in a list together), it just seemingly randomly generates the two other dictionaries.
def repeat(type, times):
best = 0
for i in range(0, times):
x = rollForCharacter(type)
if x[0] > best:
print("BEST:", x)
best = x[0]
print("The highest average success is", best)
return best
This works great. The last thing shown is:
BEST: (3.58, [{'strength': 4, 'intelligence': 1, 'charisma': 1, 'stamina': 4, 'willpower': 2, 'dexterity': 2, 'wits': 5, 'luck': 2}, {'agility': 1, 'brawl': 2, 'investigation': 3, 'larceny': 0, 'melee': 1, 'survival': 0, 'alchemy': 3, 'archery': 0, 'crafting': 0, 'drive': 1, 'magic': 0, 'medicine': 0, 'commercial': 0, 'esteem': 5, 'instruction': 2, 'intimidation': 2, 'persuasion': 0, 'seduction': 0}])
The highest average success is 3.58
But if I try something to store the list which gave this number:
def repeat(type, times):
best = 0
bestChar = []
for i in range(0, times):
x = rollForCharacter(type)
if x[0] > best:
print("BEST:", x)
best = x[0]
bestChar = x
print("The highest average success is", best)
print("Therefore the best character is", bestChar)
return best, bestChar
I get this as the last result, which is fine:
BEST: (4.15, [{'strength': 2, 'intelligence': 3, 'charisma': 4, 'stamina': 4, 'willpower': 1, 'dexterity': 2, 'wits': 4, 'luck': 1}, {'agility': 1, 'brawl': 0, 'investigation': 5, 'larceny': 0, 'melee': 0, 'survival': 0, 'alchemy': 7, 'archery': 0, 'crafting': 0, 'drive': 0, 'magic': 0, 'medicine': 0, 'commercial': 1, 'esteem': 0, 'instruction': 3, 'intimidation': 0, 'persuasion': 0, 'seduction': 0}])
The highest average success is 4.15
but the last line is
Therefore the best character is (4.15, [{'strength': 1, 'intelligence': 3, 'charisma': 4, 'stamina': 4, 'willpower': 1, 'dexterity': 2, 'wits': 2, 'luck': 3}, {'agility': 1, 'brawl': 0, 'investigation': 1, 'larceny': 4, 'melee': 2, 'survival': 0, 'alchemy': 2, 'archery': 4, 'crafting': 0, 'drive': 0, 'magic': 0, 'medicine': 0, 'commercial': 1, 'esteem': 0, 'instruction': 0, 'intimidation': 2, 'persuasion': 1, 'seduction': 0}])
As you can see this doesn't match with what I want, and what is printed literally right above it.
Through a little bit of checking, I realised what it gives out as the "Best Character" is just the last one generated, which is not the best, just the most recent. However, it isn't that simple, because the first element IS the highest result that was recorded, just not from the character in the rest of the list. This is really confusing because it means the list is somehow being edited but at no point can I see where that would happen.
Am I doing something stupid whereby the character is randomly generated every time? I wouldn't think so since x[0] gives the correct result and is stored fine, so what changes when it's the whole list?
From the function rollForCharacter() it returns rollResult, character which is just the number and then the two dictionaries.
I would greatly appreciate it if anyone could figure out and explain where I'm going wrong and why it can print the correct answer to the console yet not store it correctly a line below!
EDIT:
Dictionary 1 Code:
attributes = {}
def assignRow(row, p): # p is the number of points you have to assign to each row
rowValues = {}
for i in range(0, len(row)-1):
val = randint(0, p)
rowValues[row[i]] = val + 1
p -= val
rowValues[row[-1]] = p + 1
return attributes.update(rowValues)
def getPoints():
points = [7, 5, 3]
shuffle(points)
row1 = ['strength', 'intelligence', 'charisma']
row2 = ['stamina', 'willpower']
row3 = ['dexterity', 'wits', 'luck']
for i in range(0, len(points)):
row = eval("row" + str(i+1))
assignRow(row, points[i])
Dictionary 2 Code:
skills = {}
def assignRow(row, p): # p is the number of points you have to assign to each row
rowValues = {}
for i in range(0, len(row) - 1):
val = randint(0, p)
rowValues[row[i]] = val
p -= val
rowValues[row[-1]] = p
return skills.update(rowValues)
def getPoints():
points = [11, 7, 4]
shuffle(points)
row1 = ['agility', 'brawl', 'investigation', 'larceny', 'melee', 'survival']
row2 = ['alchemy', 'archery', 'crafting', 'drive', 'magic', 'medicine']
row3 = ['commercial', 'esteem', 'instruction', 'intimidation', 'persuasion', 'seduction']
for i in range(0, len(points)):
row = eval("row" + str(i + 1))
assignRow(row, points[i])
It does look like the dictionary is being re-generated, which could easily happen if the function rollForCharacter returns either a generator or alternatively is overwriting a global variable which is being overwritten by a subsequent cycle of the loop.
A simple-but-hacky way to solve the problem would be to take a deep copy of the dictionary at the time of storing, so that you're sure you're keeping the values at that point:
def repeat(type, times):
best = 0
bestChar = []
for i in range(0, times):
x = rollForCharacter(type)
if x[0] > best:
print("BEST:", x)
best = x[0]
# Create a brand new tuple, containing a copy of the current dict
bestChar = (x[0], x[1].copy())
The correct answer would be however to pass a unique dictionary variable that is not affected by later code.
See this SO answer with a bit more context about how passing a reference to a dictionary can be risky as it's still a mutable object.
Given a list of data as follows:
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
I would like to create an algorithm that is able to offset the list of certain number of steps. For example, if the offset = -1:
def offsetFunc(inputList, offsetList):
#make something
return output
where:
output = [0,0,0,0,1,1,5,5,5,5,5,5,3,3,3,2,2]
Important Note: The elements of the list are float numbers and they are not in any progression. So I actually need to shift them, I cannot use any work-around for getting the result.
So basically, the algorithm should replace the first set of values (the 4 "1", basically) with the 0 and then it should:
Detect the lenght of the next range of values
Create a parallel output vectors with the values delayed by one set
The way I have roughly described the algorithm above is how I would do it. However I'm a newbie to Python (and even beginner in general programming) and I have figured out time by time that Python has a lot of built-in functions that could make the algorithm less heavy and iterating. Does anyone have any suggestion to better develop a script to make this kind of job? This is the code I have written so far (assuming a static offset at -1):
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
output = []
PrevVal = 0
NextVal = input[0]
i = 0
while input[i] == NextVal:
output.append(PrevVal)
i += 1
while i < len(input):
PrevVal = NextVal
NextVal = input[i]
while input[i] == NextVal:
output.append(PrevVal)
i += 1
if i >= len(input):
break
print output
Thanks in advance for any help!
BETTER DESCRIPTION
My list will always be composed of "sets" of values. They are usually float numbers, and they take values such as this short example below:
Sample = [1.236,1.236,1.236,1.236,1.863,1.863,1.863,1.863,1.863,1.863]
In this example, the first set (the one with value "1.236") is long 4 while the second one is long 6. What I would like to get as an output, when the offset = -1, is:
The value "0.000" in the first 4 elements;
The value "1.236" in the second 6 elements.
So basically, this "offset" function is creating the list with the same "structure" (ranges of lengths) but with the values delayed by "offset" times.
I hope it's clear now, unfortunately the problem itself is still a bit silly to me (plus I don't even speak good English :) )
Please don't hesitate to ask any additional info to complete the question and make it clearer.
How about this:
def generateOutput(input, value=0, offset=-1):
values = []
for i in range(len(input)):
if i < 1 or input[i] == input[i-1]:
yield value
else: # value change in input detected
values.append(input[i-1])
if len(values) >= -offset:
value = values.pop(0)
yield value
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
print list(generateOutput(input))
It will print this:
[0, 0, 0, 0, 1, 1, 5, 5, 5, 5, 5, 5, 3, 3, 3, 2, 2]
And in case you just want to iterate, you do not even need to build the list. Just use for i in generateOutput(input): … then.
For other offsets, use this:
print list(generateOutput(input, 0, -2))
prints:
[0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 5, 5, 5, 3, 3]
Using deque as the queue, and using maxlen to define the shift length. Only holding unique values. pushing inn new values at the end, pushes out old values at the start of the queue, when the shift length has been reached.
from collections import deque
def shift(it, shift=1):
q = deque(maxlen=shift+1)
q.append(0)
for i in it:
if q[-1] != i:
q.append(i)
yield q[0]
Sample = [1.236,1.236,1.236,1.236,1.863,1.863,1.863,1.863,1.863,1.863]
print list(shift(Sample))
#[0, 0, 0, 0, 1.236, 1.236, 1.236, 1.236, 1.236, 1.236]
My try:
#Input
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
shift = -1
#Build service structures: for each 'set of data' store its length and its value
set_lengths = []
set_values = []
prev_value = None
set_length = 0
for value in input:
if prev_value is not None and value != prev_value:
set_lengths.append(set_length)
set_values.append(prev_value)
set_length = 0
set_length += 1
prev_value = value
else:
set_lengths.append(set_length)
set_values.append(prev_value)
#Output the result, shifting the values
output = []
for i, l in enumerate(set_lengths):
j = i + shift
if j < 0:
output += [0] * l
else:
output += [set_values[j]] * l
print input
print output
gives:
[1, 1, 1, 1, 5, 5, 3, 3, 3, 3, 3, 3, 2, 2, 2, 5, 5]
[0, 0, 0, 0, 1, 1, 5, 5, 5, 5, 5, 5, 3, 3, 3, 2, 2]
def x(list, offset):
return [el + offset for el in list]
A completely different approach than my first answer is this:
import itertools
First analyze the input:
values, amounts = zip(*((n, len(list(g))) for n, g in itertools.groupby(input)))
We now have (1, 5, 3, 2, 5) and (4, 2, 6, 3, 2). Now apply the offset:
values = (0,) * (-offset) + values # nevermind that it is longer now.
And synthesize it again:
output = sum([ [v] * a for v, a in zip(values, amounts) ], [])
This is way more elegant, way less understandable and probably way more expensive than my other answer, but I didn't want to hide it from you.
gzip_files=["complete-credit-ctrl-txn-SE06_2013-07-17-00.log.gz","complete-credit-ctrl-txn-SE06_2013-07-17-01.log.gz"]
def input_func():
num = input("Enter the number of MIN series digits: ")
return num
for i in gzip_files:
import gzip
f=gzip.open(i,'rb')
file_content=f.read()
digit = input_func()
file_content = file_content.split('[')
series = [] #list of MIN
for line in file_content:
MIN = line.split('|')[13:15]
for x in MIN:
n = digit
x = x[:n]
series.append(x)
break
#count the number of occurences in the list named series
for i in series:
print i
#end count
Result:
63928
63928
63929
63929
63928
63928
That is only a part of the result. the actual result shows a really long list. Now i want to just list unique numbers and specify how many times it showed on the list.
So
63928 = 4,
63929 = 2
I would use a collections.Counter class here.
>>> a = [1, 1, 1, 2, 3, 4, 4, 5]
>>> from collections import Counter
>>> Counter(a)
Counter({1: 3, 4: 2, 2: 1, 3: 1, 5: 1})
Just pass your series variable to Counter and you'll get a dictionary where the keys are the unique elements and the values are their occurences in the list.
collections.Counter was introduced in Python 2.7. Use the following list comprehension for versions below 2.7
>>> [(elem, a.count(elem)) for elem in set(a)]
[(1, 3), (2, 1), (3, 1), (4, 2), (5, 1)]
You can then just convert this into a dictionary for easy access.
>>> dict((elem, a.count(elem)) for elem in set(a))
{1: 3, 2: 1, 3: 1, 4: 2, 5: 1}
You can use a Counter() for this.
So this will print what you need:
from collections import Counter
c = Counter(series)
for item,count in c.items():
print "%s = %s" % (item,count)
Compile a dictionary using unique numbers as keys, and their total occurrences as values:
d = {} #instantiate dictionary
for s in series:
# set default key and value if key does not exist in dictionary
d.setdefault(s, 0)
# increment by 1 for every occurrence of s
d[s] += 1
If this problem were any more complex. Implementation of map reduce (aka map fold) may be appropriate.
Map Reduce:
https://en.wikipedia.org/wiki/MapReduce
Python map function:
http://docs.python.org/2/library/functions.html#map
Python reduce function:
http://docs.python.org/2/library/functions.html#reduce