Develop Reference

how to select a value referring to the last repeated item in a list

I have 5 columns ( NO = index of vehicle / LEADER = index of the vehicle of the front / SEC = instant in seconds / X = position of the vehicle)
Some vehicles stop ( X stay the same for a while) and I want to get the exact time it starts to move again. Then, calculate the difference between their instant and their respective 'leader' vehicle.
I made a code but it has so many bugs
OBS: Some vehicles never stop or stop,but never return to move. I want to remove them too.
Here's my code:
dados = pd.read_excel('teste-reaction_001.xlsx')
n=dados["NO"].unique()
final=np.zeros(1)
for i in n:
botao = 0
array_aux=dados[dados["NO"] == i ]["X"]
df = pd.DataFrame(array_aux)
aux=df.diff().to_numpy()
count1 = 0
aux1 = []
for j in aux:
if j == 0:
botao = 1
elif j != 0 and botao == 1:
aux1=np.where(aux==j)[0]
aux1=aux1[np.where(aux1>=count1)[0]]
break
else :
botao = 0
count1 = count1 + 1
aux2=dados["SEC"][dados[dados["NO"]==i]["SEC"].index[aux1]].values[0]
final=np.append(final,aux2)
tr=np.zeros(1)
for i in n:
aux=dados[dados["NO"] == i ]["LEADER"].unique()[0]
aux1=np.where(dados["NO"].unique()==i)[0]
aux2=np.where(dados["NO"].unique()==aux)[0]
if aux2>-1:
aux3=final[int(aux1)]-final[aux2]
else:
aux3 = "s"
tr=np.append(tr,aux3)
columns = ["N", "TR"]
tabela = np.array([dados["NO"].unique(), tr[1:]])
res = pd.DataFrame(data=tabela.T,index=np.arange(len(tabela.T)), columns=columns)

List values change after saving them

I am currently writing an eight queens problem solving algorithm.
The alogorithm is not the problem tho. Its the storing of the solutions.
I am trying to append the solutions (solutions are stored in a list of lists) to a list,
but after saving the solution and continuing through the algorithm,
the Values of the saved list keep changing.
I suppose the saved list and the one I am changing are somehow still "connected",
but I dont know how.
Does anyone know a solution or a different approach to my saving strategy?
This is the part of the code I am having Problems with.
# The Code only gets executed if there is a valid solution.
if found_solution():
# Saving solution
solutions.append(board)
# deleting last Queen
temp = last_queen.pop()
board[temp[0]][temp[1]] = "-"
# going back one Row
Reihe -= 1
# return
return
The board does look like this
board = [["-","-","-","-"],
["-","-","-","-"],
["-","-","-","-"],
["-","-","-","-"]]
the solution list like this
solutions = []
If anyone wants to take a look at the whole code (There are a few german variables tho):
board = [["-","-","-","-"],
["-","-","-","-"],
["-","-","-","-"],
["-","-","-","-"]]
Reihe = 0
solutions = []
last_queen = []
def found_solution():
count = 0
for i in range(4):
for j in range(4):
if board[i][j] == "D":
count += 1
if count == 4:
return True
return False
def board_clear(temp_Reihe, temp_i):
#Check Column and Row
for k in range(4):
if board[temp_Reihe][k] == "D":
return False
if board[k][temp_i] == "D":
return False
#Check Diagonals
#temp_i == x and temp_Reihe == y
st1_x = temp_i - min(temp_i, temp_Reihe)
st1_y = temp_Reihe - min(temp_i, temp_Reihe)
st2_x = temp_i - min(temp_i, 3-temp_Reihe)
st2_y = temp_Reihe + min(temp_i, 3-temp_Reihe)
while st1_x < 4 and st1_y < 4:
if board[st1_y][st1_x] == "D":
return False
st1_x += 1
st1_y += 1
while st2_x < 4 and st2_y > -1:
if board[st2_y][st2_x] == "D":
return False
st2_x += 1
st2_y -= 1
return True
def main():
global Reihe, board, solutions, last_queen
if found_solution():
#Saving solution
solutions.append(board)
#deleting last Queen
temp = last_queen.pop()
board[temp[0]][temp[1]] = "-"
#going back one Row
Reihe -= 1
return
for i in range(4):
#placing Queen if valid spot
if board_clear(Reihe, i):
board[Reihe][i] = "D"
last_queen.append([Reihe,i])
Reihe += 1
main()
#delete last Queen
temp = last_queen.pop()
board[temp[0]][temp[1]] = "-"
#going back one Row
Reihe -= 1
return
main()

People have suggested using copy or deepcopy
list2 = list1.deepcopy()

Python generator confusing behavior

This question arises when I read a generator tutorial https://realpython.com/introduction-to-python-generators/. is_palindrome(101) returns True, but the code didn't print 101. I debugged and saw yield 101 led the execution back to the main for loop, which immediately went back to the generator, which is very confusing. The generator yields 101, hence it should be an element of for j in pal_gen:, isn't it?
def is_palindrome(num):
# Skip single-digit inputs
if num // 10 == 0:
return False
temp = num
reversed_num = 0
while temp != 0:
reversed_num = (reversed_num * 10) + (temp % 10)
temp = temp // 10
if num == reversed_num:
return True
else:
return False
def infinite_palindromes():
num = 0
while True:
if is_palindrome(num):
i = (yield num)
if i is not None:
num = i
num += 1
if __name__=='__main__':
pal_gen = infinite_palindromes()
for j in pal_gen:
print(f'j={j}')
digits = len(str(j))
if digits == 5:
pal_gen.close()
pal_gen.send(10 ** (digits))
Currently the output is:
j=11
j=111
j=1111
j=10101
Traceback (most recent call last):
File "generator.py", line 33, in <module>
pal_gen.send(10 ** (digits))
StopIteration

You'll notice it is skipping one between each. 101 is missing, 1001 is missing. 10001 is missing. I'm pretty sure send leads the generator to run to the next yield statement. For example, use the following code:
if __name__=='__main__':
pal_gen = infinite_palindromes()
for j in pal_gen:
print('j={}'.format(j))
digits = len(str(j))
if digits == 5:
pal_gen.close()
print(pal_gen.send(10 ** (digits)))
This will print out the missing values between the "j's":
j=11
101
j=111
1001
j=1111
10001
j=10101
Consider redoing your for statement with a while:
if __name__=='__main__':
pal_gen = infinite_palindromes()
j=next(pal_gen)
digits=0
while digits<5:
print('j={}'.format(j))
digits = len(str(j))
j = pal_gen.send(10 ** (digits))
else:
pal_gen.close()

Slow performance in agent based model python

I originally posted this on code-review (hence the lengthy code) but failed to get an answer.
My model is based on this game https://en.wikipedia.org/wiki/Ultimatum_game . I won't go into the intuition behind it but generally speaking it functions as follows:
The game consists of a n x n lattice on which an agent is placed at each node.
During each time step, each player on each node plays against a random neighbour by playing a particular strategy.
Each of their strategies (a value between 1-9) has a propensity attached to it (which is randomly assigned and is just some number). The propensity then in turn determines the probability of playing that strategy. The probability is calculated as the propensity of that strategy over the sum of the propensities of all strategies.
If a game results in a positive payoff, then the payoffs from that game get added to the propensities for those strategies.
These propensities then determine the probabilities for their strategies in the next time step, and so on.
The simulation ends after time step N is reached.
For games with large lattices and large time steps, my code runs really really slowly. I ran cProfiler to check where the bottleneck(s) are, and as I suspected the update_probabilitiesand play_rounds functions seem to be taking up a lot time. I want to be able to run the game with gridsize of about 40x40 for about 100000+ time steps, but right now that is not happening.
So what would be a more efficient way to calculate and update the probabilities/propensities of each player in the grid? I've considered implementing NumPy arrays but I am not sure if it would be worth the hassle here?
import numpy as np
import random
from random import randint
from numpy.random import choice
from numpy.random import multinomial
import cProfile
mew = 0.001
error = 0.05
def create_grid(row, col):
return [[0 for j in range(col)] for i in range(row)]
def create_random_propensities():
propensities = {}
pre_propensities = [random.uniform(0, 1) for i in range(9)]
a = np.sum(pre_propensities)
for i in range(1, 10):
propensities[i] = (pre_propensities[i - 1]/a) * 10 # normalize sum of propensities to 10
return propensities
class Proposer:
def __init__(self):
self.propensities = create_random_propensities()
self.probabilites = []
self.demand = 0 # the amount the proposer demands for themselves
def pick_strat(self, n_trials): # gets strategy, an integer in the interval [1, 9]
results = multinomial(n_trials, self.probabilites)
i, = np.where(results == max(results))
if len(i) > 1:
return choice(i) + 1
else:
return i[0] + 1
def calculate_probability(self, dict_data, index, total_sum): # calculates probability for particular strat, taking propensity
return dict_data[index]/total_sum # of that strat as input
def calculate_sum(self, dict_data):
return sum(dict_data.values())
def initialize(self):
init_sum = self.calculate_sum(self.propensities)
for strategy in range(1, 10):
self.probabilites.append(self.calculate_probability(self.propensities, strategy, init_sum))
self.demand = self.pick_strat(1)
def update_strategy(self):
self.demand = self.pick_strat(1)
def update_probablities(self):
for i in range(9):
self.propensities[1 + i] *= 1 - mew
pensity_sum = self.calculate_sum(self.propensities)
for i in range(9):
self.probabilites[i] = self.calculate_probability(self.propensities, 1 + i, pensity_sum)
def update(self):
self.update_probablities()
self.update_strategy()
class Responder: # methods same as proposer class, can skip-over
def __init__(self):
self.propensities = create_random_propensities()
self.probabilites = []
self.max_thresh = 0 # the maximum demand they are willing to accept
def pick_strat(self, n_trials):
results = multinomial(n_trials, self.probabilites)
i, = np.where(results == max(results))
if len(i) > 1:
return choice(i) + 1
else:
return i[0] + 1
def calculate_probability(self, dict_data, index, total_sum):
return dict_data[index]/total_sum
def calculate_sum(self, dict_data):
return sum(dict_data.values())
def initialize(self):
init_sum = self.calculate_sum(self.propensities)
for strategy in range(1, 10):
self.probabilites.append(self.calculate_probability(self.propensities, strategy, init_sum))
self.max_thresh = self.pick_strat(1)
def update_strategy(self):
self.max_thresh = self.pick_strat(1)
def update_probablities(self):
for i in range(9):
self.propensities[1 + i] *= 1 - mew # stops sum of propensites from growing without bound
pensity_sum = self.calculate_sum(self.propensities)
for i in range(9):
self.probabilites[i] = self.calculate_probability(self.propensities, 1 + i, pensity_sum)
def update(self):
self.update_probablities()
self.update_strategy()
class Agent:
def __init__(self):
self.prop_side = Proposer()
self.resp_side = Responder()
self.prop_side.initialize()
self.resp_side.initialize()
def update_all(self):
self.prop_side.update()
self.resp_side.update()
class Grid:
def __init__(self, rowsize, colsize):
self.rowsize = rowsize
self.colsize = colsize
def make_lattice(self):
return [[Agent() for j in range(self.colsize)] for i in range(self.rowsize)]
#staticmethod
def von_neumann_neighbourhood(array, row, col, wrapped=True): # gets up, bottom, left, right neighbours of some node
neighbours = set([])
if row + 1 <= len(array) - 1:
neighbours.add(array[row + 1][col])
if row - 1 >= 0:
neighbours.add(array[row - 1][col])
if col + 1 <= len(array[0]) - 1:
neighbours.add(array[row][col + 1])
if col - 1 >= 0:
neighbours.add(array[row][col - 1])
#if wrapped is on, conditions for out of bound points
if row - 1 < 0 and wrapped == True:
neighbours.add(array[-1][col])
if col - 1 < 0 and wrapped == True:
neighbours.add(array[row][-1])
if row + 1 > len(array) - 1 and wrapped == True:
neighbours.add(array[0][col])
if col + 1 > len(array[0]) - 1 and wrapped == True:
neighbours.add(array[row][0])
return neighbours
def get_error_term(pay, strategy):
index_strat_2, index_strat_8 = 2, 8
if strategy == 1:
return (1 - (error/2)) * pay, error/2 * pay, index_strat_2
if strategy == 9:
return (1 - (error/2)) * pay, error/2 * pay, index_strat_8
else:
return (1 - error) * pay, error/2 * pay, 0
class Games:
def __init__(self, n_rows, n_cols, n_rounds):
self.rounds = n_rounds
self.rows = n_rows
self.cols = n_cols
self.lattice = Grid(self.rows, self.cols).make_lattice()
self.lookup_table = np.full((self.rows, self.cols), False, dtype=bool) # if player on grid has updated their strat, set to True
def reset_look_tab(self):
self.lookup_table = np.full((self.rows, self.cols), False, dtype=bool)
def run_game(self):
n = 0
while n < self.rounds:
for r in range(self.rows):
for c in range(self.cols):
if n != 0:
self.lattice[r][c].update_all()
self.lookup_table[r][c] = True
self.play_rounds(self.lattice, r, c)
self.reset_look_tab()
n += 1
def play_rounds(self, grid, row, col):
neighbours = Grid.von_neumann_neighbourhood(grid, row, col)
neighbour = random.sample(neighbours, 1).pop()
neighbour_index = [(ix, iy) for ix, row in enumerate(self.lattice) for iy, i in enumerate(row) if i == neighbour]
if self.lookup_table[neighbour_index[0][0]][neighbour_index[0][1]] == False: # see if neighbour has already updated their strat
neighbour.update_all()
player = grid[row][col]
coin_toss = randint(0, 1) # which player acts as proposer or responder in game
if coin_toss == 1:
if player.prop_side.demand <= neighbour.resp_side.max_thresh: # postive payoff
payoff, adjacent_payoff, index = get_error_term(player.prop_side.demand, player.prop_side.demand)
if player.prop_side.demand == 1 or player.prop_side.demand == 9: # extreme strategies get bonus payoffs
player.prop_side.propensities[player.prop_side.demand] += payoff
player.prop_side.propensities[index] += adjacent_payoff
else:
player.prop_side.propensities[player.prop_side.demand] += payoff
player.prop_side.propensities[player.prop_side.demand - 1] += adjacent_payoff
player.prop_side.propensities[player.prop_side.demand + 1] += adjacent_payoff
else:
return 0 # if demand > max thresh -> both get zero
if coin_toss != 1:
if neighbour.prop_side.demand <= player.resp_side.max_thresh:
payoff, adjacent_payoff, index = get_error_term(10 - neighbour.prop_side.demand, player.resp_side.max_thresh)
if player.resp_side.max_thresh == 1 or player.resp_side.max_thresh == 9:
player.resp_side.propensities[player.resp_side.max_thresh] += payoff
player.resp_side.propensities[index] += adjacent_payoff
else:
player.resp_side.propensities[player.resp_side.max_thresh] += payoff
player.resp_side.propensities[player.resp_side.max_thresh - 1] += adjacent_payoff
player.resp_side.propensities[player.resp_side.max_thresh + 1] += adjacent_payoff
else:
return 0
#pr = cProfile.Profile()
#pr.enable()
my_game = Games(10, 10, 2000) # (rowsize, colsize, n_steps)
my_game.run_game()
#pr.disable()
#pr.print_stats(sort='time')
(For those who might be wondering, the get_error_term just returns the propensities for strategies that are next to strategies that receive a positive payoff, for example if the strategy 8 works, then 7 and 9's propensities also get adjusted upwards and this is calculated by said function. And the first for loop inside update_probabilities just makes sure that the sum of propensities don't grow without bound).

Python stack overlapped element cumulation

I want to cumulatively add element on Arraystack in Python
class StockManager:
__PROFIT = 0
def __init__(self, o_name, o_surname):
self.box = ArrayStack()
self._name = o_name
self._surname = o_surname
def buy_shares(self, company, number, buy_price):
StockManager.__PROFIT -= buy_price * number
n = 0
if len(self.box) < 2:
"""there is no overlapped elements when box has no element"""
self.box.push(company)
self.box.push(number)
else:
while n < len(self.box):
if self.box._data[n] == company:
"""every even index(n = 0, 2, 4, ...) refers to company name"""
"""every odd index(n = 1, 3, 5, ...), which is next to even index refers to number of buying"""
self.box._data[n + 1] += number
n += 2
elif self.box._data[n] != company:
""" if there's no overlapping, then just put the elements """
self.box.push(company)
self.box.push(number)
n += 2
return print(self.box._data)
and class Arraystack is like this:
class ArrayStack:
"""LIFO Stack implementation using a Python list as underlying storage."""
def __init__(self):
"""Create an empty stack."""
self._data = [] # nonpublic list instance
def __len__(self):
"""Return the number of elements in the stack."""
return len(self._data)
def is_empty(self):
"""Return True if the stack is empty."""
return len(self._data) == 0
def push(self, e):
"""Add element e to the top of the stack."""
self._data.append(e) # new item stored at end of list
def top(self):
"""Return (but do not remove) the element at the top of the stack.
Raise Empty exception if the stack is empty.
"""
if self.is_empty():
raise AssertionError('Stack is empty')
return self._data[-1] # the last item in the list
def pop(self):
"""Remove and return the element from the top of the stack (i.e., LIFO).
Raise Empty exception if the stack is empty.
"""
if self.is_empty():
raise AssertionError('Stack is empty')
return self._data.pop() # remove last item from list
def print_contents(self):
print("Stack content: {0}".format(self._data))
when I run stockmanager with
if __name__ == '__main__':
P = StockManager("A","B")
P.buy_shares("hyundai", 20, 100)
P.buy_shares("hyundai", 20, 100)
P.buy_shares("hyundai", 20, 100)
P.buy_shares("lg", 20, 100)
the result is
['hyundai', 20] => O.K
['hyundai', 40] => O.K
['hyundai', 60] => O.K
['hyundai', 60, 'lg', 40] => It should be ['hyundai', 60, 'lg', 20]
['hyundai', 60, 'lg', 60, 'lg', 40] => don't know why this result comes...
How can I handle this problem?

So, the problem lies in your while loop:
while n < len(self.box):
if self.box._data[n] == company:
"""every even index(n = 0, 2, 4, ...) refers to company name"""
"""every odd index(n = 1, 3, 5, ...), which is next to even index refers to number of buying"""
self.box._data[n + 1] += number
n += 2
elif self.box._data[n] != company:
""" if there's no overlapping, then just put the elements """
self.box.push(company)
self.box.push(number)
n += 2
Imagine we've run:
P = StockManager("A","B")
P.buy_shares("hyundai", 20, 100)
P.buy_shares("hyundai", 20, 100)
P.buy_shares("hyundai", 20, 100)
And so far, everything has worked as planned. Now, you try:
P.buy_shares("lg", 20, 100)
So: n == 0 and len(self.box) == 2, so we enter the loop.
The if condition fails, self.box._data[n] != 'lg', so we go to the elif block:
self.box.push(company)
self.box.push(number)
n += 2
Now, box == ['hyundai', 60, 'lg', 20] and n == 2. But wait! The loop condition is n < len(box), but len(box) == 4, so we enter the while-loop again! This time, the if conditions passes this time since self.box._data[2] == 'lg'.
So you execute the body of the if-block:
self.box._data[n + 1] += number
n += 2
Now, n == 4 and box == ['hyundai', 60, 'lg', 40]. This time, 4 < len(box) fails and we exit the loop and your function ends.
Now, a stack seems like the wrong data structure to me. It's clunky to have to increase the size by two. If you really wanted to do this, I would suggest an approach like
offset = 0
i = -2
for i in range(0, len(self.box), 2):
if self.box._data[n] == company:
break
else:
offset = 2
self.box.push(company)
self.box.push(0)
self.box._data[i + offset + 1] += number
But again, this is clunky. You would be better off using a dict for box, then the logic could be implemented simply as:
self.box[company] = self.box.get(company, 0) + 1
Or better yet, use a defaultdict
>>> from collections import defaultdict
>>> box = defaultdict(int)
>>> box
defaultdict(<class 'int'>, {})
>>> box['lg'] += 20
>>> box
defaultdict(<class 'int'>, {'lg': 20})
>>> box['lg'] += 20
>>> box
defaultdict(<class 'int'>, {'lg': 40})
>>> box['lg']
40

You are iterating over an entire list using indices; if the first company/number doesn't match the stock you are buying, it is automatically appended to the end of the list; then the next set of indices will point to the new company that was just appended so it adds the number. The elif is incorrect - you shouldn't append a new company/number pair till you have iterated through the whole list and didn't find it.
This is what you are doing
>>> a = ['a',1]
>>> n = 0
>>> comp, nbr = 'b', 2
>>> while n < len(a):
.... if a[n] == comp:
print('comp found at', n)
a[n+1] += nbr
n += 2
else:
print('n =', n, 'appending', comp)
a.append(comp)
a.append(nbr)
n += 2
n = 0 appending b
comp found at 2
>>>
This is what you could do to fix it:
>>> a = ['a',1]
>>> n = 0
>>> comp, nbr = 'b', 2
>>> while n < len(a):
found = False
if a[n] == comp:
print('comp found at', n)
a[n+1] += nbr
found = True
n += 2
>>> if not found:
a.append(comp)
a.append(nbr)
>>> a
['a', 1, 'b', 2]
>>>
But as suggested in the comments, you would probably be better off storing your purchase in a dictionary.