Develop Reference

Creating a list of particle positions for each gridcell

I am creating a charge smear function. I have a matrix were each row is a particle with a charge and position. I then look at each particles position in a grid, to count how many particles are in each grid-cell, but I need to know which cell each particle is in, so that I may find the average of the positions for every particle in a specific grid-cell. My idea for a fix is to create an list where the number of rows is the amount of grid-cells in my matrix, and let the column be positions in x,y and z direction, but obviously I can't append more then one number to each index, but maybe some variation will work? Sorry for open ended question. Thank you in advance
import matplotlib.pyplot as plt
import random
import numpy as np
###Initalize particle lists
particle_arrayX=[]
particle_arrayY=[]
###The resolution
N = 10
###Number of particles
M = 1000
col=None
row=None
###Size of box
Box_size=100
###gridsize
Grid_size=Box_size/N
###Initalize particles
for i in range(M):
particle_arrayX.append(random.random()*Box_size)
particle_arrayY.append(random.random()*Box_size)
###Intialize matrix
ParticleMatrix_Inital=[[0 for i in range(N)]]*N
###Measure density in each cell
for i in range(M):
col=None
row=None
#The x and y components are diveded by the gridsize
#Then they are converted to integers and then asigned either to a row or column
#If value is float with decimal 0 EX 2.0, then 1 is substracted before converted to int
coln=particle_arrayX[i]/Grid_size
rown=particle_arrayY[i]/Grid_size
if coln.is_integer()==True:
col=int(coln)-1
else:
col=int(coln)
if rown.is_integer()==True:
row=int(rown)-1
else:
row=int(rown)
ParticleMatrix_Inital=np.array(ParticleMatrix_Inital)
ParticleMatrix_Inital[row,col]=ParticleMatrix_Inital[row,col]+1
ParticleMatrix_Inital=ParticleMatrix_Inital.tolist()
#Plot matrix
plt.imshow(ParticleMatrix_Inital)
plt.colorbar()
plt.show()

Welcome to SO!
There are many ways to approach the problem of "bin-ing" empirical data. I'm proposing an object oriented (OO) solution below, because (in my subjective opinion) it provides clean, tidy and highly readable code. On the other hand, OO-solutions might not be the most efficient if you're simulating huge many-particles systems. If the below code doesn't entirely solve your issues, I still hope that parts of it can be of some help to you.
That being said, I propose implementing your grid as a class. To make life easier for our self, we may apply the convention that all particles have positive coordinates. That is x, y and even z (if introduced) stretches from 0 to whatever box_size you define. However, the class Grid does not really care about the actual box_size, only the resolution of the grid!
class Grid:
def __init__(self, _delta_x, _delta_y):
self.delta_x = _delta_x
self.delta_y = _delta_y
def bounding_cell(self, x, y):
# Note that y-coordinates corresponds to matrix rows,
# and that x-coordinates corresponds to matrix columns
return (int(y/self.delta_y), int(x/self.delta_x))
Yes, this could have been a simple function. However, as a class it is easily expandable. Also, a function would have rely on global variables (yuck!) or explicitly be given the grid spanning (delta) in each dimensional direction, for every determining of which matrix cell (or bin) the given coordinate (x,y) belongs to.
Now, how does it work? Imagine the simplest of cases, where your grid resolution is 1. Then, a particle at position (x,y) = (1.2, 4,9) should be placed in the matrix at (row,col) = (4,1). That is row = int(y/delta_y) and likewise for x. The higher resolution (smaller delta) you have, the larger the matrix gets in terms of number of rows and cols.
Now that we have a Grid, let us also object orient the Particle! Rather straight forward:
class Particle:
def __init__(self, _id, _charge, _pos_x, _pos_y):
self.id = _id
self.charge = _charge
self.pos_x = _pos_x
self.pos_y = _pos_y
def __str__(self):
# implementing the __str__ method let's us 'print(a_particle)'
return "{%d, %d, %3.1f, %3.1f}" % (self.id, self.charge, self.pos_x, self.pos_y)
def get_position(self):
return self.pos_x, self.pos_y
def get_charge(self):
return self.charge
This class is more or less just a collection of data, and could easily have been replaced by a dict. However, the class screams its intent clearly, it is clean and tidy, and also easily expanded.
Now, let's create some instances of particles! Here is a function which by list comprehension creates a list of particles with an id, charge and position (x,y):
import random
def create_n_particles(n_particles, max_pos):
return [Particle(id, # unique ID
random.randint(-1,1), # charge [-1, 0, 1]
random.random()*max_pos, # x coord
random.random()*max_pos) # y coord
for id in range(n_particles)]
And finally, we get to the fun part: putting it all together:
import numpy as np
if __name__ == "__main__":
n_particles = 1000
box_size = 100
grid_resolution = 10
grid_size = int(box_size / grid_resolution)
grid = Grid(grid_resolution, grid_resolution)
particles = create_n_particles(n_particles, box_size)
charge_matrix = np.zeros((grid_size, grid_size))
id_matrix = [[ [] for i in range(grid_size)] for j in range(grid_size)]
for particle in particles:
x, y = particle.get_position()
row, col = grid.bounding_cell(x, y)
charge_matrix[row][col] += particle.get_charge()
# The ID-matrix is similar to the charge-matrix,
# but every cell contains a new list of particle IDs
id_matrix[row][col].append(particle.id)
Notice the initialization of the ID-matrix: This is the list of particle positions for each grid cell that you asked for. It is a matrix, representing the particle container, and each cell contains a list to be filled with particle IDs. You could also populate these lists with entire particle instances (not just their IDs): id_matrix[row][col].append(particle).
The last for loop does the real work, and here the Object Oriented strategy shows us how charming it is: The loop is short and it is very easy to read and understand what is going on: A cell in the charge_matrix contains the total charge within this grid cell/bin. Meanwhile, the id_matrix is filled with the IDs of the particles that is contained within this grid cell/bin.
From the way we've constructed the list of particles, particles, we see that a particle's ID is equivalent to that particle's index in the list. Hence, they may be retrieved like this,
for i,row in enumerate(id_matrix):
for j,col in enumerate(row):
print("[%d][%d] : " % (i, j), end="")
for particle_id in id_matrix[i][j]:
p = particles[particle_id]
# do some work with 'p', or just print it:
print(p, end=", ")
print("") # print new line
# Output:
# [0][0] : {32, -1, 0.2, 0.4}, ... <-- all data of that particle
# ....
I leave optimization of this retrieval to you as I don't really know what data you need and what you're going to do with it. Maybe it's better to contain all the particles in a dict instead of a list; I don't know(?). You choose!
At the very end, I'll suggest that you use matshow which is inteded for displaying matrices, as opposed to imshow which is more aiming more for images.
import matplotlib.pyplot as plt
fig = plt.figure()
ax = fig.add_subplot(111)
cax = ax.matshow(charge_matrix)
fig.colorbar(cax)
ax.invert_yaxis() # flip the matrix such that the y-axis points upwards
fig.savefig("charge_matrix.png")
We can also scatter plot the particles and add grid lines corresponding to our the grid in the matshow above. We color the scatter plots such that negative charges are blue, neutral are gray and positive are red.
def charge_color(charge):
if charge > 0: return 'red'
elif charge < 0: return 'blue'
else: return 'gray'
fig = plt.figure()
ax = fig.add_subplot(111)
ax.set_aspect('equal')
ax.set_xticks(np.arange(0, 101, grid_resolution))
ax.set_yticks(np.arange(0, 101, grid_resolution))
ax.grid()
ax.scatter([p.pos_x for p in particles],
[p.pos_y for p in particles],
c=[charge_color(p.get_charge()) for p in particles])
fig.savefig("particle_dist.png")

Variable gets updated out of a class but it should not python

I'm finally a member of the stackoverflow community, so this is my first post. I'm trying my best to create a good question.
I have a problem with a simple variable declaration in a for-loop. The variable gets declared in an if-statement and is the variable of a class. Basically what the code does is to create a box with points in it, initialise the positions and velocities of the points, edit the velocities and calculate the new positions according to their new velocities. I now want to save the initial positions of the points (means time = 0) outside of the class in a for-loop. I do this by if time = 0 then save position to a variable, but the variable gets updated to the new position in every loop iteration. The actual code is about hydrodynamics and particle interaction, but the basic structure of the code is something like this:
import numpy as np
class box():
def __init__(self, boxsize, num_points, timestep):
"""
boxsize is the size of the quadratic box in x and y direction
num_points is the number of points in the box
timestep is the time after that the positions should be updated
"""
self.boxsize = float(boxsize)
self.num_points = int(num_points)
self.timestep = float(timestep)
self.positions = np.zeros((self.num_points, 2)).astype(float)
self.velocities = np.zeros((self.num_points, 2)).astype(float)
def initialise(self):
"""initialise the positions and velocites of the points in the box, both with x- and y-components"""
self.positions[:, :] = np.random.uniform(0., self.boxsize, size=(self.num_points, 2))
self.velocities[:, :] = np.random.uniform(0., 1., size=(self.num_points, 2))
def update_positions(self):
"""update position according to velocities, x- and y-components"""
self.positions += self.velocities*self.timestep
def new_velocities(self):
""" create new velocities, x- and y-components """
self.velocities[:, :] = np.random.uniform(0., 1., size=(self.num_points, 2))
def connect_steps(self):
"""update the positions according to their velocities and create new velocities"""
self.update_positions()
self.new_velocities()
system = box(10., 1, 0.1) #box is 10 in x and y; 1 point in the box
system.initialise() #initialise the positions and velocities of the box
for i in range(10): #10 timesteps
system.connect_steps()
if i == 0.:
r0 = system.positions
print(r0) #here r0 should always be the same array from i = 0 but isn't
print(r0 == system.positions) #yields True every iteration, so r0 is always the new position
What I want is that r0 is always the position at i = 0 (initial position), but every iteration the variable r0 gets updated according to its new position, although the if-clause and so the variable definition only gets entered once at i = 0.
It is intended to first update the positions and after that generate new velocities although they are first used in the next iteration because the real algorithm behind this velocity-generation needs the structure this way.
Maybe there is just a characteristic or property of classes I don't know.
I hope the question makes sense and anybody can help me out.
Thanks a lot!

You might try np.copy(system.positions) to get a copy that won't continue to mutate as you update.
Reference https://numpy.org/doc/stable/reference/generated/numpy.copy.html

Continue program after plt.show() command without closing all windows

I know that there are plenty of similar topics in StackOverflow but none of the answers solved my specific problem.
First of all, I am trying plot 2-dimensional data points from different classes with scatter command to the figure. The program uses matplotlib events like button_press_event and motion_notify_event and I assume that right after event commands plt.show() should be used. The main problem is that once some tasks are done with events (i.e. labeling), I want to update the whole figure but plt.show() is blocking the option to continue the program. Actually plt.show() should block the program until the user of program decides to go forward. Is there any solution to control this blocking attribute?
I have tried plt.ion(), plt.ioff(), plt.show(block=False), plt.draw(), plt.pause(0.001), global variables with while loop etc. without success. The only way the program works somehow correctly is when all the figures are closed inside of button_press_event when so called forward condition is met but it is not very user-friendly solution if all figures are closed every time data points are updated.
Here is the the glimpse of the code:
def draw_original_figure(random_sample, random_sample_label, random_sample_image, X_train_chosen, y_train_chosen, images_train_chosen, classes, accuracy, embedding, model, h=0.05):
global points1, im1, s_im1, xybox, x1, y1, fig1, classes1, points_to_plot, y_train_chosen1, accuracy1, random_sample1, embedding1, y_train_chosen1, h1, random_sample_label1, result1
fig1 = plt.gcf()
.
.
.
original_figure_plot()
fig1.canvas.mpl_connect('motion_notify_event', hover)
fig1.canvas.mpl_connect('button_press_event', click)
plt.show()
def hover(event):
# if the mouse is over the scatter points
if points1.contains(event)[0]:
# find out the index within the array from the event
inds = points1.contains(event)[1]["ind"]
ind = inds[0]
# get the figure size
w,h = fig1.get_size_inches()*fig1.dpi
ws = (event.x > w/2.)*-1 + (event.x <= w/2.)
hs = (event.y > h/2.)*-1 + (event.y <= h/2.)
# if event occurs in the top or right quadrant of the figure,
# change the annotation box position relative to mouse.
ab1.xybox = (xybox[0]*ws, xybox[1]*hs)
# make annotation box visible
ab1.set_visible(True)
# place it at the position of the hovered scatter point
ab1.xy =(x1[ind], y1[ind])
# set the image corresponding to that point
im1.set_data(s_im1[ind,:,:])
else:
#if the mouse is not over a scatter point
ab1.set_visible(False)
fig1.canvas.draw_idle()
def click(event):
# if the mouse is over the scatter points
if points1.contains(event)[0]:
# find out the index within the array from the event
inds = points1.contains(event)[1]["ind"]
ind = inds[0]
# if one specific point is chosen
if ind == len(x1)-1:
plt.scatter(x1[ind], y1[ind], s=25, marker='x', c='#556B2F')
q = question(True, ind)
# do nothing
if q == "":
original_figure_plot()
# quit the program
elif q == "q":
exit()
# continue the program without updating labels
elif q == "n":
result1 = copy.deepcopy(y_train_chosen1)
plt.close("all")
# continue the program after labels are updated
else:
result1 = copy.deepcopy(y_train_chosen1)
result1 = np.append(result1, [int(q)], axis=0)
plt.close("all")
else:
# if anyone else point is chosen
plt.scatter(x1[ind], y1[ind], s=8, c='k')
q = question(False, ind)
# do nothing
if q == "":
original_figure_plot()
# quit the program
elif q == "q":
exit()
# update labels
else:
y_train_chosen1[ind] = int(q)
original_figure_plot()
fig1.canvas.draw_idle()
Probably it is better to use for example other libraries like plotly or dash but is it really true that you cannot update figure without closing it if you are using matplotlib events?? I can provide all the project files but I think so that if there is a solution, it should be done inside of these functions.

It took the whole day to find the answer but here it is!
I use now plt.show() in interactive-mode with command plt.ion() and do blocking manually with commands fig.canvas.start_event_loop() and fig.canvas.stop_event_loop(). To be honest, it was surprisingly difficult to find the solution to this problem but the lesson is learned.
matplotlib figure does not continue program flow after close event triggered inside tk app

Python - optimizing plot code

I am working on a live plot.I am getting data from a spectrum analyser which gives me the value at a certain frequency. But the program becomes slower the longer it runs.
So I hope you have some ideas. I also looked at my activity monitor while running and the RAM isn't full at all.
I tried to comment out ctf = ax.contourf( a, b, B, cmap=cma) which is responsible for plotting and if it don't need to draw it is so fast. But I need the plot so not drawing is not a solution at all.
And ax = plt.subplot( 111, polar = True) for extra information.
Here is my code:
while True :
trace = inst.query(':TRACe:DATA? TRACE1').partition(' ')[2][:-2].split(', ')# the first & last 2 entries are cut off, are random numbers
for value in trace : #write to file
f.write(value)
f.write('\n')
try : #looking if data is alright
trace = np.array(trace, np.float)
except ValueError: #if a ValueError is raised this message is displayed but the loop won't break and the piece is plotted in one color (green)
print'Some wrong data at the', i+1, 'th measurement'
longzeroarray = np.zeros(801)
a = np.linspace(i*np.pi/8-np.pi/16, i*np.pi/8+np.pi/16, 2)#Angle, circle is divided into 16 pieces
b = np.linspace(start -scaleplot, stop,801) #points of the frequency + 200 more points to gain the inner circle
A, B = np.meshgrid(a, longzeroarray)
cma = ListedColormap(['w'])
#actual plotting
ctf = ax.contourf( a, b, B, cmap=cma)
xCooPoint = i*np.pi/8 + np.pi/16 #shows the user the position of the plot
yCooPoint = stop
ax.plot(xCooPoint, yCooPoint, 'or', markersize = 15)
xCooWhitePoint = (i-1) * np.pi/8 + np.pi/16 #this erases the old red points
yCooWhitePoint = stop
ax.plot(xCooWhitePoint, yCooWhitePoint, 'ow', markersize = 15)
plt.draw()
time.sleep(60) #delaying the time to give analyser time to give us new correct data in the next step
i +=1
continue
maximasearch(trace,searchrange)
trace = np.insert(trace,0,zeroarray)
a = np.linspace(i*np.pi/8+np.pi/16-np.pi/8, i*np.pi/8+np.pi/16, 2)#Angle, circle is divided into 16 pieces
b = np.linspace(start -scaleplot, stop,801) #points of the frequency + 200 more points to gain the inner circle
A, B = np.meshgrid(a, trace)
#actual plotting
ctf = ax.contourf(a, b, B, cmap=cm.jet, vmin=-100, vmax=100)
xCooPoint = i*np.pi/8 + np.pi/16 #shows the user the position of the plot
yCooPoint = stop
ax.plot(xCooPoint, yCooPoint, 'or', markersize = 15)
xCooWhitePoint = (i-1) * np.pi/8 + np.pi/16 #this erases the old red points
yCooWhitePoint = stop
ax.plot(xCooWhitePoint, yCooWhitePoint, 'ow', markersize = 15)
plt.draw()
i+=1
Thats how the plot looks like, and with every new step a new piece of the circle is drawn.
EDIT
I found following question here on stack overflow: real-time plotting in while loop with matplotlib
I think the answer with 22 Upvotes could be helpful. Has anyone ever used blit ? I have no idea yet how to combine it with my code.
http://wiki.scipy.org/Cookbook/Matplotlib/Animations

I want to answer my own question again.
The best way to optimize the code is to calculate with modulo 2*pi for the radial values.
I changed my code a bit :
a = np.linspace((i*np.pi/8+np.pi/16-np.pi/8)%(np.pi*2), (i*np.pi/8+np.pi/16)%(np.pi*2), 2)
The problem before was that Python also plotted all the old pieces, because obviously the were still there but only under a layer of newly plotted data pieces. So although you didn't see the old plotted data, it was still drawn. Now only the circle from 0 to 2pi is redrawn.

Python: short coordinate function unexpectedly changes incoming argument value

The goal here is to take some list of coordinates, like [[1,2],[3,4],[7,1]], then figure out how big the canvas should be if you want to print all these coords. Take the maximal bottom left coordinate and minimal upper right coordinate that will snugly fit a canvas to these points. In the above list, for example, we're looking for [[1,1],[7,4]], which defines the smallest rectangle where all those points will fit.
In the middle of this function, I'm seeing the incoming "board" assigned a new value.
def print_board(board):
# import pdb; pdb.set_trace()
dimensions = None
for i in board:
if dimensions == None:
dimensions = [i, i]
else:
dimensions[0][0] = min(dimensions[0][0], i[0])
#'board' is redefined !!!
dimensions[0][1] = min(dimensions[0][1], i[1])
#dimensions[1][0] = max(dimensions[1][0], i[0])
#dimensions[1][1] = max(dimensions[1][1], i[1])
# (after we get the canvas size
# we print the canvas with the points on it
# but we never make it that far without an error)
As the for loop moves through the coordinates in the incoming board, it seems to be setting board[0] to whatever coordinate it's looking at at the time. So [[1,2],[3,4],[7,1]] will change first to [[3,4],[3,4],[7,1]], then to [[7,1],[3,4],[7,1]].
I wouldn't expect board to change at all.
(Python 3.2.2)

When you do
dimensions = [i, i]
you're setting both items in dimensions to the first point in your board -- not making copies of that point.
Then when you do
dimensions[0][0] = min(dimensions[0][0], i[0])
dimensions[0][1] = min(dimensions[0][1], i[1])
you're updating that same point --- the first point in your board -- to the results of the min functions.
Try something like this, instead:
def print_board(board):
xs, ys = zip(*board) # separate out the x and y coordinates
min_x, max_x = min(xs), max(xs) # find the mins and maxs
min_y, max_y = min(ys), max(ys)
dimensions = [[min_x, min_y], [max_x, max_y]] # make the dimensions array

As an extension of agfs answer, you can use numpy for even more efficient and succinct code:
import numpy as np
def print_board(board):
a = np.array(board)
return [a.min(axis=0).tolist(), a.max(axis=0).tolist()]
If your board is a numpy array already, and you let the function return a tuple of numpy arrays, it shortens even more:
def print_board(board):
return board.min(axis=0), board.max(axis=0)