I have a 2d numpy object array : A and a numpy array of values : l (nx2 array)
Every element in the 2d numpy object array has two values like [ax1, ay1] and similarly for l.
Im calling my own distance function and calculating the distance between the first element of l with every element in A and then taking the minimum distance. The loop looks like this:
for r in range(A.shape[0]):
for s in range(A.shape[1]):
lencent = l.shape[0]
dist = []
for p in range(lencent):
dist.append(distancefunction(A[r,s],A[r,s],l[p,0],l[p,1]))
#print dist
val = np.min(dist)
#print val
tempimg[r,s] = val
But this take huge amount of time when the numpy array A is large, or l is large, or both. Is there any pythonic way to optimize the performance of this loop?
Related
I've been looking around but I have still faced problems with understanding the error in my code. I am trying to get get_pos values listed in pos_list array of size (numstep, N, 3). The values for get_pos is clearly checking as numstep,N,3 but when I call for pos_list at the bottom, it only prints out the last value. Is there anything I've missed?
def get_pos(p_list, N):
sys_position = np.zeros([N,3])
for i in range(N):
sys_position[i] = p_list[i].pos
return sys_position
pos_list = np.zeros([numstep, N, 3])
for num in range(numstep):
# Update particle position
update_pos(p_list, dt, N)
# Update force
force_new = get_force(p_list, dt, N)
# Update particle velocity by averaging
# current and new forces
update_vel(p_list, 0.5*(force+force_new), dt, N)
# Re-define force value
force = force_new
# Increase time
time += dt
print(get_pos(p_list,N))
# Append information to data lists
pos_list = get_pos(p_list, N)
print(pos_list)
ANy help would be appreciated I've been staring at this for far too long.
EDIT: attaching get_pos; function to list down array of position. Calling get_pos gets me the array of position at time t in the range of (0, numstep).
From the looks of it, pos_list is getting overwritten by whatever the get_pos function is returning and therefor is no longer numpy array after the first iteration of the loop.
EDIT:
pos_list = get_pos(p_list, N) overwrites the entire pos_list 3D numpy array with the 2D numpy array returned by get_pos(p_list, N).
To append the 2D numpy array from get_pos to the 3D array, at the specified index use pos_list[num, ...] = get_pos(p_list, N). The returned 2D array will be recorded on the row pointed by num provided the 2D array is the same shape as the 3D array excluding the first column of the 3D array.
In Python, I am trying to initialize 2-element arrays of zeros within a size N by N array. The code I'm using works but I'm looking for something more efficient and elegant:
array1 = np.empty((N,N), dtype=object)
for i in range(N):
for j in range(N):
array1[i,j] = np.zeros(2, dtype=np.int)
Thank ahead for the help
As I understand it, you should probably use a 3D array:
import numpy as np
array1 = np.empty((N,N,2), dtype=object)
which returns an array of N rows, N columns and 2 depth. If you want to pass a (NxN) array to let's say the first depth, just use:
tmp = np.ones(N,N) #for instance
array1(:,:,0) = tmp
I have a numpy array consisting of
[1,3,8,6,0,2,4,5,9,7]
This array is a random array consisting of 10 numbers 0-9.
I also have a 2D numpy array, a 10X10 2D numpy array with numerical values.
I would like to use my 1D numpy array (above) to access specific instances in my 2D numpy array, by looping through the 1D array
Loop 1: takes in 1 and 3, and finds the value at [1:3] in my 2D numpy array.
Loop 2: takes in 3 and 8, and finds the value at [3:8] in my 2D numpy array.
.
Loop 10: takes in 7 and 1, and finds the value at [7:1] in my 2D numpy array.
I would like to add up these values in my 2D numpy array.
so far I have :
array=[1,3,8,6,0,2,4,5,9,7]
values =0
for i in range (0, len(array)): #this is 10
a=array2[i,array[i]+1] #array2 is the 2D numpy array with the values
values=values+a
This works to some degree but how to I get it to access the last element to the first? i.e. find [7,1]
You can use simple slicing to make this work.
arr = np.random.randint(0, 10, (10,10))
pos = np.array([1,3,8,6,0,2,4,5,9,7])
pos = np.append(pos, pos[0])
rows = pos[0:-1]
cols = pos[1:]
result = sum(arr[rows, cols])
You can do the slicing twice to make it work.
values = 0
for i in range(len(array)):
a = Matrix[array[i],array[i+1]]
values += a
Also, the array you put has 11 elements which means the 10-th loop will not be what you intended.
I'm not sure I fully understood what you were trying to achieve but...
What about something like this?
a = np.array([1,3,8,6,0,9,2,4,5,9,7])
b = np.array(range(100)).reshape(10,10)
for i in range (len(a)):
print (a[i%len(a)],a[(i+1)%len(a)])
print (b[a[i%len(a)],a[(i+1)%len(a)]])
I removed 10 from the a array to avoid an index out of range error.
I also took the value [x,y] (and not the range [x:y] from the 2D array.
In the following code we calculate magnitudes of vectors between all pairs of given points. To speed up this operation in NumPy we can use broadcasting
import numpy as np
points = np.random.rand(10,3)
pair_vectors = points[:,np.newaxis,:] - points[np.newaxis,:,:]
pair_dists = np.linalg.norm(pair_vectors,axis=2).shape
or outer product iteration
it = np.nditer([points,points,None], flags=['external_loop'], op_axes=[[0,-1,1],[-1,0,1],None])
for a,b,c in it:
c[...] = b - a
pair_vectors = it.operands[2]
pair_dists = np.linalg.norm(pair_vectors,axis=2)
My question is how could one use broadcasting or outer product iteration to create an array with the form 10x10x6 where the last axis contains the coordinates of both points in a pair (extension). And in a related way, is it possible to calculate pair distances using broadcasting or outer product iteration directly, i.e. produce a matrix of form 10x10 without first calculating the difference vectors (reduction).
To clarify, the following code creates the desired matrices using slow looping.
pair_coords = np.zeros(10,10,6)
pair_dists = np.zeros(10,10)
for i in range(10):
for j in range(10):
pair_coords[i,j,0:3] = points[i,:]
pair_coords[i,j,3:6] = points[j,:]
pair_dists[i,j] = np.linalg.norm(points[i,:]-points[j,:])
This is a failed attempt to calculate distanced (or apply any other function that takes 6 coordinates of both points in a pair and produce a scalar) using outer product iteration.
res = np.zeros((10,10))
it = np.nditer([points,points,res], flags=['reduce_ok','external_loop'], op_axes=[[0,-1,1],[-1,0,1],None])
for a,b,c in it: c[...] = np.linalg.norm(b-a)
pair_dists = it.operands[2]
Here's an approach to produce those arrays in vectorized ways -
from itertools import product
from scipy.spatial.distance import pdist, squareform
N = points.shape[0]
# Get indices for selecting rows off points array and stacking them
idx = np.array(list(product(range(N),repeat=2)))
p_coords = np.column_stack((points[idx[:,0]],points[idx[:,1]])).reshape(N,N,6)
# Get the distances for upper triangular elements.
# Then create a symmetric one for the final dists array.
p_dists = squareform(pdist(points))
Few other vectorized approaches are discussed in this post, so have a look there too!
I have a 2D numpy array and a list of lists of indices for which I wish to compute the sum of the corresponding 1D vectors from the numpy array. This can be easily done through a for loop or via list comprehension, but I wonder if it's possible to vectorize it. With similar code I gain about 40x speedups from the vectorization.
Here's sample code:
import numpy as np
indices = [[1,2],[1,3],[2,0,3],[1]]
array_2d = np.array([[0.5, 1.5],[1.5,2.5],[2.5,3.5],[3.5,4.5]])
soln = [np.sum(array_2d[x], axis=-1) for x in indices]
(edit): Note that the indices are not (x,y) coordinates for array_2d, instead indices[0] = [1,2] represents the first and second vectors (rows) in array_2d. The number of elements of each list in indices can be variable.
This is what I would hope to be able to do:
vectorized_soln = np.sum(array_2d[indices[:]], axis=-1)
Does anybody know if there are any ways of achieving this?
First to all, I think you have a typo in the third element of indices...
The easy way to do that is building a sub_array with two arrays of indices:
i = np.array([1,1,2])
j = np.array([2,3,?])
sub_arr2d = array_2d[i,j]
and finally, you can take the sum of sub_arr2d...