Suppose I am working with numpy in Python and I have a two-dimensional array of arbitrary size. For convenience, let's say I have a 5 x 5 array. The specific numbers are not particularly important to my question; they're just an example.
a = numpy.arrange(25).reshape(5,5)
This yields:
[[0, 1, 2, 3, 4 ],
[5, 6, 7, 8, 9 ],
[10,11,12,13,14],
[15,16,17,18,19],
[20,21,22,23,24]]
Now, let's say I wanted to take a 2D slice of this array. In normal conditions, this would be easy. To get the cells immediately adjacent to 2,2 I would simply use a[1:4,1,4] which would yield the expected
[[6, 7, 8 ],
[11, 12, 13],
[16, 17, 18]]
But what if I want to take a slice that wraps
around the edges of the array? For example a[-1:2,-1:2] would yield:
[24, 20, 21],
[4, 0, 1 ],
[9, 5, 6 ]
This would be useful in several situations where the edges don't matter, for example game graphics that wrap around a screen. I realize this can be done with a lot of if statements and bounds-checking, but I was wondering if there was a cleaner, more idiomatic way to accomplish this.
Looking around, I have found several answers such as this: https://stackoverflow.com/questions/17739543/wrapping-around-slices-in-python-numpy that work for 1-dimensional arrays, but I have yet to figure out how to apply this logic to a 2D slice.
So essentially, the question is: how do I take a 2D slice of a 2D array in numpy that wraps around the edges of the array?
Thank you in advance to anyone who can help.
This will work with numpy >= 1.7.
a = np.arange(25).reshape(5,5)
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
The pad routine has a 'wrap' method...
b = np.pad(a, 1, mode='wrap')
array([[24, 20, 21, 22, 23, 24, 20],
[ 4, 0, 1, 2, 3, 4, 0],
[ 9, 5, 6, 7, 8, 9, 5],
[14, 10, 11, 12, 13, 14, 10],
[19, 15, 16, 17, 18, 19, 15],
[24, 20, 21, 22, 23, 24, 20],
[ 4, 0, 1, 2, 3, 4, 0]])
Depending on the situation you may have to add 1 to each term of any slice in order to account for the padding around b.
After playing around with various methods for a while, I just came to a fairly simple solution that works using ndarray.take. Using the example I provided in the question:
a.take(range(-1,2),mode='wrap', axis=0).take(range(-1,2),mode='wrap',axis=1)
Provides the desired output of
[[24 20 21]
[4 0 1]
[9 5 6]]
It turns out to be a lot simpler than I thought it would be. This solution also works if you reverse the two axes.
This is similar to the previous answers I've seen using take, but I haven't seen anyone explain how it'd be used with a 2D array before, so I'm posting this in the hopes it helps someone with the same question in the future.
You can also use roll, to roll the array and then take your slice:
b = np.roll(np.roll(a, 1, axis=0), 1, axis=1)[:3,:3]
gives
array([[24, 20, 21],
[ 4, 0, 1],
[ 9, 5, 6]])
I had a similar challenge working with wrap-around indexing, only in my case I needed to set values in the original matrix. I've solved this by 'fancy indexing' and making use of meshgrid function:
A = arange(25).reshape((5,5)) # destinatoin matrix
print 'A:\n',A
k =-1* np.arange(9).reshape(3,3)# test kernel, all negative
print 'Kernel:\n', k
ix,iy = np.meshgrid(arange(3),arange(3)) # create x and y basis indices
pos = (0,-1) # insertion position
# create insertion indices
x = (ix+pos[0]) % A.shape[0]
y = (iy+pos[1]) % A.shape[1]
A[x,y] = k # set values
print 'Result:\n',A
The output:
A:
[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[10 11 12 13 14]
[15 16 17 18 19]
[20 21 22 23 24]]
Kernel:
[[ 0 -1 -2]
[-3 -4 -5]
[-6 -7 -8]]
Result:
[[-3 -6 2 3 0]
[-4 -7 7 8 -1]
[-5 -8 12 13 -2]
[15 16 17 18 19]
[20 21 22 23 24]]
As I mentioned in the comments, there is a good answer at How do I select a window from a numpy array with periodic boundary conditions?
Here is another simple way to do this
# First some setup
import numpy as np
A = np.arange(25).reshape((5, 5))
m, n = A.shape
and then
A[np.arange(i-1, i+2)%m].reshape((3, -1))[:,np.arange(j-1, j+2)%n]
It is somewhat harder to obtain something that you can assign to.
Here is a somewhat slower version.
In order to get a similar slice of values I would have to do
A.flat[np.array([np.arange(j-1,j+2)%n+a*n for a in xrange(i-1, i+2)]).ravel()].reshape((3,3))
In order to assign to this I would have to avoid the call to reshape and work directly with the flattened version returned by the fancy indexing.
Here is an example:
n = 7
A = np.zeros((n, n))
for i in xrange(n-2, 0, -1):
A.flat[np.array([np.arange(i-1,i+2)%n+a*n for a in xrange(i-1, i+2)]).ravel()] = i+1
print A
which returns
[[ 2. 2. 2. 0. 0. 0. 0.]
[ 2. 2. 2. 3. 0. 0. 0.]
[ 2. 2. 2. 3. 4. 0. 0.]
[ 0. 3. 3. 3. 4. 5. 0.]
[ 0. 0. 4. 4. 4. 5. 6.]
[ 0. 0. 0. 5. 5. 5. 6.]
[ 0. 0. 0. 0. 6. 6. 6.]]
Related
I am trying to write a code that chooses one by one from a and b. I want to make a 2 dimensional array where the first index is either 0 or 1. 0 representing a and 1 representing b and the second index would just be the values in array a or b so it will be something like this [[0 7][1 13]]. I want the function to also have it in order so it will be The function starts off with a then it will be like a,b,a,b,a... if its the other way around b,a,b,a,b.... Comparing which index function comes before the other so since the first index of b is 0 and the first index of a is 7, since 0 < 7 the code will start off with b [[1 0]] and then it will go for the next index on 'a' which is 7 so the [[1 0],[0, 7]]. It will keep on doing this until it reaches the end of the array a and b. How can I get the expected output below?
import numpy as np
a = np.array([ 7, 9, 12, 15, 17, 22])
b = np.array([ 0, 13, 17, 18])
Expected Output:
[[ 1 0]
[ 0 7]
[ 1 13]
[ 0 15]
[ 1 17]
[ 0 17]
[ 1 18]
[ 0 22]]
You can combine the two arrays and sort the values while preserving the origin of each value (using 2N and 2N+1 offsetting).
Then filter out the consecutive odd/even values to only retain values with alternating origin indicator (1 or 0)
Finally, build the resulting array of [origin,value] pairs by reversing the 2N and 2N+1 tagging.
import numpy as np
a = np.array([ 7, 9, 12, 15, 17, 22])
b = np.array([ 0, 13, 17, 18])
p = 1 if a[0] > b[0] else 0 # determine first entry
c = np.sort(np.concatenate((a*2+p,b*2+1-p))) # combine/sort tagged values
c = np.concatenate((c[:1],c[1:][c[:-1]%2 != c[1:]%2])) # filter out same-array repeats
c = np.concatenate(((c[:,None]+p)%2,c[:,None]//2),axis=1) # build result
print(c)
[[ 1 0]
[ 0 7]
[ 1 13]
[ 0 15]
[ 1 17]
[ 0 17]
[ 1 18]
[ 0 22]]
This isn't a Numpy solution, but may work if you are okay processing these as lists. You can make iterators out of the lists, then alternate between them using itertools.dropwhile to proceed through the elements until you get the next in line. It might look something like:
from itertools import dropwhile
def pairs(a, b):
index = 0 if a[0] <= b[0] else 1
iters = [iter(a), iter(b)]
while True:
try:
current = next(iters[index])
yield [index,current]
index = int(not index)
except StopIteration:
break
iters[index] = dropwhile(lambda n: n < current, iters[index])
list(pairs(a, b))
Which results in:
[[1, 0], [0, 7], [1, 13], [0, 15], [1, 17], [0, 17], [1, 18], [0, 22]]
you can use conditions of which array element is from -> with sorted values -> including condition of ->group wise split and ->flip
c = np.hstack([np.vstack([a, np.zeros(len(a))]), np.vstack([b, np.ones(len(b))])]).T
c = c[c[:, 0].argsort()]
# Group wise split and flip array - 2nd possiblity
d = np.vstack(np.apply_along_axis(np.flip, 0, np.split(c, np.unique(c[:,0], return_index = True)[1])))[::-1]
res1 = np.vstack([d[0], d[1:][d[:,1][:-1]!=d[:,1][1:]]])
res2 = np.vstack([c[0], c[1:][c[:,1][:-1]!=c[:,1][1:]]])
if res1.shape[0]>res2.shape[0]:
print(res1)
else:
print(res2)
Out:
[[ 0. 1.]
[ 7. 0.]
[13. 1.]
[15. 0.]
[17. 1.]
[17. 0.]
[18. 1.]
[22. 0.]]
I am trying to insert data from a 1D array into a 2D array and still maintain the shape from the 2D array.
My code below reformats the 2D array into 1D. Also, why do I now have 26 indexes?
What am I missing?
import numpy as np
oneD_Array = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25])
twoD_Array = np.zeros((5, 5))
print(oneD_Array.shape)
print(oneD_Array)
print()
print()
print(twoD_Array.shape)
print(twoD_Array)
for i in range(len(oneD_Array), -1, -1):
# for subArray in twoD_Array:
twoD_Array = np.insert(oneD_Array, 0, [i])
print()
print(twoD_Array)
print(twoD_Array.shape)
The output is:
(25,)
[ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
25]
(5, 5)
[[0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0.]]
[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
24 25]
(26,)
You simply can use np.reshape:
twoD_Array = np.reshape(oneD_Array, (5, 5))
output:
array([[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25]])
because np.insert actually inserts an element to the array at the given index.
How about:
twoD_Array.ravel()[:] = oneD_Array
If you insist on using a loop, it could be written like this:
for i in range(len(oneD_Array)):
twoD_Array[i//twoD_Array.shape[1], i%twoD_Array.shape[1]] = oneD_Array[i]
But it's definitely not the fastest way.
On my machine, for a 500x500 array, my loop takes 85 ms, using ndarray.ravel takes 223 µs, using np.reshape takes 1.17 µs and using ndarray.reshape takes 357 ns. So I would go with
twoD_Array = oneD_Array.reshape((5, 5))
I'm trying to build a grid world using numpy.
The grid is 4*4 and laid out in a square.
The first and last squares (i.e. 1 and 16) are terminal squares.
At each time step you can move one step in any direction either: up, down , left or right.
Once you enter one of the terminal squares no further moves are possible and the game terminates.
The first and last columns are the left and right edges of the square whilst the first and last rows represent the top and bottom edges.
If you are on an edge, for example the left one and attempt to move left, instead of moving left you stay in the square you started in. Similarly you remain in the same square if you try and cross any of the other edges.
Although the grid is a square I've implemented it as an array.
States_r calculates the position of the states after a move right. 1 and 16 stay where they are because they are terminal states (note the code uses zero based counting so 1 and 16 are 0 and 15 respectively in the code).
The rest of the squares are in increased by one. The code for states_r works however those squares on the right edge i.e. (4, 8, 12) should also stay where they are but states_r code doesn't do that.
State_l is my attempt to include the edge condition for the left edge of the square. The logic is the same the terminal states (1, 16) should not move nor should those squares on the left edge (5, 9, 13). I think the general logic is correct but it's producing an error.
states = np.arange(16)
states_r = states[np.where((states + 1 <= 15) & (states != 0), states + 1, states)]
states_l = states[np.where((max(1, (states // 4) * 4) <= states - 1) & (states != 15), states - 1, states)]
The first example states_r works, it handles the terminal state but does not handle the edge condition.
The second example is my attempt to include the edge condition, however it is giving me the following error:
"The truth value of an array with more than one element is ambiguous."
Can someone please explain how to fix my code?
Or alternatively suggest another solution,ideally I want the code to be fast (so I can scale it up) so I want to avoid for loops if possible?
If I understood correctly you want arrays which indicate for each state where the next state is, depending on the move (right, left, up, down).
If so, I guess your implementation of state_r is not quit right. I would suggest to switch to a 2D representation of your grid, because a lot of the things you describe are easier and more intuitive to handle if you have x and y directly (at least for me).
import numpy as np
n = 4
states = np.arange(n*n).reshape(n, n)
states_r, states_l, states_u, states_d = (states.copy(), states.copy(),
states.copy(), states.copy())
states_r[:, :n-1] = states[:, 1:]
states_l[:, 1:] = states[:, :n-1]
states_u[1:, :] = states[:n-1, :]
states_d[:n-1, :] = states[1:, :]
# up [[ 0, 1, 2, 3],
# left state right [ 0, 1, 2, 3],
# down [ 4, 5, 6, 7],
# [ 8, 9, 10, 11]]
#
# [[ 0, 0, 1, 2], [[ 0, 1, 2, 3], [[ 1, 2, 3, 3],
# [ 4, 4, 5, 6], [ 4, 5, 6, 7], [ 5, 6, 7, 7],
# [ 8, 8, 9, 10], [ 8, 9, 10, 11], [ 9, 10, 11, 11],
# [12, 12, 13, 14]] [12, 13, 14, 15]] [13, 14, 15, 15]]
#
# [[ 4, 5, 6, 7],
# [ 8, 9, 10, 11],
# [12, 13, 14, 15],
# [12, 13, 14, 15]]
If you want to exclude the terminal states, you can do something like this:
terminal_states = np.zeros((n, n), dtype=bool)
terminal_states[0, 0] = True
terminal_states[-1, -1] = True
states_r[terminal_states] = states[terminal_states]
states_l[terminal_states] = states[terminal_states]
states_u[terminal_states] = states[terminal_states]
states_d[terminal_states] = states[terminal_states]
If you prefer the 1D approach:
import numpy as np
n = 4
states = np.arange(n*n)
valid_s = np.ones(n*n, dtype=bool)
valid_s[0] = False
valid_s[-1] = False
states_r = np.where(np.logical_and(valid_s, states % n < n-1), states+1, states)
states_l = np.where(np.logical_and(valid_s, states % n > 0), states-1, states)
states_u = np.where(np.logical_and(valid_s, states > n-1), states-n, states)
states_d = np.where(np.logical_and(valid_s, states < n**2-n), states+n, states)
Another way of doing it without preallocating arrays:
states = np.arange(16).reshape(4,4)
states_l = np.hstack((states[:,0][:,None],states[:,:-1],))
states_r = np.hstack((states[:,1:],states[:,-1][:,None]))
states_d = np.vstack((states[1:,:],states[-1,:]))
states_u = np.vstack((states[0,:],states[:-1,:]))
To get them all in 1-D, you can always flatten()/ravel()/reshape(-1) the 2-D arrays.
[[ 0 1 2 3]
[ 0 1 2 3]
[ 4 5 6 7]
[ 8 9 10 11]]
[[ 0 0 1 2] [[ 0 1 2 3] [[ 1 2 3 3]
[ 4 4 5 6] [ 4 5 6 7] [ 5 6 7 7]
[ 8 8 9 10] [ 8 9 10 11] [ 9 10 11 11]
[12 12 13 14]] [12 13 14 15]] [13 14 15 15]]
[[ 4 5 6 7]
[ 8 9 10 11]
[12 13 14 15]
[12 13 14 15]]
And for corners you can do:
states_u[-1,-1] = 15
states_l[-1,-1] = 15
I have an array and need to normalize it in a way that the results will be numbers between 0 and 1. I already normalized the entire array as follows:
C = A / A.max(axis=0)
print(C)
____________________________________________________________________
[[0. 0.05263158 0.1 0.14285714 0.18181818 0.2173913 ]
[0.33333333 0.36842105 0.4 0.42857143 0.45454545 0.47826087]
[0.66666667 0.68421053 0.7 0.71428571 0.72727273 0.73913043]
[1. 1. 1. 1. 1. 1. ]]
But now I have to normalize by column and by line. How can I do that with axis reduction? If there is a better way to what I did, suggest me alterations.
My expected result is two arrays with the values normalized. One considering the columns and the other by the lines
This is my data
A = [[ 0 1 2 3 4 5]
[ 6 7 8 9 10 11]
[12 13 14 15 16 17]
[18 19 20 21 22 23]]
You skip the minimum part. Normally a 0-1 normalization demanding subtracting a minimum value from denominator and numerator.
https://stats.stackexchange.com/questions/70801/how-to-normalize-data-to-0-1-range
import numpy as np
A = np.matrix([[ 0, 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10, 11],
[12, 13, 14, 15, 16, 17],
[18, 19, 20, 21, 22, 23]])
(A-A.min(axis=1))/(A.max(axis=1)-A.min(axis=1))
(A-A.min(axis=0))/(A.max(axis=0)-A.min(axis=0))
My expected result is two arrays with the values normalized. One considering the columns and the other by the lines
a = np.array([[ 0, 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10, 11],
[12, 13, 14, 15, 16, 17],
[18, 19, 20, 21, 22, 23]])
If
c = a / a.max(axis=0)
gives you what you want for the columns then
d = a / a.max(axis=1)[:,None]
will suffice for the rows.
>>> d.round(4)
array([[0. , 0.2 , 0.4 , 0.6 , 0.8 , 1. ],
[0.5455, 0.6364, 0.7273, 0.8182, 0.9091, 1. ],
[0.7059, 0.7647, 0.8235, 0.8824, 0.9412, 1. ],
[0.7826, 0.8261, 0.8696, 0.913 , 0.9565, 1. ]])
https://numpy.org/doc/stable/user/basics.broadcasting.html
I have the following code that iterates along the diagonals that are orthogonal to the diagonals normally returned by np.diagonal. It starts at position (0, 0) and works its way towards the lower right coordinate.
The code works as intended but is not very numpy with all its loops and inefficient in having to create many arrays to do the trick.
So I wonder if there is a nicer way to do this, because I don't see how I would stride my array or use the diagonal-methods of numpy to do it in a nicer way (though I expect there are some tricks I fail to see).
import numpy as np
A = np.zeros((4,5))
#Construct a distance array of same size that uses (0, 0) as origo
#and evaluates distances along first and second dimensions slightly
#differently so that no values in the array is the same
D = np.zeros(A.shape)
for i in range(D.shape[0]):
for j in range(D.shape[1]):
D[i, j] = i * (1 + 1.0 / (grid_shape[0] + 1)) + j
print D
#[[ 0. 1. 2. 3. 4. ]
# [ 1.05882353 2.05882353 3.05882353 4.05882353 5.05882353]
# [ 2.11764706 3.11764706 4.11764706 5.11764706 6.11764706]
# [ 3.17647059 4.17647059 5.17647059 6.17647059 7.17647059]]
#Make a flat sorted copy
rD = D.ravel().copy()
rD.sort()
#Just to show how it works, assigning incrementing values
#iterating along the 'orthagonal' diagonals starting at (0, 0) position
for i, v in enumerate(rD):
A[D == v] = i
print A
#[[ 0 1 3 6 10]
# [ 2 4 7 11 14]
# [ 5 8 12 15 17]
# [ 9 13 16 18 19]]
Edit
To clarify, I want to iterate element-wise through the entire A but doing so in the order the code above invokes (which is displayed in the final print).
It is not important which direction the iteration goes along the diagonals (if 1 and 2 switched placed, and 3 and 5 etc. in A) only that the diagonals are orthogonal to the main diagonal of A (the one produced by np.diag(A)).
The application/reason for this question is in my previous question (in the solution part at the bottom of that question): Constructing a 2D grid from potentially incomplete list of candidates
Here is a way that avoids Python for-loops.
First, let's look at our addition tables:
import numpy as np
grid_shape = (4,5)
N = np.prod(grid_shape)
y = np.add.outer(np.arange(grid_shape[0]),np.arange(grid_shape[1]))
print(y)
# [[0 1 2 3 4]
# [1 2 3 4 5]
# [2 3 4 5 6]
# [3 4 5 6 7]]
The key idea is that if we visit the sums in the addition table in order, we would be iterating through the array in the desired order.
We can find out the indices associated with that order using np.argsort:
idx = np.argsort(y.ravel())
print(idx)
# [ 0 1 5 2 6 10 3 7 11 15 4 8 12 16 9 13 17 14 18 19]
idx is golden. It is essentially everything you need to iterate through any 2D array of shape (4,5), since a 2D array is just a 1D array reshaped.
If your ultimate goal is to generate the array A that you show above at the end of your post, then you could use argsort again:
print(np.argsort(idx).reshape(grid_shape[0],-1))
# [[ 0 1 3 6 10]
# [ 2 4 7 11 14]
# [ 5 8 12 15 17]
# [ 9 13 16 18 19]]
Or, alternatively, if you need to assign other values to A, perhaps this would be more useful:
A = np.zeros(grid_shape)
A1d = A.ravel()
A1d[idx] = np.arange(N) # you can change np.arange(N) to any 1D array of shape (N,)
print(A)
# [[ 0. 1. 3. 6. 10.]
# [ 2. 4. 7. 11. 15.]
# [ 5. 8. 12. 16. 18.]
# [ 9. 13. 14. 17. 19.]]
I know you asked for a way to iterate through your array, but I wanted to show the above because generating arrays through whole-array assignment or numpy function calls (like np.argsort) as done above will probably be faster than using a Python loop. But if you need to use a Python loop, then:
for i, j in enumerate(idx):
A1d[j] = i
print(A)
# [[ 0. 1. 3. 6. 10.]
# [ 2. 4. 7. 11. 15.]
# [ 5. 8. 12. 16. 18.]
# [ 9. 13. 14. 17. 19.]]
>>> D
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]])
>>> D[::-1].diagonal(offset=1)
array([16, 12, 8, 4])
>>> D[::-1].diagonal(offset=-3)
array([0])
>>> np.hstack([D[::-1].diagonal(offset=-x) for x in np.arange(-4,4)])[::-1]
array([ 0, 1, 5, 2, 6, 10, 3, 7, 11, 15, 4, 8, 12, 16, 9, 13, 17,
14, 18, 19])
Simpler as long as it is not a large matrix.
I'm not sure if this is what you really want, but maybe:
>>> import numpy as np
>>> ar = np.random.random((4,4))
>>> ar
array([[ 0.04844116, 0.10543146, 0.30506354, 0.4813217 ],
[ 0.59962641, 0.44428831, 0.16629692, 0.65330539],
[ 0.61854927, 0.6385717 , 0.71615447, 0.13172049],
[ 0.05001291, 0.41577457, 0.5579213 , 0.7791656 ]])
>>> ar.diagonal()
array([ 0.04844116, 0.44428831, 0.71615447, 0.7791656 ])
>>> ar[::-1].diagonal()
array([ 0.05001291, 0.6385717 , 0.16629692, 0.4813217 ])
Edit
As a general solution, for arbitrarily shape arrays, you can use
import numpy as np
shape = tuple([np.random.randint(3,10) for i in range(2)])
ar = np.arange(np.prod(shape)).reshape(shape)
out = np.hstack([ar[::-1].diagonal(offset=x) \
for x in np.arange(-ar.shape[0]+1,ar.shape[1]-1)])
print ar
print out
giving, for example
[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[10 11 12 13 14]
[15 16 17 18 19]
[20 21 22 23 24]]
[ 0 5 1 10 6 2 15 11 7 3 20 16 12 8 4 21 17 13 9 22 18 14 23 19]