I'd like to return a 2D numpy.array with multiple rolls of a given 1D numpy.array.
>>> multiroll(np.arange(10), [-1, 0, 1, 2])
array([[1., 0., 9., 8.],
[2., 1., 0., 9.],
[3., 2., 1., 0.],
[4., 3., 2., 1.],
[5., 4., 3., 2.],
[6., 5., 4., 3.],
[7., 6., 5., 4.],
[8., 7., 6., 5.],
[9., 8., 7., 6.],
[0., 9., 8., 7.]])
Is there some combination of numpy.roll, numpy.tile, numpy.repeat, or other functions that does this?
Here's what I've tried
def multiroll(array, rolls):
"""Create multiple rolls of 1D vector"""
m = len(array)
n = len(rolls)
shape = (m, n)
a = np.empty(shape)
for i, roll in enumerate(rolls):
a[:,i] = np.roll(array, roll)
return a
I'd expected there's a more "Numpythonic" way of doing this that doesn't use the loop.
Approach #1 : For elegance
Here's one way with broadcasting -
In [44]: a
Out[44]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
In [45]: rolls
Out[45]: array([-1, 0, 1, 2])
In [46]: a[(np.arange(len(a))[:,None]-rolls) % len(a)]
Out[46]:
array([[1, 0, 9, 8],
[2, 1, 0, 9],
[3, 2, 1, 0],
[4, 3, 2, 1],
[5, 4, 3, 2],
[6, 5, 4, 3],
[7, 6, 5, 4],
[8, 7, 6, 5],
[9, 8, 7, 6],
[0, 9, 8, 7]])
Approach #2 : For memory/perf-efficiency
Idea mostly borrowed from - this post.
We can leverage np.lib.stride_tricks.as_strided based scikit-image's view_as_windows to get sliding windows. More info on use of as_strided based view_as_windows.
from skimage.util.shape import view_as_windows
def multiroll_stridedview(a, r):
r = np.asarray(r)
# Concatenate with sliced to cover all rolls
a_ext = np.concatenate((a,a[:-1]))
# Get sliding windows; use advanced-indexing to select appropriate ones
n = len(a)
return view_as_windows(a_ext,n)[:,(n-r)%n]
Approach #3 : For mathematical beauty (and efficiency ?)
Using a fft kernel in the frequency domain you can process a whole matrix at once. This method only work with integer
A = np.array([[1., 1., 1., 1.],
[2., 2., 2., 2.],
[3., 3., 3., 3.],
[4., 4., 4., 4.],
[5., 5., 5., 5.],
[6., 6., 6., 6.],
[7., 7., 7., 7.],
[8., 8., 8., 8.],
[9., 9., 9., 9.],
[0., 0., 0., 0.]]).transpose()
m,n = A.shape
#shift vector
s=[-1,0,1,2]
#transformation kernel (shift theorem)
fftkernel = np.exp(-2*1j*np.pi/n*np.outer(v,np.arange(0,n)))
#Apply the shift
res=np.round(np.fft.ifft(np.fft.fft(A,axis = 1) * fftkernel ,axis = 1)).real.transpose()
We get:
array([[1., 0., 9., 8.],
[2., 1., 0., 9.],
[3., 2., 1., 0.],
[4., 3., 2., 1.],
[5., 4., 3., 2.],
[6., 5., 4., 3.],
[7., 6., 5., 4.],
[8., 7., 6., 5.],
[9., 8., 7., 6.],
[0., 9., 8., 7.]])
You can get more information about how this code work here
For left circular shift you can use:
fftkernel = np.exp(2*1j*np.pi/n*np.outer(v,np.arange(0,n)))
Without minus sign.
Related
I have a 3d numpy table with shape=(2,3,4) like below:
a = np.array([[[1., 2., 3., 4.],
[1., 2., 3., 4.],
[1., 2., 3., 4.]],
[[5., 6., 7., 8.],
[5., 6., 7., 8.],
[5., 6., 7., 8.]]])
And want to reshape this in a way where the columns in each dimension are stacked into a new column in a 2d matrix.
1 5
1 5
1 5
2 6
2 6
2 6
3 7
3 7
3 7
4 8
4 8
4 8
Here you go:
res = a.T.reshape((-1,2))
Output:
array([[1., 5.],
[1., 5.],
[1., 5.],
[2., 6.],
[2., 6.],
[2., 6.],
[3., 7.],
[3., 7.],
[3., 7.],
[4., 8.],
[4., 8.],
[4., 8.]])
To reshape a numpy array, use the reshape method.
Basically it looks at the array as it was flattened and works over it with the new given shape. It does however iterates over the last index first, ie, the inner-most list will be processed, then the next and so on.
So both a np.array([1, 2, 3, 4, 5, 6]).reshape((3, 2)) and a np.array([[1, 2, 3], [4, 5, 6]]).reshape((3, 2)) will give [[1, 2], [3, 4], [5, 6]], since these two originating arrays are the same when flattened.
You want a (12, 2) array, or if you read the reshape docs, you can pass (-1, 2) and numpy will figure the other dimension.
So if you just give the new shape for your array as is, it will start working with the first list x[0, 0] = [1, 2, 3, 4], which would become [[1, 2], [3, 4]], ... That's not what you want.
But note that if you transpose your array first, then you'll have the items you want in the inner lists (fast varying index):
In : x.T
Out:
array([[[1., 5.],
[1., 5.],
[1., 5.]],
[[2., 6.],
[2., 6.],
[2., 6.]],
[[3., 7.],
[3., 7.],
[3., 7.]],
[[4., 8.],
[4., 8.],
[4., 8.]]])
Which is almost what you want, except for the extra dimension. So now you can just reshape this and get your (12, 2) array the way you want:
In : x.T.reshape((-1, 2))
Out:
array([[1., 5.],
[1., 5.],
[1., 5.],
[2., 6.],
[2., 6.],
[2., 6.],
[3., 7.],
[3., 7.],
[3., 7.],
[4., 8.],
[4., 8.],
[4., 8.]])
I have a 3d numpy array of following form:
array([[[ 1., 5., 4.],
[ 1., 5., 4.],
[ 1., 2., 4.]],
[[ 3., 6., 4.],
[ 6., 6., 4.],
[ 6., 6., 4.]]])
Is there a efficient way to convert it to a 2d array of form:
array([[1, 1, 1, 5, 5, 2, 4, 4, 4],
[3, 6, 6, 6, 6, 6, 4, 4, 4]])
Thanks a lot!
In [54]: arr = np.array([[[ 1., 5., 4.],
[ 1., 5., 4.],
[ 1., 2., 4.]],
[[ 3., 6., 4.],
[ 6., 6., 4.],
[ 6., 6., 4.]]])
In [61]: arr.reshape((arr.shape[0], -1), order='F')
Out[61]:
array([[ 1., 1., 1., 5., 5., 2., 4., 4., 4.],
[ 3., 6., 6., 6., 6., 6., 4., 4., 4.]])
The array arr has shape (2, 3, 3). We wish to keep the first axis of length 2, and flatten the two axes of length 3.
If we call arr.reshape(h, w) then NumPy will attempt to reshape arr to shape (h, w). If we call arr.reshape(h, -1) then NumPy will replace the -1 with whatever integer is needed for the reshape to make sense -- in this case, arr.size/h.
Hence,
In [63]: arr.reshape((arr.shape[0], -1))
Out[63]:
array([[ 1., 5., 4., 1., 5., 4., 1., 2., 4.],
[ 3., 6., 4., 6., 6., 4., 6., 6., 4.]])
This is almost what we want, but notice that the values in each subarray, such as
[[ 1., 5., 4.],
[ 1., 5., 4.],
[ 1., 2., 4.]]
are being traversed by marching from left to right before going down to the next row.
We want to march down the rows before going on to the next column.
To achieve that, use order='F'.
Usually the elements in a NumPy array are visited in C-order -- where the last index moves fastest. If we visit the elements in F-order then the first index moves fastest. Since in a 2D array of shape (h, w), the first axis is associated with the rows and the last axis the columns, traversing the array in F-order marches down each row before moving on to the next column.
I am attempting to add two arrays.
np.zeros((6,9,20)) + np.array([1,2,3,4,5,6,7,8,9])
I want to get something out that is like
array([[[ 1., 1., 1., ..., 1., 1., 1.],
[ 2., 2., 2., ..., 2., 2., 2.],
[ 3., 3., 3., ..., 3., 3., 3.],
...,
[ 7., 7., 7., ..., 7., 7., 7.],
[ 8., 8., 8., ..., 8., 8., 8.],
[ 9., 9., 9., ..., 9., 9., 9.]],
[[ 1., 1., 1., ..., 1., 1., 1.],
[ 2., 2., 2., ..., 2., 2., 2.],
[ 3., 3., 3., ..., 3., 3., 3.],
...,
[ 7., 7., 7., ..., 7., 7., 7.],
[ 8., 8., 8., ..., 8., 8., 8.],
[ 9., 9., 9., ..., 9., 9., 9.]],
So adding entries to each of the matrices at the corresponding column. I know I can code it in a loop of some sort, but I am trying to use a more elegant / faster solution.
You can bring broadcasting into play after extending the dimensions of the second array with None or np.newaxis, like so -
np.zeros((6,9,20))+np.array([1,2,3,4,5,6,7,8,9])[None,:,None]
If I understand you correctly, the best thing to use is NumPy's Broadcasting. You can get what you want with the following:
np.zeros((6,9,20))+np.array([1,2,3,4,5,6,7,8,9]).reshape((1,9,1))
I prefer using the reshape method to using slice notation for the indices the way Divakar shows, because I've done a fair bit of work manipulating shapes as variables, and it's a bit easier to pass around tuples in variables than slices. You can also do things like this:
array1.reshape(array2.shape)
By the way, if you're really looking for something as simple as an array that runs from 0 to N-1 along an axis, check out mgrid. You can get your above output with just
np.mgrid[0:6,1:10,0:20][1]
You could use tile (but you would also need swapaxes to get the correct shape).
A = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9])
B = np.tile(A, (6, 20, 1))
C = np.swapaxes(B, 1, 2)
In order to do calculations, I have a set of arrays: "sub" array (as you can see below), and I want to reshape it in an array as given by "test" array:
import numpy as np
sub = np.array([[[[ 1., 1.],
[ 1., 1.]],
[[ 2., 2.],
[ 2., 2.]],
[[ 3., 3.],
[ 3., 3.]],
[[ 4., 4.],
[ 4., 4.]]],
[[[ 5., 5.],
[ 5., 5.]],
[[ 6., 6.],
[ 6., 6.]],
[[ 7., 7.],
[ 7., 7.]],
[[ 8., 8.],
[ 8., 8.]]]])
test=np.array([[[ 1., 1., 2., 2.],
[ 1., 1., 2., 2.],
[ 3., 3., 4., 4.],
[ 3., 3., 4., 4.]],
[[ 5., 5., 6., 6.],
[ 5., 5., 6., 6.],
[ 7., 7., 8., 8.],
[ 7., 7., 8., 8.]]])
I have found on a post a part of code which seems to work for my case, but I have some errors...
k,l,m,n,p =2,2,2,2,2
conc = np.array([np.ones([p,m,n],dtype=int)*i for i in range(k*l)])
test_reshape=np.vstack([np.hstack(sub[i:i+l]) for i in range(0,k*l*p,l)])
Here's an alternative way to swap, slice and stack your array into shape:
>>> t = sub.swapaxes(1, 3).T.swapaxes(1, 3)
>>> x = np.c_[t[::2, 0], t[1::2, 0]]
>>> y = np.c_[t[::2, 1], t[1::2, 1]]
>>> np.array((np.r_[x[0], x[1]], np.r_[y[0], y[1]]))
array([[[ 1., 1., 2., 2.],
[ 1., 1., 2., 2.],
[ 3., 3., 4., 4.],
[ 3., 3., 4., 4.]],
[[ 5., 5., 6., 6.],
[ 5., 5., 6., 6.],
[ 7., 7., 8., 8.],
[ 7., 7., 8., 8.]]])
Edit: Or instead, squeeze, slice and stack:
>>> x = np.c_[sub[:1][:,::2], sub[:1][:,1::2]].squeeze()
>>> y = np.c_[sub[1:][:,::2], sub[1:][:,1::2]].squeeze()
>>> np.array((np.r_[x[0], x[1]], np.r_[y[0], y[1]]))
# the required array
Perhaps there exists a pure numpy solution, but I'm not aware of it and it'll use quite a few tricks with strides. The solution below is thus not as efficient, because it uses python's for loops (making it less quick), but it 'll get your result in a general way, so without it depending on the size of your actual 4D array.
np.vstack( (sub[vol,2*sheet:2*sheet+2].reshape((4,-1)).T for vol in range(2) for sheet in range(2))).reshape((2,4,-1)
import numpy as np
sub = np.array(...)
test = np.array([np.hstack((np.vstack(( s[0],s[1] )),
np.vstack(( s[2],s[3] )))) for s in sub])
print test
In the OP's example the shape of sub is (2,4,2,2), but that the code above would work as is for an array of shape (n,4,m,m). For different shapes of the type (n,k,m,m) the code above can be adapted to different requirements.
Eventually I would like to add that when you look at the code you literally see what the code is achieving, and this may be compensating other defects of the code in terms of efficiency (i.e., copying vs reshaping).
A better solution (i.e, not mine ;-) and some aftertoughts
I have found this answer from unutbu (that contains a link to a more general solution) that the OP can easily (?) adapt to her/his needs. Due to the complex reshaping that is
involved data is however copied, hence the OP may want to measure the different performances of the two approaches, taking into account the incidence of the "reshaping" on the total run time of her/his program (i.e., imho shaving 0.3s on a runtime of 2' wouldn't be worth the effort)
Examplar interactive session
In the following, the data and the procedures are literally lifted from
the above mentioned answer from unutbu, with the last two statements added by me to show the addresses of the data buffers of the three ndarrays, x, y and z.
In [1]: import numpy as np
In [2]: x = np.arange(16).reshape((4,2,2))
In [3]: y = x.reshape(2,2,2,2).swapaxes(1,2).reshape(4,-1)
In [4]: x
Out[4]:
array([[[ 0, 1],
[ 2, 3]],
[[ 4, 5],
[ 6, 7]],
[[ 8, 9],
[10, 11]],
[[12, 13],
[14, 15]]])
In [5]: y
Out[5]:
array([[ 0, 1, 4, 5],
[ 2, 3, 6, 7],
[ 8, 9, 12, 13],
[10, 11, 14, 15]])
In [6]: z = x.T
In [7]: [a.__array_interface__['data'][0] for a in (x, y, z)]
Out[7]: [46375856, 45578800, 46375856]
In [8]:
This can be done using a reshape/swapaxes trick:
In [92]: sub.reshape(2,2,2,2,2).swapaxes(2,3).reshape(test.shape)
Out[92]:
array([[[ 1., 1., 2., 2.],
[ 1., 1., 2., 2.],
[ 3., 3., 4., 4.],
[ 3., 3., 4., 4.]],
[[ 5., 5., 6., 6.],
[ 5., 5., 6., 6.],
[ 7., 7., 8., 8.],
[ 7., 7., 8., 8.]]])
In [94]: np.allclose(sub.reshape(2,2,2,2,2).swapaxes(2,3).reshape(test.shape), test)
Out[94]: True
I confess I do not know how to generate this kind of solution without some guessing. But it appears that when you want to rearrange "blocks" in an array, there is a way to do it by reshaping to a higher dimension, swapping some axes, then reshaping to the desired shape. Given that sub.shape is (2, 4, 2, 2) reshaping to a higher dimension must mean (2, 2, 2, 2, 2). So you only have to test for a solution of the form
sub.reshape(2,2,2,2,2).swapaxes(i,j).reshape(test.shape)
and that is easy to do:
for i,j in IT.combinations(range(5), 2):
if np.allclose(sub.reshape(2,2,2,2,2).swapaxes(i,j).reshape(test.shape), test):
print(i,j)
reveals the right axes to swap:
(2, 3)
I'm working with 3-dimensional arrays (for the purpose of this example you can imagine they represent the RGB values at X, Y coordinates of the screen).
>>> import numpy as np
>>> a = np.floor(10 * np.random.random((2, 2, 3)))
>>> a
array([[[ 7., 3., 1.],
[ 9., 6., 9.]],
[[ 4., 6., 8.],
[ 8., 1., 1.]]])
What I would like to do, is to set to an arbitrary value the G channel for those pixels whose G channel is already below 5. I can manage to isolate the pixel I am interested in using:
>>> a[np.where(a[:, :, 1] < 5)]
array([[ 7., 3., 1.],
[ 8., 1., 1.]])
but I am struggling to understand how to assign a new value to the G channel only. I tried:
>>> a[np.where(a[:, :, 1] < 5)][1] = 9
>>> a
array([[[ 7., 3., 1.],
[ 9., 6., 9.]],
[[ 4., 6., 8.],
[ 8., 1., 1.]]])
...but it seems not to produce any effect. I also tried:
>>> a[np.where(a[:, :, 1] < 5), 1] = 9
>>> a
array([[[ 7., 3., 1.],
[ 9., 9., 9.]],
[[ 4., 6., 8.],
[ 9., 9., 9.]]])
...(failing to understand what is happening). Finally I tried:
>>> a[np.where(a[:, :, 1] < 5)][:, 1] = 9
>>> a
array([[[ 7., 3., 1.],
[ 9., 6., 9.]],
[[ 4., 6., 8.],
[ 8., 1., 1.]]])
I suspect I am missing something fundamental on how NumPy works (this is the first time I use the library). I would appreciate some help in how to achieve what I want as well as some explanation on what happened with my previous attempts.
Many thanks in advance for your help and expertise!
EDIT: The outcome I would like to get is:
>>> a
array([[[ 7., 9., 1.], # changed the second number here
[ 9., 6., 9.]],
[[ 4., 6., 8.],
[ 8., 9., 1.]]]) # changed the second number here
>>> import numpy as np
>>> a = np.array([[[ 7., 3., 1.],
... [ 9., 6., 9.]],
...
... [[ 4., 6., 8.],
... [ 8., 1., 1.]]])
>>> a
array([[[ 7., 3., 1.],
[ 9., 6., 9.]],
[[ 4., 6., 8.],
[ 8., 1., 1.]]])
>>> a[:,:,1][a[:,:,1] <; 5 ] = 9
>>> a
array([[[ 7., 9., 1.],
[ 9., 6., 9.]],
[[ 4., 6., 8.],
[ 8., 9., 1.]]])
a[:,:,1] gives you G channel, I subsetted it by a[:,:,1] < 5 using it as index. then assigned value 9 to that selected elements.
there is no need to use where, you can directly index an array with the boolean array resulting from your comparison operator.
a=array([[[ 7., 3., 1.],
[ 9., 6., 9.]],
[[ 4., 6., 8.],
[ 8., 1., 1.]]])
>>> a[a[:, :, 1] < 5]
array([[ 7., 3., 1.],
[ 8., 1., 1.]])
>>> a[a[:, :, 1] < 5]=9
>>> a
array([[[ 9., 9., 9.],
[ 9., 6., 9.]],
[[ 4., 6., 8.],
[ 9., 9., 9.]]])
you do not list the expected output in your question, so I am not sure this is what you want.