I want to initialise a numpy array of a specific shape such that when I append numbers to it it will 'fill up' in that shape.
The length of the array will vary - and that is fine I do not mind how long it is - but I want it to have 4 columns. Ideally somthing similar to the following:
array = np.array([:, 4])
print(array)
array = [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]
Again the actual length of the array would not be defines. That way if I was to append a different array it would work as follows
test_array = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]
array = np.append(array, test_array)
print(array)
array = [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]]
Is there any way to do this?
If I understand well your issue, I think you do not need to initialize your array.
You sould check first that your array size divides by 4.
import numpy as np
l = test_array.shape[0]
cols = 4
rows = l / cols
my_array = np.reshape(test_array, (rows, cols))
The kind of behavior that you seek is unusual. You should explain why you need it. If you want something readily grows, use Python list. numpy arrays have a fixed size. Values can be assigned to an array in various ways, but to grow it, you need to create a new array with some version of concatenate. (Yes, there is a resize function/method, but that's not commonly used.)
I'll illustrate the value assignment options:
Initial an array with a known size. In your case the 5 could be larger than anticipated, and the 4 is the desired number of 'columns'.
In [1]: arr = np.zeros((5,4), dtype=int)
In [2]: arr
Out[2]:
array([[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]])
Assign 4 values to one row:
In [3]: arr[0] = [1,2,3,4]
Assign 3 values starting at a given point in a flat view of the array:
In [4]: arr.flat[4:7] = [1,2,3]
In [5]: arr
Out[5]:
array([[1, 2, 3, 4],
[1, 2, 3, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]])
This array, while defined as (5,4) shape, can be viewed as (20,) 1d array. I had to choose the appropriate slice values in the flat view.
More commonly we assign values to a block of rows (or a variety of other indexed areas). arr[2:, :] is a (3,4) portion of arr. So we need to assign (3,4) array to it (or an equivalent list structure). To get full benefit of this sort of assignment you need to read up on broadcasting.
In [6]: arr[2:,:] = np.reshape(list(range(10,22)),(3,4))
In [7]: arr
Out[7]:
array([[ 1, 2, 3, 4],
[ 1, 2, 3, 0],
[10, 11, 12, 13],
[14, 15, 16, 17],
[18, 19, 20, 21]])
In [8]: arr.ravel()
Out[8]:
array([ 1, 2, 3, 4, 1, 2, 3, 0, 10, 11, 12, 13, 14, 15, 16, 17, 18,
19, 20, 21])
Related
Hi everyone and thank you for your assistance. I am new to python and failed to find an efficient alternative to for loops for the following task.
I want to multiply ndarrays A and B of dimension (d,n,m) and (d,m), respectively. With some abuse of terminology to help understanding, A is a list of nxm matrices and B is a list of vectors in R^m.
For example:
A = np.array([[[0,0,0,0,0],[1,1,1,1,1],[2,2,2,2,2]],[[3,3,3,3,3],[4,4,4,4,4],[5,5,5,5,5]]])
B = np.array([[1,2,3,4,5],[5,6,7,8,9]])
My solution uses a for loop
for i in range(2):
print(A[i]*B[i])
Is there any cheaper alternative (no loops)?
Thank you again
In this case, you can use broadcasting by adding in a new dimension in the "middle" for B:
>>> import numpy as np
>>> A = np.array([[[0,0,0,0,0],[1,1,1,1,1],[2,2,2,2,2]],[[3,3,3,3,3],[4,4,4,4,4],[5,5,5,5,5]]])
>>> B = np.array([[1,2,3,4,5],[5,6,7,8,9]])
>>> A * B[:, None, :]
array([[[ 0, 0, 0, 0, 0],
[ 1, 2, 3, 4, 5],
[ 2, 4, 6, 8, 10]],
[[15, 18, 21, 24, 27],
[20, 24, 28, 32, 36],
[25, 30, 35, 40, 45]]])
Here is a link to the official docs
Note, your original solution already relied on broadcasting:
>>> A[0]
array([[0, 0, 0, 0, 0],
[1, 1, 1, 1, 1],
[2, 2, 2, 2, 2]])
>>> B[0]
array([1, 2, 3, 4, 5])
>>> A[0] * B[0]
array([[ 0, 0, 0, 0, 0],
[ 1, 2, 3, 4, 5],
[ 2, 4, 6, 8, 10]])
To improve the speed I would like to avoid forloops.
I have a image array looking like :
image = np.zeros_like(np.zeros(shape=(480,640,1)),dtype=np.uint8)
and a typed np array Events with the following types
dtype = [('x', '<f8'),('y', '<f8'),('grayVal','<u2')
where 'x' = row and 'y' = column of the image array.
The Question is:
How can I assign the grayVal in Events to all the x and y in the image ?
So far I tried (and more not displayable):
The For Loop:
for event in Events:
image[event['y'],event['x']] = event['grayVal']
and Indexing
events['y'].shape
(98210,)
events['x'].shape
(98210,)
events['grayVal'].shape
(98210,)
image[np.ix_(events['y'],events['x'])] = events['grayVal']
which somehow does not work due to the error message:
ValueError: shape mismatch: value array of shape (98210,) could not be broadcast to indexing result of shape (98210,98210,1)
What am I missing? Thanks for the help.
Let's work with a small example, one we can actually examine and play with!
Make a structured array:
In [32]: dt = np.dtype([('x', int),('y', int) ,('grayVal','u2')])
In [33]: events = np.zeros(5, dt)
In [34]: events['x'] = np.arange(5)
In [35]: events['y'] = np.array([3,4,0,2,1])
In [36]: events['grayVal'] = np.arange(1,6)
To examine indexing lets make a nice 2d array:
In [38]: image = np.arange(25).reshape(5,5)
In [39]: image
Out[39]:
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
Look at what ix_ produces - 2 arrays that can broadcast against each other. A (5,1) and (1,5), which broadcast to (5,5):
In [40]: np.ix_(events['y'], events['x'])
Out[40]:
(array([[3],
[4],
[0],
[2],
[1]]),
array([[0, 1, 2, 3, 4]]))
Using those arrays to index image just shuffles values - the result is still a 2d array:
In [41]: image[np.ix_(events['y'], events['x'])]
Out[41]:
array([[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24],
[ 0, 1, 2, 3, 4],
[10, 11, 12, 13, 14],
[ 5, 6, 7, 8, 9]])
If instead we index with the arrays, not with the ix_ arrays:
In [42]: image[events['y'], events['x']]
Out[42]: array([15, 21, 2, 13, 9])
This is just the diagonal of the array produced with ix_. Indexing with a (n,) and (n,) arrays produces a (n,) array of values (as opposed to the ix_ (n,n) array).
So starting with a zeros image, we can assign values with:
In [43]: image= np.zeros((5,5), 'u2')
In [44]: image[events['y'], events['x']]=events['grayVal']
In [45]: image
Out[45]:
array([[0, 0, 3, 0, 0],
[0, 0, 0, 0, 5],
[0, 0, 0, 4, 0],
[1, 0, 0, 0, 0],
[0, 2, 0, 0, 0]], dtype=uint16)
I can only think of a slow version, with a for loop for now. But that could be OK if the array is sparse. Maybe someone else can vectorize that.
import numpy as np
image = np.zeros(shape=(3,4 ),dtype=np.uint8) # image is empty
# evy is just a bag of nonzero pixels
evy=np.zeros(shape=(3), dtype = [('x', '<u2'),('y', '<u2') ,('grayVal','<u2') ])
evy[0]=(1,1,128)
evy[1]=(0,0,1)
evy[2] =(2,3,255)
#slow version
for i in range(3):
image[evy[i][0],evy[i][1]]=evy[i][2]
output:
array([[ 1, 0, 0, 0],
[ 0, 128, 0, 0],
[ 0, 0, 0, 255]], dtype=uint8)
hope doing well.
I have an extremely big numpy array and want to split it into several ones. My array has three columns and I want to split it where the all the columns are reaching their maximum values:
array = [[0, 0, 0],
[0, 0, 5],
[10, 5, 10],
[1, 1, 1],
[5, 5, 15],
[10, 8, 20],
[2, 0, 0],
[10, 10, 12],
[1, 2, 0],
[2, 5, 9]]
Now, I want to split it into four array:
sub_array_1=[[0, 0, 0],
[0, 0, 5],
[10, 5, 10]]
sub_array_2=[[1, 1, 1],
[5, 5, 15],
[10, 8, 20]]
sub_array_3=[[2, 0, 0],
[10, 10, 12]]
sub_array_4=[[1, 2, 0],
[2, 5, 9]]
I tried to it in a for loop having if statements saying that give me an array when each element of my input is bigger than the element stored in the both upper and lower rows. And I also should figure out the last row:
import numpy as np
sub_array_1=np.array([])
for i in array:
if array[i,:]>array[i+1,:] and array[i,:]>array[i+1,:]:
vert_1=np.append(sub_array_1,array[0:i,:])
My code doesn't work, but it simply shows my idea.
I am quite new in Python and I could not find the way to write my idea as a code. So, I appreciate any help and contribution.
Cheers,
Ali
IIUC, one way using numpy.diff with numpy.array_split:
indices = np.argwhere(np.all(np.diff(array, axis=0) < 0, axis=1))
np.array_split(array, indices.ravel()+1, axis=0)
Output:
[array([[ 0, 0, 0],
[ 0, 0, 5],
[10, 5, 10]]),
array([[ 1, 1, 1],
[ 5, 5, 15],
[10, 8, 20]]),
array([[ 2, 0, 0],
[10, 10, 12]]),
array([[1, 2, 0],
[2, 5, 9]])]
np.all and np.diff find a row where all elements of the row as a negative difference with a next row (i.e. where the peak ends)
np.array_split will then split the given array based on the locations of the peak found.
Given an index array idx that only contains 0 and 1 elements, and 1s represent the sample indices of interest, and a sample array A (A.shape[0] = idx.shape[0]). The objective here is to extract a subset of samples based on the index vector.
In matlab, it is trivial to do:
B = A(idx,:) %assuming A is 2D matrix and idx is a logical vector
How to achieve this in Python in a simple manner?
If your mask array idx has the same shape as your array A, then you should be able to extract elements specified by the mask if you convert idx to a boolean array, using astype.
Demo -
>>> A
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
>>> idx
array([[1, 0, 0, 1, 1],
[0, 0, 0, 1, 0],
[1, 0, 0, 1, 1],
[1, 0, 0, 1, 1],
[0, 1, 1, 1, 1]])
>>> A[idx.astype(bool)]
array([ 0, 3, 4, 8, 10, 13, 14, 15, 18, 19, 21, 22, 23, 24])
Using the bool operation is equivalent to that logical one in Matlab:
B = A[idx.astype(bool)]
Can I use numpy to generate repeating patterns of indices for example.
0, 1, 2, 3, 4, 5, 0, 6, 7, 8, 9, 10, 0, 11, 12, 13, 14, 15
or
0,1,2,1,2,3,4,5,6,5,6,7
Is there a method in numpy i can use to generate these lists between a range ?
currently I am doing this using lists in python but I was curious if I could use numpy to speed things up.
I am not sure what methods to even look into other than numpy.arange.
Just to further clarify I am generating indices to triangles in opengl in various patterns.
so for traingles in a circle I have some code like this.
for fan_set in range(0, len(self.vertices) / vertex_length, triangle_count):
for i in range(fan_set + 1, fan_set + 8):
self.indices.append(fan_set)
self.indices.append(i)
self.indices.append(i + 1)
Your first example can be produced via numpy methods as:
In [860]: np.concatenate((np.zeros((3,1),int),np.arange(1,16).reshape(3,5)),axis=1).ravel()
Out[860]:
array([ 0, 1, 2, 3, 4, 5, 0, 6, 7, 8, 9, 10, 0, 11, 12, 13, 14,
15])
That's because I see this 2d repeated pattern
array([[ 0, 1, 2, 3, 4, 5],
[ 0, 6, 7, 8, 9, 10],
[ 0, 11, 12, 13, 14, 15]])
The second pattern can be produced by ravel of this 2d array (produced by broadcasting 2 arrays):
In [863]: np.array([0,1,4,5])[:,None]+np.arange(3)
Out[863]:
array([[0, 1, 2],
[1, 2, 3],
[4, 5, 6],
[5, 6, 7]])
I can produce the 1st pattern with a variation on the 2nd (the initial column of 0s disrupts the pattern)
I=np.array([0,5,10])[:,None]+np.arange(0,6)
I[:,0]=0
I think your double loop can be expressed as a list comprehension as
In [872]: np.array([ [k,i,i+1] for k in range(0,1,1) for i in range(k+1,k+8)]).ravel()
Out[872]: array([0, 1, 2, 0, 2, 3, 0, 3, 4, 0, 4, 5, 0, 5, 6, 0, 6, 7, 0, 7, 8])
or without the ravel:
array([[0, 1, 2],
[0, 2, 3],
[0, 3, 4],
[0, 4, 5],
[0, 5, 6],
[0, 6, 7],
[0, 7, 8]])
though I don't know what parameters produce your examples.
I'm not sure I understand exactly what you mean, but the following is what I use to generate unique indices for 3D points;
def indexate(points):
"""
Convert a numpy array of points into a list of indices and an array of
unique points.
Arguments:
points: A numpy array of shape (N, 3).
Returns:
An array of indices and an (M, 3) array of unique points.
"""
pd = {}
indices = [pd.setdefault(tuple(p), len(pd)) for p in points]
pt = sorted([(v, k) for k, v in pd.items()], key=lambda x: x[0])
unique = np.array([i[1] for i in pt])
return np.array(indices, np.uint16), unique
You can find this code in my stltools package on github.
It works like this;
In [1]: import numpy as np
In [2]: points = np.array([[1,0,0], [0,0,1], [1,0,0], [0,1,0]])
In [3]: pd = {}
In [4]: indices = [pd.setdefault(tuple(p), len(pd)) for p in points]
In [5]: indices
Out[5]: [0, 1, 0, 2]
In [6]: pt = sorted([(v, k) for k, v in pd.items()], key=lambda x: x[0])
In [7]: pt
Out[7]: [(0, (1, 0, 0)), (1, (0, 0, 1)), (2, (0, 1, 0))]
In [8]: unique = np.array([i[1] for i in pt])
In [9]: unique
Out[9]:
array([[1, 0, 0],
[0, 0, 1],
[0, 1, 0]])
The key point (if you'll pardon the pun) is to use a tuple of the point (because a tuple is immutable and thus hashable) as the key in a dictionary with the setdefault method, while the length of the dict is the value. In effect, the value is the first time this exact point was seen.
I am not 100% certain this is what you're after, I think you can achieve this using pair of range values and increment n times 3 (the gap between each group), then use numpy.concatenate to concatenate the final array, like this:
import numpy as np
def gen_list(n):
return np.concatenate([np.array(range(i, i+3) + range(i+1, i+4)) + i*3
for i in xrange(n)])
Usage:
gen_list(2)
Out[16]: array([0, 1, 2, 1, 2, 3, 4, 5, 6, 5, 6, 7])
gen_list(3)
Out[17]:
array([ 0, 1, 2, 1, 2, 3, 4, 5, 6, 5, 6, 7, 8, 9, 10, 9, 10,
11])
list(gen_list(2))
Out[18]: [0, 1, 2, 1, 2, 3, 4, 5, 6, 5, 6, 7]
In my sample I only use n as how many groups you want to generate, you may change this to suit your triangle-ish requirements.