I am trying to get a 2d array, by randomly generating its rows and appending
import numpy as np
my_nums = np.array([])
for i in range(100):
x = np.random.rand(2, 1)
my_nums = np.append(my_nums, np.array(x))
But I do not get what I want but instead get a 1d array.
What is wrong?
Transposing x did not help either.
You could do this by using np.append(axis=0) or np.vstack. This however requires the rows appended to have the same length as the rows already in the array.
You cannot use the same code to append a row with two values to an empty array, and to append a row to an already existing 2D array: numpy will throw a
ValueError: all the input arrays must have same number of dimensions.
You could initialize my_nums to work around this:
my_nums = np.random.rand(1, 2)
for i in range(99):
x = np.random.rand(1, 2)
my_nums = np.append(my_nums, x, axis=0)
Note the decrease in the range by one due to the initialization row. Also note that I changed the dimensions to (1, 2) to get actual row vectors.
Much easier than appending row-wise will of course be to create the array in the wanted final shape:
my_nums = np.random.rand(100, 2)
Related
This question already has answers here:
Good ways to "expand" a numpy ndarray?
(6 answers)
Closed 1 year ago.
I have a numpy array with dimensions (1316, 21) and I need to increase it to (1329, 21). It doesn't matter what values are stored in the added space at the end. I tried to do:
x = np.append(x, np.zeros(13))
But that changes the dimensions of the array to (27649,) which shows that it is converting it into a one dimensional array then adding the zeros to the end.
How do I append empty 2 dimensional values to an array like this?
Use np.concatenate or np.vstack
np.concatenate([x, np.zeros((13, x.shape[1]))], axis=0)
# or
np.vstack([x, np.zeros((13, x.shape[1]))])
Ummm...there is no converting the dimensions of a numpy array in python. A numpy array is simply a section of your RAM. You can't append to it in the sense of literally adding bytes to the end of the array, but you can create another array and copy over all the data (which is what np.append(), or np.vstack(), or np.concatenate(), etc.). In general, the dimensions of your array is simply a few variables that python keeps track of to present the data in the array to you, same thing as it's dtype.
For example,
X = np.array([1,2,3,4,5],dtype='int32')
print(X)
X.dtype = 'int16' #The data is not converted it is simply displayed differently now.
print(X) #Displays the data for the array.
X.shape = (5,2) #Does not convert the data or touch it.
print(X) #Displays the data for you using the parameter set in .shape.
For your data, you can simply update the .shape when you append more data.
x = np.append(x, np.zeros((13,21)))
x.shape = (1329, 21)
May be like this:
import numpy as np
x = np.array([[1, 2, 3,4], [4, 5, 6,7]])
x = np.append(x, [np.zeros(4) for _ in range(13)] , axis=0)
print(x.shape)
print(x)
I am trying to sample with replacement a base 2D numpy array with shape of (4,2) by rows, say 10 times. The final output should be a 3D numpy array.
Have tried the code below, it works. But is there a way to do it without the for loop?
base=np.array([[20,30],[50,60],[70,80],[10,30]])
print(np.shape(base))
nsample=10
tmp=np.zeros((np.shape(base)[0],np.shape(base)[1],10))
for i in range(nsample):
id_pick = np.random.choice(np.shape(base)[0], size=(np.shape(base)[0]))
print(id_pick)
boot1=base[id_pick,:]
tmp[:,:,i]=boot1
print(tmp)
Here's one vectorized approach -
m,n = base.shape
idx = np.random.randint(0,m,(m,nsample))
out = base[idx].swapaxes(1,2)
Basic idea is that we generate all the possible indices with np.random.randint as idx. That would an array of shape (m,nsample). We use this array to index into the input array along the first axis. Thus, it selects random rows off base. To get the final output with a shape (m,n,nsample), we need to swap last two axes.
You can use the stack function from numpy. Your code would then look like:
base=np.array([[20,30],[50,60],[70,80],[10,30]])
print(np.shape(base))
nsample=10
tmp = []
for i in range(nsample):
id_pick = np.random.choice(np.shape(base)[0], size=(np.shape(base)[0]))
print(id_pick)
boot1=base[id_pick,:]
tmp.append(boot1)
tmp = np.stack(tmp, axis=-1)
print(tmp)
Based on #Divakar 's answer, if you already know the shape of this 2D-array, you can treat it as an (8,) 1D array while bootstrapping, and then reshape it:
m, n = base.shape
flatbase = np.reshape(base, (m*n,))
idxs = np.random.choice(range(8), (numReps, m*n))
bootflats = flatbase[idx]
boots = np.reshape(flatbase, (numReps, m, n))
I have a numpy array consisting of
[1,3,8,6,0,2,4,5,9,7]
This array is a random array consisting of 10 numbers 0-9.
I also have a 2D numpy array, a 10X10 2D numpy array with numerical values.
I would like to use my 1D numpy array (above) to access specific instances in my 2D numpy array, by looping through the 1D array
Loop 1: takes in 1 and 3, and finds the value at [1:3] in my 2D numpy array.
Loop 2: takes in 3 and 8, and finds the value at [3:8] in my 2D numpy array.
.
Loop 10: takes in 7 and 1, and finds the value at [7:1] in my 2D numpy array.
I would like to add up these values in my 2D numpy array.
so far I have :
array=[1,3,8,6,0,2,4,5,9,7]
values =0
for i in range (0, len(array)): #this is 10
a=array2[i,array[i]+1] #array2 is the 2D numpy array with the values
values=values+a
This works to some degree but how to I get it to access the last element to the first? i.e. find [7,1]
You can use simple slicing to make this work.
arr = np.random.randint(0, 10, (10,10))
pos = np.array([1,3,8,6,0,2,4,5,9,7])
pos = np.append(pos, pos[0])
rows = pos[0:-1]
cols = pos[1:]
result = sum(arr[rows, cols])
You can do the slicing twice to make it work.
values = 0
for i in range(len(array)):
a = Matrix[array[i],array[i+1]]
values += a
Also, the array you put has 11 elements which means the 10-th loop will not be what you intended.
I'm not sure I fully understood what you were trying to achieve but...
What about something like this?
a = np.array([1,3,8,6,0,9,2,4,5,9,7])
b = np.array(range(100)).reshape(10,10)
for i in range (len(a)):
print (a[i%len(a)],a[(i+1)%len(a)])
print (b[a[i%len(a)],a[(i+1)%len(a)]])
I removed 10 from the a array to avoid an index out of range error.
I also took the value [x,y] (and not the range [x:y] from the 2D array.
I am trying to compare a 1D array element-wise to a 2D array, and returns the elements of the 2D array which fulfils the condition in a 2D array form without using a for loop. Preferably using numpy or quicker method.
a = range(1,10)
Tna = np.random.choice(a, size=[250,10,1000], replace=True)
sum_Ta = np.sum(Tna, axis = 1)
percent = np.percentile(sum_Ta, 5, axis =0)
Now I would like to get a 2D array which contains the elements of sum_Ta if the elements are smaller the percent. Such that 250 elements of sum_Ta are comparing with 1 element of percent for 1000 times. Originally I can do, ES = sum_Ta[sum_Ta < percent[:,None]], but it only gives me a 1D array, not a 2D array.
Assuming you mean that for each row, you want the element of the row to be included if it is less than the percentage associated with its column.
Try the following:
mask = sum_Ta < (percent * np.ones((250,1)))
ES = np.zeros((250, 1000))
ES[mask] = sum_Ta[mask]
I have a list of several hundred 10x10 arrays that I want to stack together into a single Nx10x10 array. At first I tried a simple
newarray = np.array(mylist)
But that returned with "ValueError: setting an array element with a sequence."
Then I found the online documentation for dstack(), which looked perfect: "...This is a simple way to stack 2D arrays (images) into a single 3D array for processing." Which is exactly what I'm trying to do. However,
newarray = np.dstack(mylist)
tells me "ValueError: array dimensions must agree except for d_0", which is odd because all my arrays are 10x10. I thought maybe the problem was that dstack() expects a tuple instead of a list, but
newarray = np.dstack(tuple(mylist))
produced the same result.
At this point I've spent about two hours searching here and elsewhere to find out what I'm doing wrong and/or how to go about this correctly. I've even tried converting my list of arrays into a list of lists of lists and then back into a 3D array, but that didn't work either (I ended up with lists of lists of arrays, followed by the "setting array element as sequence" error again).
Any help would be appreciated.
newarray = np.dstack(mylist)
should work. For example:
import numpy as np
# Here is a list of five 10x10 arrays:
x = [np.random.random((10,10)) for _ in range(5)]
y = np.dstack(x)
print(y.shape)
# (10, 10, 5)
# To get the shape to be Nx10x10, you could use rollaxis:
y = np.rollaxis(y,-1)
print(y.shape)
# (5, 10, 10)
np.dstack returns a new array. Thus, using np.dstack requires as much additional memory as the input arrays. If you are tight on memory, an alternative to np.dstack which requires less memory is to
allocate space for the final array first, and then pour the input arrays into it one at a time.
For example, if you had 58 arrays of shape (159459, 2380), then you could use
y = np.empty((159459, 2380, 58))
for i in range(58):
# instantiate the input arrays one at a time
x = np.random.random((159459, 2380))
# copy x into y
y[..., i] = x