I have some weights that are generated via the command:
weights = np.random.rand(9+1, 8)
for i in range(8): # 7 to 8
weights[9][i] = random.uniform(.5,1.5)
Then, I try to insert it into an element of the following lattice:
lattice = np.zeros((2,10,5))
lattice[0][0][0] = weights
print(lattice)
This results in the error:
ValueError: setting an array element with a sequence.
My question is:
How can I insert the weights into the lattice?
I am aware that the problem is that the lattice is filled with float values, so it cannot accept a matrix.
I'm interested in finding a way to generate a lattice with the correct number of elements so that I can insert my matrices. An example would be very helpful.
I've read several posts on stackoverflow, including:
how to append a numpy matrix into an empty numpy array
ValueError: setting an array element with a sequence
Numpy ValueError: setting an array element with a sequence. This message may appear without the existing of a sequence?
Initialize the lattice like so in order to have entries that can be filled with matrices.
lattice = np.empty(shape=(2,10,5), dtype='object')
Presumably you won't need this to actually be a numpy array until you've finished filling the lattice. Thus, what you can do is just use nested lists, and then call numpy array on the entire list. You could do something like:
lattice = [[[None for _ in range(5)] for _ in range(10)] for _ in range(2)]
and then use:
lattice[0][0][0] = weights
and when you've filled in all the elements, call:
lattice = np.array(lattice)
Related
I'm trying to initialize an "empty" array with each elements containing t_list a 8x8 np.zeros array :
t_list = np.zeros((8,8), dtype=np.float32)
I would now want to have a np.array with multiple t_list at each indexes:
result = np.array((t_list, t_list, ...., tlist))
I would like to be able to control the number of time t_list is in result.
I know that I could use list instead of arrays. The problem is, I put this in a numba njit function so I need to precise everything.
The aim is then to change each values in a double for loop.
The shape param of numpy.zeros can be a tuple of ints of any length, so you can create an ndarray with multiple dimensions.
e.g.:
n = 5 # or any other number that you want
result = np.zeros((n,8,8), dtype=np.float32)
I am trying to sample with replacement a base 2D numpy array with shape of (4,2) by rows, say 10 times. The final output should be a 3D numpy array.
Have tried the code below, it works. But is there a way to do it without the for loop?
base=np.array([[20,30],[50,60],[70,80],[10,30]])
print(np.shape(base))
nsample=10
tmp=np.zeros((np.shape(base)[0],np.shape(base)[1],10))
for i in range(nsample):
id_pick = np.random.choice(np.shape(base)[0], size=(np.shape(base)[0]))
print(id_pick)
boot1=base[id_pick,:]
tmp[:,:,i]=boot1
print(tmp)
Here's one vectorized approach -
m,n = base.shape
idx = np.random.randint(0,m,(m,nsample))
out = base[idx].swapaxes(1,2)
Basic idea is that we generate all the possible indices with np.random.randint as idx. That would an array of shape (m,nsample). We use this array to index into the input array along the first axis. Thus, it selects random rows off base. To get the final output with a shape (m,n,nsample), we need to swap last two axes.
You can use the stack function from numpy. Your code would then look like:
base=np.array([[20,30],[50,60],[70,80],[10,30]])
print(np.shape(base))
nsample=10
tmp = []
for i in range(nsample):
id_pick = np.random.choice(np.shape(base)[0], size=(np.shape(base)[0]))
print(id_pick)
boot1=base[id_pick,:]
tmp.append(boot1)
tmp = np.stack(tmp, axis=-1)
print(tmp)
Based on #Divakar 's answer, if you already know the shape of this 2D-array, you can treat it as an (8,) 1D array while bootstrapping, and then reshape it:
m, n = base.shape
flatbase = np.reshape(base, (m*n,))
idxs = np.random.choice(range(8), (numReps, m*n))
bootflats = flatbase[idx]
boots = np.reshape(flatbase, (numReps, m, n))
I am generating a large set of monte carlo data and ideally want to store in it an array of arrays.
When i use the array.append(x) and then cycle over a loop that returns a new array for x when I look at the elements of the array at the end they are all the same as the last array x added to the list. I believe this must be because i'm adding the memory location to the list and not the actual array data hence when I add more arrays all the other elements that point to the same location also update.
Is there anyway to prevent this by setting a kwarg or something or do i have to construct my arrays in a different way?
#test to illustrate point
import numpy as np
x = np.random.choice((-1, 1), size=(5, 5))
array_test = []
for T in range(10):
array_test.append(x)
print(x)
x += 10
print(array_test)
If i have an array:
myzeros=scipy.zeros((c*pos,c*pos)) , c=0.1 , pos=100
and an array:
grid=scipy.ones((pos,pos))
How can i insert the zeros into the grid in random positions? The problem is with the dimensions.
I know that in 1d you can do:
myzeros=sc.zeros(c*pos) # array full of (zeros)
grid=sc.ones(pos) # grid full of available positions(ones)
dist=sc.random.permutation(pos)[:c*pos] # distribute c*pos zeros in random
# positions
grid[dist]=myzeros
I tried something similar but it doesn't work. I tried also: myzeros=sc.zeros(c*pos), but it still does not work.
There are several ways, but the easiest seems to be to first convert the 2D grid into a 1D grid and proceed as in the 1D case, then convert back to 2D:
c = 0.1
pos = 100
myzeros=scipy.zeros((c*pos,c*pos))
myzeros1D = myzeros.ravel()
grid=scipy.ones((pos,pos))
grid1D = grid.ravel()
dist=sc.random.permutation(pos*pos)[:c*pos*c*pos]
grid1D[dist]=myzeros1D
myzeros = myzeros1D.reshape((c*pos,c*pos))
grid = grid1D.reshape((pos, pos))
EDIT: to answer your comment: if you only want a part of the myzeros to go into the grid array, you have to make the dist array smaller. Example:
dist = scipy.random.permutation(pos*pos)[:c*pos]
grid1D[dist] = myzeros1D[:c*pos]
And I hope you are aware, that this last line can be written as
grid1D[dist] = 0
if you really only want to set those elements to a single instead of using the elements from another array.
I have a list of several hundred 10x10 arrays that I want to stack together into a single Nx10x10 array. At first I tried a simple
newarray = np.array(mylist)
But that returned with "ValueError: setting an array element with a sequence."
Then I found the online documentation for dstack(), which looked perfect: "...This is a simple way to stack 2D arrays (images) into a single 3D array for processing." Which is exactly what I'm trying to do. However,
newarray = np.dstack(mylist)
tells me "ValueError: array dimensions must agree except for d_0", which is odd because all my arrays are 10x10. I thought maybe the problem was that dstack() expects a tuple instead of a list, but
newarray = np.dstack(tuple(mylist))
produced the same result.
At this point I've spent about two hours searching here and elsewhere to find out what I'm doing wrong and/or how to go about this correctly. I've even tried converting my list of arrays into a list of lists of lists and then back into a 3D array, but that didn't work either (I ended up with lists of lists of arrays, followed by the "setting array element as sequence" error again).
Any help would be appreciated.
newarray = np.dstack(mylist)
should work. For example:
import numpy as np
# Here is a list of five 10x10 arrays:
x = [np.random.random((10,10)) for _ in range(5)]
y = np.dstack(x)
print(y.shape)
# (10, 10, 5)
# To get the shape to be Nx10x10, you could use rollaxis:
y = np.rollaxis(y,-1)
print(y.shape)
# (5, 10, 10)
np.dstack returns a new array. Thus, using np.dstack requires as much additional memory as the input arrays. If you are tight on memory, an alternative to np.dstack which requires less memory is to
allocate space for the final array first, and then pour the input arrays into it one at a time.
For example, if you had 58 arrays of shape (159459, 2380), then you could use
y = np.empty((159459, 2380, 58))
for i in range(58):
# instantiate the input arrays one at a time
x = np.random.random((159459, 2380))
# copy x into y
y[..., i] = x