I'm trying to build a matrix in numpy. The matrix dimensions should be (5001x7). Here is my code:
S=np.array([.0788,.0455,.0222,.0042,.0035,.0029,.0007])
#This is vector S, comprised of 7 scalars.
lamb=list(range(0,5001))
#This is a list of possible values for lambda, a parameter in my data.
M = np.empty([5001,7], order='C')
#This is the empty matrix which is to be filled in the iterations below.
for i in S:
for j in lamb:
np.append(M,((S[i]**2)/(lamb[j]+S[i]**2)))
The problem I'm having is that M remains a matrix of zero vectors.
Important details:
1) I've assigned the final line as:
M=np.append(M,((S[i]**2)/(lamb[j]+S[i]**2)))
I then get an array of values of length 70,014 in a 1d array. I'm not really sure what to make of it.
2) I've already tried switching the dtype parameter between 'float' and 'int' for matrix M.
3) I receive this warning when I run the code:
VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
app.launch_new_instance()
4) I'm working in Python 3.4
I really appreciate your help. Thank you!
1) append adds to the end of the array, which is why your final array has 5001x7x2=70014 elements. Only the first half is zeros. It flattens the array to 1D because you didn't specify an axis to append.
2) A much more "numpy" way to do this whole process is broadcasting
S=np.array([.0788,.0455,.0222,.0042,.0035,.0029,.0007])
lamb=np.arange(0,5001)
M=(S[:,None]**2)/(lamb[None,:]+S[:,None]**2)
np.append makes a copy of the array and appends values to the end of the copy (making the array larger each time), whereas I think you want to modify M in place:
for i in range(len(S)):
for j in range(len(lamb)):
M[j][i] = ((S[i]**2)/(lamb[j]+S[i]**2))
Related
In order to make say it simply, I have a list of dimension [32, 31, 4] which I would like to reduce to shape [32, 31, 3] in order to replace every array in the last dimension by an array of size (3).
for a in range(len(liste)): #len(list) = 95
for b in range(len(liste[a])): #shape = [32, 31, 3], b travels in the 1st dim.
#print('frame : ', liste[a][b].shape) #[31, 4]
#print('b', b) #32 frames each time ok
for c in range(len(liste[a][b])):
#print('c', c) #31 each time ok
#print('norme du quaternion', np.abs(np.linalg.norm(liste[a][b][c]))) #norm = 1
r = quat2expmap(liste[a][b][c]) #convertion to expmap successful
#print('ExpMap : ', r)
quat = liste[a][b][c]
quat = r #this works
#print('quat', quat)
liste[a][b][c] = r #this doesn't work
To be more precise, I have a dataset of 95 different gestures each represented by 32 frames and quaternions. I converted the quaternions into ExpMap but due to the difference of shapes I am unable to replace the quaternions by their corresponding ExpMap. The error code I receive the most is the following:
ValueError: could not broadcast input array from shape (3) into shape (4)
It comes from the last line of the code.
The weirdest thing is that when I take the quaternion apart and replace it, it works parfectly, yet python would refuse that I do it inside my list. I don't really get why.
Could you lighten me about it? How could I get the proper dimension in my list? I tried all the tricks such as del, remove() but got no result...
You seem to be using numpy arrays (not Python lists). Numpy does not allow changing dimensions on assignment to an element of an array because it would become irregular (some entries with 4 and some with 3).
Also, iterating through numpy arrays using loops is the wrong way to use numpy. In this case you're probably looking at applying the quat2expmap function to the 4th dimension of your matrix to produce a new matrix of shape (95,32,31,3). This will make maximum use of numpy's parallelism and can be written in a couple of lines without any loops.
You could either modify the quat2expmap function so that it works directly on your 4d matrix (will be fastest approach) or use np.apply_along_axis (which is not much faster than loops).
Let's say I have a function (called numpyarrayfunction) that outputs an array every time I run it. I would like to run the function multiple times and store the resulting arrays. Obviously, the current method that I am using to do this -
numpyarray = np.zeros((5))
for i in range(5):
numpyarray[i] = numpyarrayfunction
generates an error message since I am trying to store an array within an array.
Eventually, what I would like to do is to take the average of the numbers that are in the arrays, and then take the average of these averages. But for the moment, it would be useful to just know how to store the arrays!
Thank you for your help!
As comments and other answers have already laid out, a good way to do this is to store the arrays being returned by numpyarrayfunction in a normal Python list.
If you want everything to be in a single numpy array (for, say, memory efficiency or computation speed), and the arrays returned by numpyarrayfunction are of a fixed length n, you could make numpyarray multidimensional:
numpyarray = np.empty((5, n))
for i in range(5):
numpyarray[i, :] = numpyarrayfunction
Then you could do np.average(numpyarray, axis = 1) to average over the second axis, which would give you back a one-dimensional array with the average of each array you got from numpyarrayfunction. np.average(numpyarray) would be the average over all the elements, or np.average(np.average(numpyarray, axis = 1)) if you really want the average value of the averages.
More on numpy array indexing.
I initially misread what was going on inside the for loop there. The reason you're getting an error is because numpy arrays will only store numeric types by default, and numpyarrayfunction is returning a non-numeric value (from the name, probably another numpy array). If that function already returns a full numpy array, then you can do something more like this:
arrays = []
for i in range(5):
arrays.append(numpyarrayfunction(args))
Then, you can take the average like so:
avgarray = np.zeros((len(arrays[0])))
for array in arrays:
avgarray += array
avgarray = avgarray/len(arrays)
I have an array of 2d indices.
indices = [[2,4], [6,77], [102,554]]
Now, I have a different 4-dimensional array, arr, and I want to only extract an array (it is an array, since it is 4-dimensional) with corresponding index in the indices array. It is equivalent to the following code.
for i in range(len(indices)):
output[i] = arr[indices[i][0], indices[i][1]]
However, I realized that using explicit for-loop yields a slow result. Is there any built-in numpy API that I can utilized? At this point, I tried using np.choose, np.put, np.take, but did not succeed to yield what I wanted. Thank you!
We need to index into the first two axes with the two columns from indices (thinking of it as an array).
Thus, simply convert to array and index, like so -
indices_arr = np.array(indices)
out = arr[indices_arr[:,0], indices_arr[:,1]]
Or we could extract those directly without converting to array and then index -
d0,d1 = [i[0] for i in indices], [i[1] for i in indices]
out = arr[d0,d1]
Another way to extract the elements would be with conversion to tuple, like so -
out = arr[tuple(indices_arr.T)]
If indices is already an array, skip the conversion process and use indices in places where we had indices_arr.
Try using the take function of numpy arrays. Your code should be something like:
outputarray= np.take(arr,indices)
Hi I'm stuck on what on the face of it seems a simple problem, so I must be missing something!
I have a list (of indeterminate length) of matrices calculated from user values. - ttranspose
I also have another single matrix, Qbar which I would like to multiply (matrix form) each of the matrices in ttranspose, and output a list of the resultant matrices. << Which should be the same length as ttranspose.
def Q_by_transpose(ttranspose, Qmatrix):
Q_by_transpose = []
for matrix in ttranspose:
Q_by_transpose_ind = np.matmul(ttranspose, Qmatrix)
Q_by_transpose.append(Q_by_transpose_ind)
return (Q_by_transpose)
Instead when I test this with a list of 6 matrices (ttranspose) I get the a long list of mtrices, which appears to be in 6 arrays (as expected) but each array is made up of 6 matrices?
Im hoping to create a list of matrices for which I would then perform elementwise multiplication between this and another list. So solving this will help on both fronts!
Any help would be greatly appreciated!
I am new to Python and Numpy so am hopeful you guys will be able to help!
Thanks
It appears that instead of passing a single matrix to the np.matmul function, you are passing the entire list of matrices. Instead of
for matrix in ttranspose:
Q_by_transpose_ind = np.matmul(ttranspose, Qmatrix)
Q_by_transpose.append(Q_by_transpose_ind)
do this:
for matrix in ttranspose:
Q_by_transpose_ind = np.matmul(matrix, Qmatrix)
Q_by_transpose.append(Q_by_transpose_ind)
This will only pass one matrix to np.matmul instead of the whole list. Essentially what you're doing right now is multiplying the entire list of matrices n times, where n is the number of matrices in ttranspose.
I am new to python, so please, bear with me!
This function:
def kerf(X,m):
[n_samples, ]= X.shape
n_sa, n_fe = m.shape
ker = np.zeros((n_samples, n_sa))
for i, x_i in enumerate(X):
for j, m_j in enumerate(m):
ker[i, j] = (np.dot(x_i, m_j)) # problem is here!!
return ker
I call it like this:
Z=kerf(myarray[0,[0,1]],myarray[:,[0,1]])
ker[i, j] = (np.dot(x_i, m_j))
ValueError: setting an array element with a sequence.
myarray is basically the same matrix. Why?
When I replace the problem line with:
print(np.dot(x_i, m_j).shape)
it repeatedly prints (2,).
ker[i, j] takes 1 value; 2 values is sequence.
Please give us the dimensions of the arrays at various points, such as myarray (I guessed and tried a (3,4)), and at the problem point. print(...shape) is an essential debugging tool in numpy.
Do you need help on figure out why it's (2,)? May I suggest stepping through the loop in an interactive shell, looking at shapes at various points along the way.
the 2 inputs to the dot look like:
(1.0, array([ 1., 1.]))
a scalar, and a 2 element array - so the dot is also a 2 element array.
You need to explain what size you expect these 2 arrays to be, and what size you expect the dot. Actually we can get the result - it's got to be (1,) or a scalar - 1 value to put in the one slot ker.
You can probably replace the double iteration with a single dot product (or if need be with an einsum call). But let's get this iteration working first.