Suppose I have multiple NxN 2D arrays stored into a list in Python 3. I want to collapse all the arrays into 1 array, with the same dimensions NxN, but such that each element of this new array contains a 1xN array of the corresponding values from the original arrays.
To give you some more context, each array in this list corresponds to the set of values at a given time. For each new time point, I am storing the updated version of that array into the list. Once that's done, I want to compute the standard deviation of the values at each (i,j) element in the array.
I tried using a for loop, but it takes far too long for my simulations because this is a set of 100,000 arrays. I was wondering if there were any numpy or vectorized functions that can help me perform this operation more efficiently. Thanks!
Lets say l is your list of arrays. You need to get std of corresponding elements of those arrays into a single array:
std_l = np.std(np.stack(l),axis=0)
Related
lets say I have 10 arrays of size 10x10 each, called a,b,c...j
I create np.zeros of 10x2 called arr. I've already made arr[:,1]=1,2,3...10
so as of now my new array is
[0,1]
[0,2]
...
[0,10]
What i need now is to make
arr[0,0] = a, arr[1,0] = b ... arr[10,0]=j
Since Im also dealing tensors, the error I get is
"only one element tensors can be converted to Python scalars"
Basically, to reiterate, I want to put those 2D arrays into my new array as individual elements of the new array.
Final output reqd:
[a,1]
[b,2]
...
[j,10]
EDIT: The final array will be multidimensional, but Im not sure of the value of the final shape, 10x10x2 maybe??
EDIT: Think of the problem as images and labels. I want to store an entire image and its corresponding label as each row of an array
Let's say I have a function (called numpyarrayfunction) that outputs an array every time I run it. I would like to run the function multiple times and store the resulting arrays. Obviously, the current method that I am using to do this -
numpyarray = np.zeros((5))
for i in range(5):
numpyarray[i] = numpyarrayfunction
generates an error message since I am trying to store an array within an array.
Eventually, what I would like to do is to take the average of the numbers that are in the arrays, and then take the average of these averages. But for the moment, it would be useful to just know how to store the arrays!
Thank you for your help!
As comments and other answers have already laid out, a good way to do this is to store the arrays being returned by numpyarrayfunction in a normal Python list.
If you want everything to be in a single numpy array (for, say, memory efficiency or computation speed), and the arrays returned by numpyarrayfunction are of a fixed length n, you could make numpyarray multidimensional:
numpyarray = np.empty((5, n))
for i in range(5):
numpyarray[i, :] = numpyarrayfunction
Then you could do np.average(numpyarray, axis = 1) to average over the second axis, which would give you back a one-dimensional array with the average of each array you got from numpyarrayfunction. np.average(numpyarray) would be the average over all the elements, or np.average(np.average(numpyarray, axis = 1)) if you really want the average value of the averages.
More on numpy array indexing.
I initially misread what was going on inside the for loop there. The reason you're getting an error is because numpy arrays will only store numeric types by default, and numpyarrayfunction is returning a non-numeric value (from the name, probably another numpy array). If that function already returns a full numpy array, then you can do something more like this:
arrays = []
for i in range(5):
arrays.append(numpyarrayfunction(args))
Then, you can take the average like so:
avgarray = np.zeros((len(arrays[0])))
for array in arrays:
avgarray += array
avgarray = avgarray/len(arrays)
Hi I'm stuck on what on the face of it seems a simple problem, so I must be missing something!
I have a list (of indeterminate length) of matrices calculated from user values. - ttranspose
I also have another single matrix, Qbar which I would like to multiply (matrix form) each of the matrices in ttranspose, and output a list of the resultant matrices. << Which should be the same length as ttranspose.
def Q_by_transpose(ttranspose, Qmatrix):
Q_by_transpose = []
for matrix in ttranspose:
Q_by_transpose_ind = np.matmul(ttranspose, Qmatrix)
Q_by_transpose.append(Q_by_transpose_ind)
return (Q_by_transpose)
Instead when I test this with a list of 6 matrices (ttranspose) I get the a long list of mtrices, which appears to be in 6 arrays (as expected) but each array is made up of 6 matrices?
Im hoping to create a list of matrices for which I would then perform elementwise multiplication between this and another list. So solving this will help on both fronts!
Any help would be greatly appreciated!
I am new to Python and Numpy so am hopeful you guys will be able to help!
Thanks
It appears that instead of passing a single matrix to the np.matmul function, you are passing the entire list of matrices. Instead of
for matrix in ttranspose:
Q_by_transpose_ind = np.matmul(ttranspose, Qmatrix)
Q_by_transpose.append(Q_by_transpose_ind)
do this:
for matrix in ttranspose:
Q_by_transpose_ind = np.matmul(matrix, Qmatrix)
Q_by_transpose.append(Q_by_transpose_ind)
This will only pass one matrix to np.matmul instead of the whole list. Essentially what you're doing right now is multiplying the entire list of matrices n times, where n is the number of matrices in ttranspose.
I have two arrays in Python (numpy arrays):
a=array([5,7,3,5])
b=array([1,2,3,4])
and I wish to create a third array with each element from b appearing a times in the new array, as:
c=array([1,1,1,1,1,2,2,2,2,2,2,2,3,3,3,4,4,4,4,4])
Is there a fast, numPythonic way of doing this with a minimum of looping? I need to use this operation thousands of times in a loop over a fairly large array, so I would like to have it be as fast as possible.
Cheers,
Mike
I believe repeat is what you want:
c = repeat(b, a)
In Matlab, there is something called struct, which allow the user to have a dynamic set of matrices.
I'm basically looking for a function that allows me to index over dynamic matrices that have different sizes.
Example: (with 3 matrices)
Matrix 1: 3x2
Matrix 2: 2x2
Matrix 3: 2x1
Basically I want to store the 3 matrices on the same variable. To called them by their index number afterward (i.e. Matrix[1], Matrx[2]). Conventional python arrays do not allow for arrays with different dimensions to be stacked.
I was looking into creating classes, but maybe someone her has a better alternative to this.
Thanks
Just use a tuple or list.
A tuple matrices = tuple(matrix1, matrix2, matrix3) will be slightly more efficient;
A list matrices = [matrix1, matrix2, matrix3] is more flexible as you can matrix.append(matrix4).
Either way you can access them as matrices[0] or for matrix in matricies: pass # do stuff.
Put those arrays into a list.