I'm trying to do a grid search over a model I've trained. So, producing a mesh, then predicting that mesh with the model to find a maximum.
I'm producing the mesh with:
def generate_random_grid(n_scanning_parameters,n_points_each_dimension):
points=np.linspace(1,0,num=n_points_each_dimension,endpoint=False)
x_points=[points for dimension in range(n_scanning_parameters)]
mesh=np.array(np.meshgrid(*x_points))
return mesh
As you can see, I don't know the dimensions in advance. So later when I want to index the mesh to predict different points, I don't know how to index.
E.g, if I have 4 dimensions and 10 points along each dimension, the mesh has the shape (4,10,10,10,10). And I need to access points like e.g. [:,0,0,0,0] or [:,1,2,3,4]. Which would give me a 1-D vector with 4 elements.
Now I can produce the 4 last indices using
for index in np.ndindex(*mesh.shape[1:]):
, but then indexing my mesh like mesh[:,index] doesn't result in a 1-D vector with 4 elements as I expect it to.
How can I index the mesh?
Since you're working with tuples, and numpy supports tuple indexing, let's start with that.
Effectively, you want to do your slicing like a[:, 0, 0, 0, 0]. But your index is a tuple, and you're attempting something like a[:, (0,0,0,0)] - this gives you four hyperplanes along the second dimension instead. Your indexing should be more like a[(:,0,0,0,0)] - but this gives a syntax error.
So the solution would be to use the slice built-in.
a[(slice(None),0,0,0,0)]
This would give you your one dimensional vector.
In terms of your code, you can simply add the tuples to make this work.
for index in np.ndindex(*mesh.shape[1:]):
vector = mesh[(slice(None), ) + index]
An alternative approach would be to simply use a transposed array and reversed indices. The first dimension is at the end, removing the need for :.
for index in np.ndindex(*mesh.shape[1:]):
vector = mesh.T[index[::-1]]
Related
I have a (square) 2 dimensional numpy array where I would like to compare (subtract) all of the values within each row to each other but not to other rows so the output should be a 3D array.
matrix = np.array([[10,1,32],[32,4,15],[6,3,1]])
Output should be a 3x3x3 array which looks like:
output = [[[0,-9,22],[0,-28,-17],[0,-3,-5]], [[9,0,31],[28,0,11],[3,0,-2]], [[-22,-31,0],[17,-11,0],[5,2,0]]]
I.e. for output[0], for each of the 3 rows of matrix, subtract that row's zeroth element from every other, for output[1] subtract each row's first element etc.
This seems to me like a reduced version of numpy's ufunc.outer functionality which should be possible with
tryouter = np.subtract(matrix, matrix)
and then taking some clever slice and/or transposition.
Indeed, if you do this, one finds that: output[i,j] = tryouter[i,j,i]
This looks like it should be solvable by using np.transpose to switch the 1 and 2 axes and then taking the arrays on the new 0,1 diagonal but I can't work out how to do this with numpy diagonal or any slicing method.
Is there a way to do this or is there a simpler approach to this whole problem built into numpy?
Thanks :)
You're close, you can do it with broadcasting:
out = matrix[None, :, :] - matrix.T[:, :, None]
Here .T is the same as np.transpose, and using None as an index introduces a new dummy dimension of size 1.
My goal is to to turn a row vector into a column vector and vice versa. The documentation for numpy.ndarray.transpose says:
For a 1-D array, this has no effect. (To change between column and row vectors, first cast the 1-D array into a matrix object.)
However, when I try this:
my_array = np.array([1,2,3])
my_array_T = np.transpose(np.matrix(myArray))
I do get the wanted result, albeit in matrix form (matrix([[66],[640],[44]])), but I also get this warning:
PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
my_array_T = np.transpose(np.matrix(my_array))
How can I properly transpose an ndarray then?
A 1D array is itself once transposed, contrary to Matlab where a 1D array doesn't exist and is at least 2D.
What you want is to reshape it:
my_array.reshape(-1, 1)
Or:
my_array.reshape(1, -1)
Depending on what kind of vector you want (column or row vector).
The -1 is a broadcast-like, using all possible elements, and the 1 creates the second required dimension.
If your array is my_array and you want to convert it to a column vector you can do:
my_array.reshape(-1, 1)
For a row vector you can use
my_array.reshape(1, -1)
Both of these can also be transposed and that would work as expected.
IIUC, use reshape
my_array.reshape(my_array.size, -1)
This is for a Machine Learning problem (in Python of course).
I have a 2 dimensional array, the rows are set of points, and the columns are indices into another 1 dimensional array of values for those points.
data = [[1,3,2], [3,3,1], [5,1,2]]
# yes there are duplicates in the labels
labels = [2,8,9,8,8,9]
What I need is to create a 2D array that is the original data array, but where the values in it are now the value from labels that the index represented.
new_data = [[8,8,9], [8,8,8], [9,8,9]]
I can do this with for loops obviously. I'm asking here in case numpy or something has a call that does this.
Use the indices as indices:
np.array(labels)[np.array(data)]
The output of an advanced (integer) index is the shape of the index array (data).
I got a np.ndarray with ~3000 trajectories. Each trajectory has x, y and z coordinates and a different length; between 150 and 250 (points in time). Now I want to remove the z coordinate for all of these trajectories.
So arr.shape gives me (3000,),(3000 trajectories) and (for example) arr[0].shape yields (3,178) (three axis of coordinates and 178 values).
I have found multiple explanations for removing lines in 2D-arrays and I found np.delete(arr[0], 2, axis=0) working for me. However, I don't just want to delete the z coordinates for the first trajectory; I want to do this for every trajectory.
If I want to do this with a loop for arr[i] I would need to know the exact length of every trajectory (It doesn't suit my purpose to just create the array with the length of the longest and fill it up with zeroes).
TL;DR: So how do I get from a ndarray with [amountOfTrajectories][3][value] to [amountOfTrajectories][2][value]?
The purpose is to use these trajectories as labels for a neural net that creates trajectories. So I guess it's a entirely new question but is the shape I'm asking for suitable for usage as labels for tensorflow?
Also: What would have been a better title and some terms to find results for this with google? I just started with Python and I'm afraid I'm missing some keywords here...
If this comes from loadmat, the source is probably a MATLAB workspace with a cell, which contains these matrices.
loadmat has, evidently created a 1d array of object dtype (the equivalent of a cell, with squeeze on).
A 1d object array is similar to a Python list - it contains pointers to arrays else where in memory. Most operations on such an array use Python iteration. Iterating on the equivalent list is usually faster. (arr.tolist()).
alist = [a[:2,:] for a in arr]
should give you a list of arrays, each of shape (2, n) (n varying). This makes new arrays - but then so does np.delete.
You can't operate on all arrays in the 1d array with one operation. It has to be iterative.
I'm still getting the hang of working with numpy and array-wise operations.
I'm looking for the way of getting the row-wise average of a list of 2D arrays.
E.g I have a 4x3x25 array and I'm looking to get a 3x25 array of the row-wise averages.
If everything’s in one 3D array already, you can just do:
A.mean(axis=0)
…which will operate along the first dimension.
If it’s actually just a list of 2D arrays, you’ll have to convert it to a 3D array first. I would do:
A = np.dstack(list_of_arrays) # Combine the 2D arrays along a new 3rd dimension
A.mean(axis=2) # Calculate the means along that new dimension