In Python/Numpy I can slice arrays in this form:
arr = np.ones((3,4,5))
arr[2]
and the shape will be maintained:
(arr[2]).shape # prints (4, 5)
Which means that, if I want to keep the shape of the array, the following code works for N-dimensional arrays
arr = np.ones((3,4,5,2,2))
(arr[2]).shape # prints (4, 5, 2, 2)
This is great if I want to write functions that work for N-dim arrays preserving their output.
In Julia, however, the same action does not preserve the structure:
arr = ones(3,4,5)
size(arr[3]) # prints () (0-dimensinoal)
size(arr[3,:]) # prints (20,)
because of partial linear indexing. So if want to keep the original dimensions I need to write arr[3,:,:], which only works for 3D arrays. If I want a 4D array I would have to use arr[3,:,:,:] and so on. The code isn't general.
Furthermore, when you get to array that are 5 dimensions or more (which is the case I'm working with now) this notation gets extremely cumbersome.
Is there any way I can write code like I do in Python and make it general? I couldn't even think of a nice clean way with reshape, let alone a way that's as clean as Python.
Notice that in Python the shape is only preserved if you slice the first dimension of the array. In Julia you can use slicedim(A, d, i) to slice dimension d of array A at index i.
Related
I'm just learning python, but have decided to do so by recoding and improving some old java based school AI project.
My project involved a mathematical operation that is basically a discrete convolution operation, but without one of the functions time reversed.
So, while in my original java project I just wrote all the code to do the operation myself, since I'm working in python, and it's got great math libraries like numpy and scipy, I figured I could just make use of an existing convolution function like scipy.convolve. However, this would require me to pre-reverse one of the two arrays so that when scipy.convolve runs, and reverses one of the arrays to perform the convolution, it's really un-reversing the array. (I also still don't know how I can be sure to pre-reverse the right one of the two arrays so that the two arrays are still slid past each other both forwards rather than both backwards, but I assume I should ask that as a separate question.)
Unlike my java code, which only handled one dimensional data, I wanted to extend this project to multidimensional data. And so, while I have learned that if I had a numpy array of known dimension, such as a three dimensional array a, I could fully reverse the array (or rather get back a view that is reversed, which is much faster), by
a = a(::-1, ::-1, ::-1)
However, this requires me to have a ::-1 for every dimension. How can I perform this same reversal within a method for an array of arbitrary dimension that has the same result as the above code?
You can use np.flip. From the documentation:
numpy.flip(m, axis=None)
Reverse the order of elements in an array along the given axis.
The shape of the array is preserved, but the elements are reordered.
Note: flip(m) corresponds to m[::-1,::-1,...,::-1] with ::-1 at all positions.
This is a possible solution:
slices = tuple([slice(-1, -n-1, -1) for n in a.shape])
result = a[slices]
extends to arbitrary number of axes. Verification:
a = np.arange(8).reshape(2, 4)
slices = tuple([slice(-1, -n-1, -1) for n in a.shape])
result = a[slices]
yields:
>>> a
array([[0, 1, 2, 3],
[4, 5, 6, 7]])
>>> result
array([[7, 6, 5, 4],
[3, 2, 1, 0]])
I find this behaviour an utter nonsense. This happens only with numpy arrays, typical Python's arrays will just throw an error.
Let's create two arrays:
randomNumMatrix = np.random.randint(0,20,(3,3,3), dtype=np.int)
randRow = np.array([0,1,2], dtype=np.int)
If we pass an array as index to get something from another array, an original array is returned.
randomNumMatrix[randRow]
The code above returns an equivalent of randomNumMatrix. I find this unintuitive. I would expect it, not to work or at least return an equivalent of
randomNumMatrix[randRow[0]][randRow[1]][randRow[2]].
Additional observations:
A)
The code below does not work, it throws this error: IndexError: index 3 is out of bounds for axis 0 with size 3
randRow = np.array([0, 1, 3], dtype=np.int)
B)
To my surprise, the code below works:
randRow = np.array([0, 1, 2, 2,0,1,2], dtype=np.int)
Can somebody please explain what are the advantages of this feature?
In my opinion it only creates much confusion.
What is?
randomNumMatrix[randRow[0]][randRow[1]][randRow[2]]
That's not a valid Python.
In numpy there is a difference between
arr[(x,y,z)] # equivalent to arr[x,y,z]
and
arr[np.array([x,y,z])] # equivalent to arr[np.array([x,y,z]),:,:]
The tuple provides a scalar index for each dimension. The array (or list) provides multiple indices for one dimension.
You may need to study the numpy docs on indexing, especially advanced indexing.
In order to make say it simply, I have a list of dimension [32, 31, 4] which I would like to reduce to shape [32, 31, 3] in order to replace every array in the last dimension by an array of size (3).
for a in range(len(liste)): #len(list) = 95
for b in range(len(liste[a])): #shape = [32, 31, 3], b travels in the 1st dim.
#print('frame : ', liste[a][b].shape) #[31, 4]
#print('b', b) #32 frames each time ok
for c in range(len(liste[a][b])):
#print('c', c) #31 each time ok
#print('norme du quaternion', np.abs(np.linalg.norm(liste[a][b][c]))) #norm = 1
r = quat2expmap(liste[a][b][c]) #convertion to expmap successful
#print('ExpMap : ', r)
quat = liste[a][b][c]
quat = r #this works
#print('quat', quat)
liste[a][b][c] = r #this doesn't work
To be more precise, I have a dataset of 95 different gestures each represented by 32 frames and quaternions. I converted the quaternions into ExpMap but due to the difference of shapes I am unable to replace the quaternions by their corresponding ExpMap. The error code I receive the most is the following:
ValueError: could not broadcast input array from shape (3) into shape (4)
It comes from the last line of the code.
The weirdest thing is that when I take the quaternion apart and replace it, it works parfectly, yet python would refuse that I do it inside my list. I don't really get why.
Could you lighten me about it? How could I get the proper dimension in my list? I tried all the tricks such as del, remove() but got no result...
You seem to be using numpy arrays (not Python lists). Numpy does not allow changing dimensions on assignment to an element of an array because it would become irregular (some entries with 4 and some with 3).
Also, iterating through numpy arrays using loops is the wrong way to use numpy. In this case you're probably looking at applying the quat2expmap function to the 4th dimension of your matrix to produce a new matrix of shape (95,32,31,3). This will make maximum use of numpy's parallelism and can be written in a couple of lines without any loops.
You could either modify the quat2expmap function so that it works directly on your 4d matrix (will be fastest approach) or use np.apply_along_axis (which is not much faster than loops).
Hello I have the following question. I create zero arrays of dimension (40,30,80). Now I need 7*7*7 of these zero arrays in an array. How can I do this?
One of my matrices is created like this:
import numpy as np
zeroMatrix = np.zeros((40,30,80))
My first method was to put the zero matrices in a 7*7*7 list. But i want to have it all in a numpy array. I know that there is a way with structured arrays I think, but i dont know how. If i copy my 7*7*7 list with np.copy() it creates a numpy array with the given shape, but there must be a way to do this instantly, isnt there?
EDIT
Maybe I have to make my question clearer. I have a 7*7 list of my zero matrices. In a for loop all of that arrays will be modified. In another step, this tempory list is appended to an empty list which will have a length of 7 in the end ( So i append the 7*7 list 7 times to the empty list. In the end I have a 7*7*7 List of those matrices. But I think this will be better If I have a numpy array of these zero matrices from the beginning.
Building an array of same-shaped arrays is not well supported by numpy which prefers to create a maximum depth array of minimum depth elements instead.
It turns out that numpy.frompyfunc is quite useful in circumventing this tendency where it is unwanted.
In your specific case one could do:
result = np.frompyfunc(zeroMatrix.copy, 0, 1)(np.empty((7, 7, 7), object))
Indeed:
>>> result.shape
(7, 7, 7)
>>> result.dtype
dtype('O')
>>> result[0, 0, 0].shape
(40, 30, 80)
I have a 2-d numpy array that I convert into a 3-d array by using:
trainSet = numpy.reshape(trainSet, (trainSet.shape[0], 1, trainSet.shape[1]))
My problem is I'm trying to adjust values in the 1st axis (the new one). However, no matter what I try I can not actually figure out how to access the 1's in the new axis. I've tried a multitude of print statements such as
print(trainSet[0][0][0])
print(trainSet[1][0][1])
but no matter what I try I can not print out any 1's, just the contents of the 2-d array. I know it's a 3d array because when I use
print(trainSet.shape)
I get
(12, 1, 793)
Any help would be immensely appreciated!
The one refers to the size of that particular dimension, not the content of the array. The array [[5 5]] has shape (1, 2) but, still, you don't really expect its values must include 1 for that reason, do you?
What inserting a dimension of length one does is increasing the nesting level by one, so a 2d array [[a b] [c d]] will become [[[a b] [c d]]] or [[[a b]] [[c d]]] or [[[a] [b]] [[c] [d]]] depending on where you insert the new axis.
In particular, there are no new elements.
If I didn't mistake your question, what you want is
trainSet[:,0,:]
the amount of values remains the same, you're just creating a 3D structure instead of the 2D structure.