For the array:
import numpy as np
arr2d = np.array([[1,2,3],[4,5,6],[7,8,9]])
>>> arr2d
array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
>>> arr2d[2].shape
(3,)
>>> arr2d[2:,:].shape
(1, 3)
Why do I get different shapes when both statements return the 3rd row? and shouldn't the result be (1,3) in both cases since we are returning a single row with 3 columns?
Why do I get different shapes when both statements return the 3rd row?
Because with the first operation you are indexing the rows, and selecting just ONE element, which -as mentioned in the single-element indexing paragraph of a multidimensional array- returns an array with a lower dimension (a 1D array).
In the 2nd example, you are using a slice as evident by the colon. Slicing operations do not reduce the dimensions of an array. This is also logical, because imagine the array would not have 3 but 4 rows. Then arr2d[2:,:].shape would be (2,3). The developers of numpy made slicing operations consistent and therefor they (slices) never reduce the number of dimensions of the array.
and shouldn't the result be (1,3) in both cases since we are returning a single row with 3 columns?
No, just because of the previous reasons.
When doing arr2d[2], you are taking a row out of the array;
While when doing arr2d[2:, :], you are taking a subset of rows out of the array ('slicing'), in this case being the rows starting from the 3rd to the end, which is only the 3rd, but it didn't change that you are taking a subset, not an element.
Related
An example is shown as follows:
>>> import numpy as np
>>> a=np.zeros((288,512))
>>> x1,x2,y1,y2=0,16,0,16
>>> p=a[x1:x2][y1:y2]
>>> p.shape
(16, 512)
>>> p=a[x1:x2,y1:y2]
>>> p.shape
I try to query a patch from an array, ranging from columns 0 to 16, and rows 0 to 16. I index the array in two ways and get very different result. a[x1:x2][y1:y2] gives me the wrong result.
Why?
Thx for helping me!!!
When you do a[x1:x2][y1:y2], you are slicing by rows twice. That is, a[x1:x2] will give you a shape (16,512). The second slice operation in a[x1:x2][y1:y2] is slicing the result of the first operation and will give you the same result.
In the second case, when you do a[x1:x2,y1:y2], you are slicing by the two dimensions of your 2-dimensional array.
Important note: If you have a 2-dimensional array and you slice like this:
a = np.zeros((10,15))
a[1:3].shape
Output:
(2, 15)
you will slice only by rows. Your resulting array will have 2 rows and the total number of columns (15 columns). If you want to slice by rows and columns, you will have to use a[1:3, 1:3].
The two methods of indexing you tried are not equivalent. In the first one (a[x1:x2][y1:y2]), you are essentially indexing the first axis twice. In the second, you are indexing the first and second axes.
a[x1:x2][y1:y2] can be rewritten as
p = a[x1:x2] # result still has two dimensions
p = p[y1:y2]
You are first indexing 0:16 in the first dimension. Then you index 0:16 in the first dimension of the result of the previous operation (which will simply return the same as a[x1:x2] because x1==y1 and x2==y2).
In the second method, you index the first and second dimensions directly. I would not write it this way, but one could write it like this to contrast it with the first method:
a[x1:x2][:, y1:y2]
Here is the sample numpy array
new2 = np.array([[0, np.array([ 4928722, 3922609, 14413953, 10103423, 8948498])],
[1,
np.array([12557217, 5572869, 13415223, 2532000, 14609022, 9830632,
9800679, 7504595, 10752682])],
[2,
np.array([10458710, 7176517, 10268240, 4173086, 8617671, 4674075,
12580461, 2434641, 3694004, 9734870, 1314108, 8879955,
6597761, 7034485, 3008940, 9816877, 1748801, 10159466,
2745090, 14842579, 788308, 5984365])],
[62711, np.array([ 6159359, 5003282, 11818909, 11760670])],
[62712,
np.array([ 4363069, 8566447, 9547966, 14554871, 2108131, 12207856,
14840255, 13087558])],
[62713,
np.array([11252023, 8710787, 4233645, 11415316, 13888594, 7410770,
13672430, 6677251, 10431890, 3447966, 12675925, 729773])]] )
I want to extract only the 2nd element in each row; the varying length numpy arrays. I want these vary length numpy arrays to be in it's own numpy array.
I tried doing this
new2[:][1]
Which usually would mean that to contain all row indexes, and column index of 1. But for some reason it results exactly the same as new2[1]. The result is
array([1,
array([12557217, 5572869, 13415223, 2532000, 14609022, 9830632,
9800679, 7504595, 10752682])], dtype=object)
Only one row, which contains both the int and the numpy array.
new2[:] selects the entire array which is why new[:][1] selects the second row. arr[X,Y] and arr[X][Y] only yield the same result if X is a number. As far as I can tell, the only legit reason for using arr[X][Y] is if you want to support arrays and nested lists etc. with the same code. Failing that arr[X,Y] should always be preferred. So use new2[:,1].
I am trying to use indices stored in one set of arrays (indexPositions) to perform a simple array operation using a matrix. It is easier to explain with an example
u[(indexPositions[:,1]):(indexPositions[:,2]),(indexPositions[:,0])]=0
The object u is a big matrix whose values I want to set to zero for a given region of space. indexPositions[:,1] contains the 'lower bound' indices and indexPositions[:,2] contains the 'upper bound' indices. This reflects the fact that I want to set to zero anything in between them and therefore want to iterate between these indices.
indexPositions[:,0] contains the column index for which the aforementioned range of rows must be set to zero.
I do not understand why it is not possible to do this (I hope its clear what I'm trying to achieve). I'm sure it has something to do with python not understanding what order its supposed to do these operations in. Is there a way of specifying this? The matrix is quite huge and these operations are happening many many times so I really don't want to use a slow python loop.
Just to make sure we are talking about the same thing, I'll create a simple example:
In [77]: u=np.arange(16).reshape(4,4)
In [78]: I=np.array([[0,2,3],[1,4,2]])
In [79]: i=0
In [80]: u[I[i,0]:I[i,1],I[i,2]]
Out[80]: array([3, 7])
In [85]: i=1
In [86]: u[I[i,0]:I[i,1],I[i,2]]
Out[86]: array([ 6, 10, 14])
I'm using different column order for I, but that doesn't matter.
I selecting 2 elements from the 4th column, and 3 from the 3rd. Different lengths of results suggests that I'll have problems operation with both rows of I at once. I might have to operate on a flattened view of u.
In [93]: [u[slice(x,y),z] for x,y,z in I]
Out[93]: [array([3, 7]), array([ 6, 10, 14])]
If the lengths of the slices are all the same it's more likely that I'd be able to do all with out a loop on I rows.
I'll think about this some more, but I just want to make sure I understood the problem, and why it might be difficult.
1u[I[:,0]:I[:,1],I[:,2]] with : in the slice is defintely going to be a problem.
In [90]: slice(I[:,0],I[:,1])
Out[90]: slice(array([0, 1]), array([2, 4]), None)
Abstractly a slice object accepts arrays or lists, but the numpy indexing does not. So instead of one complex slice, you have to create 2 or more simple ones.
In [91]: [slice(x,y) for x,y in I[:,:2]]
Out[91]: [slice(0, 2, None), slice(1, 4, None)]
I've answered a similar question, one where the slice starts came from a list, but all slices had the same length. i.e. 0:3 from the 1st row, 2:5 from the 2nd, 4:7 from the 3rd etc.
Access multiple elements of an array
How can I select values along an axis of an nD array with an (n-1)D array of indices of that axis?
If the slices are all the same length, then it is possible to use broadcasting to construct the indexing arrays. But in the end the indexing will still be with arrays, not slices.
Fast slicing of numpy array multiple times
Numpy Array Slicing
deal with taking multiple slices from a 1d array, slices with differing offsets and lengths. Your problem could, I think, be cast that way. The alterantives considered all require a list comprehension to construct the slice indexes. The indexes can then be concatenated, followed by one indexing operation, or alteratively, index multiple times and concanentate the results. Timings vary with the number and length of the slices.
An example, adapted from those earlier questions, of constructing a flat index list is:
In [130]: il=[np.arange(v[0],v[1])+v[2]*u.T.shape[1] for v in I]
# [array([12, 13]), array([ 9, 10, 11])]
In [132]: u.T.flat[np.concatenate(il)]
# array([ 3, 7, 6, 10, 14])
Same values as my earlier examples, but in 1 list, not 2.
If the slice arrays have same length, then we can get back an array
In [145]: I2
Out[145]:
array([[0, 2, 3],
[1, 3, 2]])
In [146]: il=np.array([np.arange(v[0],v[1]) for v in I2])
In [147]: u[il,I2[:,2]]
Out[147]:
array([[ 3, 6],
[ 7, 10]])
In this case, il = I2[:,[0]]+np.arange(2) could be used to construct the 1st indexing array instead of the list comprehension (this is the broadcasting I mentioned earlier).
Does numpy have the cell2mat function? Here is the link to matlab. I found an implementation of something similar but it only works when we can split it evenly. Here is the link.
In a sense Python has had 'cells' at lot longer than MATLAB - list. a python list is a direct substitute for a 1d cell (or rather, cell with size 1 dimension). A 2d cell could be represented as a nested list. numpy arrays with dtype object also work. I believe that is what scipy.io.loadmat uses to render cells in .mat files.
np.array() converts a list, or lists of lists, etc, to a ndarray. Sometimes it needs help specifying the dtype. It also tries to render the input to as high a dimensional array as possible.
np.array([1,2,3])
np.array(['1',2,'abc'],dtype=object)
np.array([[1,2,3],[1,2],[3]])
np.array([[1,2],[3,4]])
And MATLAB structures map onto Python dictionaries or objects.
http://docs.scipy.org/doc/scipy/reference/generated/scipy.io.loadmat.html
loadmat can also represent structures as numpy structured (record) arrays.
There is np.concatenate that takes a list of arrays, and its convenience derivatives vstack, hstack, dstack. Mostly they tweak the dimensions of the arrays, and then concatenate on one axis.
Here's a rough approximation to the MATLAB cell2mat example:
C = {[1], [2 3 4];
[5; 9], [6 7 8; 10 11 12]}
construct ndarrays with same shapes
In [61]: c11=np.array([[1]])
In [62]: c12=np.array([[2,3,4]])
In [63]: c21=np.array([[5],[9]])
In [64]: c22=np.array([[6,7,8],[10,11,12]])
Join them with a combination of hstack and vstack - i.e. concatenate along the matching axes.
In [65]: A=np.vstack([np.hstack([c11,c12]),np.hstack([c21,c22])])
# or A=np.hstack([np.vstack([c11,c21]),np.vstack([c12,c22])])
producing:
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
Or more generally (and compactly)
In [75]: C=[[c11,c12],[c21,c22]]
In [76]: np.vstack([np.hstack(c) for c in C])
I usually use object arrays as a replacement for Matlab's cell arrays. For example:
cell_array = np.array([[np.arange(10)],
[np.arange(30,40)] ],
dtype='object')
Is a 2x1 object array containing length 10 numpy array vectors. I can perform the cell2mat functionality by:
arr = np.concatenate(cell_array).astype('int')
This returns a 2x10 int array. You can change .astype('int') to be whatever data type you need, or you could grab it from one of the objects in your cell_array,
arr = np.concatenate(cell_array).astype(cell_array[0].dtype)
Good luck!
I have an array like:
a = array([[1,2,3],[3,4,5],[4,5,6]])
What's the most efficient way to slice out a 1x2 array out of this that has only the first two columns of "a"?
i.e.
array([[2,3],[4,5],[5,6]]) in this case.
Two dimensional numpy arrays are indexed using a[i,j] (not a[i][j]), but you can use the same slicing notation with numpy arrays and matrices as you can with ordinary matrices in python (just put them in a single []):
>>> from numpy import array
>>> a = array([[1,2,3],[3,4,5],[4,5,6]])
>>> a[:,1:]
array([[2, 3],
[4, 5],
[5, 6]])
Is this what you're looking for?
a[:,1:]
To quote documentation, the basic slice syntax is i:j:k where i is the starting index, j is the stopping index, and k is the step (when k > 0).
Now if i is not given, it defaults to 0 if k > 0. Otherwise i defaults to n - 1 for k < 0 (where n is the length of the array).
If j is not given, it defaults to n (length of array).
That's for a one dimensional array.
Now a two dimensional array is a different beast. The slicing syntax for that is a[rowrange, columnrange].
So if you want all the rows, but just the last two columns, like in your case, you do:
a[0:3, 1:3]
Here, "[0:3]" means all the rows from 0 to 3. and "[1:3]" means all columns from column 1 to column 3.
Now as you may be wondering, even though you have only 3 columns and the numbering starts from 1, it must return 3 columns right? i.e: column 1, column 2, column 3
That is the tricky part of this syntax. The first column is actually column 0. So when you say "[1:3]", you are actually saying give me column 1 and column 2. Which are the last two columns you want. (There actually is no column 3.)
Now if you don't know how long your matrix is or if you want all the rows, you can just leave that part empty.
i.e.
a[:, 1:3]
Same goes for columns also. i.e if you wanted say, all the columns but just the first row, you would write
a[0:1,:]
Now, how the above answer a[:,1:] works is because when you say "[1:]" for columns, it means give me everything except for column 0, and till the end of all the columns. i.e empty means 'till the end'.
By now you must realize that anything on either side of the comma is all a subset of the one dimensional case I first mentioned above. i.e if you want to specify your rows using step sizes you can write
a[::2,1]
Which in your case would return
array([[2, 3],
[5, 6]])
i.e. a[::2,1] elucidates as: give me every other row, starting with the top most, and give me only the 2nd column.
This took me some time to figure out. So pasting it here, just in case it helps someone.