I have an array of 3 dimensional vectors. The dimension of the array is arbitrary: it could be a single (N×3), double (M×N×3), triple (K×M×N×3) etc. I need to operate on two components of the vector while preserving the other dimensions.
For example, if I know it is three dimensionsional, I could do the following:
R = numpy.arctan2(A[:,:,:,1], A[:,:,:,0])
which gives me a three dimensional array of scalar values.
Now, to be able to do this on arbitrary number of dimensions. I need to slice over all other dimensions except the the last. So far, I'm able to do it with this:
s = [numpy.s_[:]] * (len(A.shape)-1)
R = numpy.arctan2(A[s+[1]], A[s+[0]])
which works even for single vectors. Is there a more numpythonic way of achieving the above?
I found an even nicer way. This here works for me
R = numpy.arctan2(A[...,1],A[...,0])
Related
For two-dimensaol array, according Guide to numpy
The two-styles of memory layout for arrays are connected through the transpose operation. Thus, if A is a (contiguous) C-style array, then the same block memory can be used to represent AT as a (contiguous) Fortran-style array. This kindof undorctondina non ho ucoful mhon trrina to orn mmonnina of Fortron
In three dimensional matrix
A = np.arange(6).reshape([1,2,3])
we can view A as a 1 by 1 block matrix, which means A has one entry, and that entry is a matrix with two rows and three columns.
So we can iteratively take the transpose according to the rules above.
My question is:
A= np.arange(6).reshape([1,2,3])
B = A.transpose(1,2,0)
In this case, how does it work. Is there a rule that can tell me how the elements of B are arranged
I'm having some trouble understanding the rules for array broadcasting in Numpy.
Obviously, if you perform element-wise multiplication on two arrays of the same dimensions and shape, everything is fine. Also, if you multiply a multi-dimensional array by a scalar it works. This I understand.
But if you have two N-dimensional arrays of different shapes, it's unclear to me exactly what the broadcasting rules are. This documentation/tutorial explains that: In order to broadcast, the size of the trailing axes for both arrays in an operation must either be the same size or one of them must be one.
Okay, so I assume by trailing axis they are referring to the N in a M x N array. So, that means if I attempt to multiply two 2D arrays (matrices) with equal number of columns, it should work? Except it doesn't...
>>> from numpy import *
>>> A = array([[1,2],[3,4]])
>>> B = array([[2,3],[4,6],[6,9],[8,12]])
>>> print(A)
[[1 2]
[3 4]]
>>> print(B)
[[ 2 3]
[ 4 6]
[ 6 9]
[ 8 12]]
>>>
>>> A * B
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: shape mismatch: objects cannot be broadcast to a single shape
Since both A and B have two columns, I would have thought this would work. So, I'm probably misunderstanding something here about the term "trailing axis", and how it applies to N-dimensional arrays.
Can someone explain why my example doesn't work, and what is meant by "trailing axis"?
Well, the meaning of trailing axes is explained on the linked documentation page.
If you have two arrays with different dimensions number, say one 1x2x3 and other 2x3, then you compare only the trailing common dimensions, in this case 2x3. But if both your arrays are two-dimensional, then their corresponding sizes have to be either equal or one of them has to be 1. Dimensions along which the array has size 1 are called singular, and the array can be broadcasted along them.
In your case you have a 2x2 and 4x2 and 4 != 2 and neither 4 or 2 equals 1, so this doesn't work.
From http://cs231n.github.io/python-numpy-tutorial/#numpy-broadcasting:
Broadcasting two arrays together follows these rules:
If the arrays do not have the same rank, prepend the shape of the lower rank array with 1s until both shapes have the same length.
The two arrays are said to be compatible in a dimension if they have the same size in the dimension, or if one of the arrays has size 1 in that dimension.
The arrays can be broadcast together if they are compatible in all dimensions.
After broadcasting, each array behaves as if it had shape equal to the elementwise maximum of shapes of the two input arrays.
In any dimension where one array had size 1 and the other array had size greater than 1, the first array behaves as if it were copied along that dimension
If this explanation does not make sense, try reading the explanation from the documentation or this explanation.
we should consider two points about broadcasting. first: what is possible. second: how much of the possible things is done by numpy.
I know it might look a bit confusing, but I will make it clear by some example.
lets start from the zero level.
suppose we have two matrices. first matrix has three dimensions (named A) and the second has five (named B). numpy tries to match last/trailing dimensions. so numpy does not care about the first two dimensions of B. then numpy compares those trailing dimensions with each other. and if and only if they be equal or one of them be 1, numpy says "O.K. you two match". and if it these conditions don't satisfy, numpy would "sorry...its not my job!".
But I know that you may say comparison was better to be done in way that can handle when they are devisable(4 and 2 / 9 and 3). you might say it could be replicated/broadcasted by a whole number(2/3 in out example). and i am agree with you. and this is the reason I started my discussion with a distinction between what is possible and what is the capability of numpy.
I am working with multiple multidimensional arrays. Let us consider dummy example for simplicity:
array_list=[np.ones(3), np.ones((3,3,3)), np.ones((3,3)), np.ones(3)]
I need to subscribe the outermost dimension of each array in the list. For example, my goal is to set some of the elements to zero according to a specified range in the outermost dimension:
array_list[0][0:2]=0
array_list[1][:,:,0:2]=0
array_list[2][:,0:2]=0
array_list[3][0:2]=0
In my real application I don't know how many arrays I have and how many dimensions are in there.
I would like to do the task in a for loop:
for array in array_list:
array[???]=0
But I am struggling how to implement this if I don't know the dimensionality of each array.
Use the Ellipsis to select all dimensions except the last (if there's only 1, nothing is selected).
for array in array_list:
array[..., 0:2] = 0
I'm using Numpy and have a 7x12x12 matrix whose values I would like to populate in 12x12 chunks, 7 different times. Suppose I have these 12x12 matrices:
first_Matrix
second_Matrix
third_Matrix
... (etc)
seventh_Matrix = first_Matrix + second_Matrix + third_Matrix...
that I'd like to add to:
grand_Matrix
How can I do this? I assume there is a better way than loops that map the coordinates from one matrix to the next, and if there's not, could someone please write out the code for mapping first_Matrix into the first 12x12 element of grand_Matrix?
grand_Matrix[0,...] = first_Matrix
grand_Matrix[1,...] = second_Matrix
and so on.
Anyway, as #Lattyware commented, it is a bad design to have extra names for so many such homogenous objects.
If you have a list of 12x12 matrices:
grand_Matrix = np.vstack(m[None,...] for m in matrices)
None adds a new dimension to each matrix and stacks them along this dimension.
I am reading a vendor-provided large binary array into a 2D numpy array tempfid(M, N)
# load data
data=numpy.fromfile(file=dirname+'/fid', dtype=numpy.dtype('i4'))
# convert to complex data
fid=data[::2]+1j*data[1::2]
tempfid=fid.reshape(I*J*K, N)
and then I need to reshape it into a 4D array useful4d(N,I,J,K) using non-trivial mappings for the indices. I do this with a for loop along the following lines:
for idx in range(M):
i=f1(idx) # f1, f2, and f3 are functions involving / and % as well as some lookups
j=f2(idx)
k=f3(idx)
newfid[:,i,j,k] = tempfid[idx,:] #SLOW! CAN WE IMPROVE THIS?
Converting to complex takes 33% of the time while the copying of these slices M slices takes the remaining 66%. Calculating the indices is fast irrespective of whether I do this one by one in a loop as shown or by numpy.vectorizing the operation and applying it to an arange(M).
Is there a way to speed this up? Any help on more efficient slicing, copying (or not) etc appreciated.
EDIT:
As learned in the answer to question "What's the fastest way to convert an interleaved NumPy integer array to complex64?" the conversion to complex can be sped up by a factor of 6 if a view is used instead:
fid = data.astype(numpy.float32).view(numpy.complex64)
idx = numpy.arange(M)
i = numpy.vectorize(f1)(idx)
j = numpy.vectorize(f2)(idx)
k = numpy.vectorize(f3)(idx)
# you can index arrays with other arrays
# that lets you specify this operation in one line.
newfid[:, i,j,k] = tempfid.T
I've never used numpy's vectorize. Vectorize just means that numpy will call your python function multiple times. In order to get speed, you need use array operations like the one I showed here and you used to get complex numbers.
EDIT
The problem is that the dimension of size 128 was first in newfid, but last in tempfid. This is easily by using .T which takes the transpose.
How about this. Set us your indicies using the vectorized versions of f1,f2,f3 (not necessarily using np.vectorize, but perhaps just writing a function that takes an array and returns an array), then use np.ix_:
http://docs.scipy.org/doc/numpy/reference/generated/numpy.ix_.html
to get the index arrays. Then reshape tempfid to the same shape as newfid and then use the results of np.ix_ to set the values. For example:
tempfid = np.arange(10)
i = f1(idx) # i = [4,3,2,1,0]
j = f2(idx) # j = [1,0]
ii = np.ix_(i,j)
newfid = tempfid.reshape((5,2))[ii]
This maps the elements of tempfid onto a new shape with a different ordering.