j=np.arange(20,dtype=np.int)
site=np.ones((20,200),dtype=np.int)
sumkma=np.ones((100,20))
[sumkma[site[x],x] for x in range(20)]
This works, but I don't want to use for loop. When I try
sumkma[site[j],j]
I get this error:
IndexError: shape mismatch: indexing arrays could not be broadcast together with shapes (20,200) (20,)
How to fix the error?
When accessing a numpy multi-dimensional array with other multi-dimensional arrays of integer type the arrays used for the indices need to have the same shape.
Numpy will happily broadcast, if possible - but for that to be possible the arrays need to have the same dimensionality, e.g. this works:
sumkma[site[j], j[:,np.newaxis]]
The np.newaxis results in j[:,np.newaxis] being two-dimensional (shape is now (20,1), whereas shape of j is one-dimensional (20,)). It now has a shape that can be broadcasted to the shape of site[j]:
>>> j.shape
(20,)
>>> site[j].shape
(20,200)
>>> j[:,np.newaxis].shape
(20,1)
The same dimensionality for the index arrays allows numpy to broadcast them to have the same shape (20,200).
https://docs.scipy.org/doc/numpy-1.13.0/user/basics.indexing.html#indexing-multi-dimensional-arrays
Related
I am trying to generate a numpy array with elements as two other numpy arrays, as below.
W1b1 = np.zeros((256, 161))
W2b2 = np.zeros((256, 257))
Wx = np.array([W1b1, W2b2], dtype=np.object)
this gives an error:
ValueError: could not broadcast input array from shape (256,161) into shape (256).
However, if I take entirely different dimensions for of W1b1 and W2b2 then I do not get an error, as below.
A1 = np.zeros((256, 161))
A2 = np.zeros((257, 257))
A3 = np.array([A1, A2], dtype=np.object)
I do not get what is wrong in the first code and why is numpy array trying to broadcast one of the input arrays.
I have tried on below versions (Python 2.7.6, Numpy 1.13.1) and (Python 3.6.4, Numpy 1.14.1).
Don't count on np.array(..., object) making the right object array. At the moment we don't have control over how many dimensions it makes. Conceivably it could make a (2,) array, or (2, 256) (with 1d contents). Sometimes it works, sometimes raises an error. There's something of a pattern, but I haven't seen an analysis of the code that shows exactly what is happening.
For now it is safer to allocate the array, and fill it:
In [57]: arr = np.empty(2, object)
In [58]: arr[:] = [W1b1, W2b2]
np.array([np.zeros((3,2)),np.ones((3,4))], object) also raises this error. So the error arises when the first dimensions match, but the later ones don't. Now that I think about, I've seen this error before.
Earlier SO questions on the topic
numpy array 1.9.2 getting ValueError: could not broadcast input array from shape (4,2) into shape (4)
Creation of array of arrays fails, when first size of first dimension matches
Creating array of arrays in numpy with different dimensions
I am trying to perform non-consectuitive slicing of a multidimensional array like this (Matlab peudo code)
A = B(:,:,[1,3],[2,4,6]) %A and B are two 4D matrices
But when I try to write this code in Python:
A = B[:,:,np.array([0,2]),np.array([1,3,5])] #A and B are two 4D arrays
it gives an error: IndexError: shape mismatch: indexing arrays could not be broadcast...
It should be noted that slicing for one dimension each time works fine!
In numpy, if you use more than one fancy index (i.e. array) to index different dimension of the same array at the same time, they must broadcast. This is designed such that indexing can be more powerful. For your situation, the simplest way to solve the problem is indexing twice:
B[:, :, [0,2]] [..., [1,3,5]]
where ... stands for as many : as possible.
Indexing twice this way would generate some extra data moving time. If you want to index only once, make sure they broadcast (i.e. put fancy indices on different dimension):
B[:, :, np.array([0,2])[:,None], [1,3,5]]
which will result in a X by Y by 2 by 3 array. On the other hand, you can also do
B[:, :, [0,2], np.array([1,3,5])[:,None]]
which will result in a X by Y by 3 by 2 array. The [1,3,5] axis is transposed before the [0,2] axis.
Yon don't have to use np.array([0,2]) if you don't need to do fancy operation with it. Simply [0,2] is fine.
np.array([0,2])[:,None] is equivalent to [[0],[2]], where the point of [:,None] is to create an extra dimension such that the shape becomes (2,1). Shape (2,) and (3,) cannot broadcast, while shape (2,1) and (3,) can, which becomes (2,3).
I am trying to generate a numpy array with elements as two other numpy arrays, as below.
W1b1 = np.zeros((256, 161))
W2b2 = np.zeros((256, 257))
Wx = np.array([W1b1, W2b2], dtype=np.object)
this gives an error:
ValueError: could not broadcast input array from shape (256,161) into shape (256).
However, if I take entirely different dimensions for of W1b1 and W2b2 then I do not get an error, as below.
A1 = np.zeros((256, 161))
A2 = np.zeros((257, 257))
A3 = np.array([A1, A2], dtype=np.object)
I do not get what is wrong in the first code and why is numpy array trying to broadcast one of the input arrays.
I have tried on below versions (Python 2.7.6, Numpy 1.13.1) and (Python 3.6.4, Numpy 1.14.1).
Don't count on np.array(..., object) making the right object array. At the moment we don't have control over how many dimensions it makes. Conceivably it could make a (2,) array, or (2, 256) (with 1d contents). Sometimes it works, sometimes raises an error. There's something of a pattern, but I haven't seen an analysis of the code that shows exactly what is happening.
For now it is safer to allocate the array, and fill it:
In [57]: arr = np.empty(2, object)
In [58]: arr[:] = [W1b1, W2b2]
np.array([np.zeros((3,2)),np.ones((3,4))], object) also raises this error. So the error arises when the first dimensions match, but the later ones don't. Now that I think about, I've seen this error before.
Earlier SO questions on the topic
numpy array 1.9.2 getting ValueError: could not broadcast input array from shape (4,2) into shape (4)
Creation of array of arrays fails, when first size of first dimension matches
Creating array of arrays in numpy with different dimensions
Scratching my head about this assignment, which does not behave as expected:
a = np.arange(24).reshape(4,3,2)
b = np.array([-1,-2,-3])
c = np.array([1])
a[...,c] = b
=> ValueError: shape mismatch: value array of shape (3,) could not be broadcast to indexing result of shape (1,4,3)
I expected
The indexing result shape to be (4,3,1) instead of (1,4,3), and
The right side shape of (3,) to actually be broadcastable to (1,4,3)?
Am I missing something here?
This should work. Indexing with [:, None] changes the orientation of the array.
a[...,c] = b[:, None]
Your first expectation is True. If you try a[...,c].shape it is 4,3,1. Not sure why the error says 1,3,4.
For the second expectation, you are assigning a vector (1D) of size 3 to a 3-D matrix (4,3,1). To do this, you need to make the vector b 2-D of shape (3,1). This can be done:
a[...,c] = b[None].T
See Numpy broadcasting
I am trying to use argsort on an array of float arrays, but faced some problem.
Here is what I try to do:
import numpy as np
z = np.array([np.array([30.9,29.0,5.87],dtype=float),np.array([20.3,1.3,8.8,4.4],dtype=float)]) # actually z is transferred from a tree using root2array whose corrseponding branches is a vector<vector<float>>
Index_list = np.argsort(z)
Then I　received:
ValueError: operands could not be broadcast together with shapes (4,) (3,)
So what should I do to modify z or change the way of argsort to make it work?