Related
Hi everyone and thank you for your assistance. I am new to python and failed to find an efficient alternative to for loops for the following task.
I want to multiply ndarrays A and B of dimension (d,n,m) and (d,m), respectively. With some abuse of terminology to help understanding, A is a list of nxm matrices and B is a list of vectors in R^m.
For example:
A = np.array([[[0,0,0,0,0],[1,1,1,1,1],[2,2,2,2,2]],[[3,3,3,3,3],[4,4,4,4,4],[5,5,5,5,5]]])
B = np.array([[1,2,3,4,5],[5,6,7,8,9]])
My solution uses a for loop
for i in range(2):
print(A[i]*B[i])
Is there any cheaper alternative (no loops)?
Thank you again
In this case, you can use broadcasting by adding in a new dimension in the "middle" for B:
>>> import numpy as np
>>> A = np.array([[[0,0,0,0,0],[1,1,1,1,1],[2,2,2,2,2]],[[3,3,3,3,3],[4,4,4,4,4],[5,5,5,5,5]]])
>>> B = np.array([[1,2,3,4,5],[5,6,7,8,9]])
>>> A * B[:, None, :]
array([[[ 0, 0, 0, 0, 0],
[ 1, 2, 3, 4, 5],
[ 2, 4, 6, 8, 10]],
[[15, 18, 21, 24, 27],
[20, 24, 28, 32, 36],
[25, 30, 35, 40, 45]]])
Here is a link to the official docs
Note, your original solution already relied on broadcasting:
>>> A[0]
array([[0, 0, 0, 0, 0],
[1, 1, 1, 1, 1],
[2, 2, 2, 2, 2]])
>>> B[0]
array([1, 2, 3, 4, 5])
>>> A[0] * B[0]
array([[ 0, 0, 0, 0, 0],
[ 1, 2, 3, 4, 5],
[ 2, 4, 6, 8, 10]])
Suppose I have a 2D numpy array, say 5*3. Now I would like to map each element i in it to a new array [i, i*i], so the resulting array is 5*3*2.
What is the most efficient (and elegant) way to achieve this purpose?
A naive solution using for:
a = np.arange(15).reshape(5, 3)
r = []
for row in a:
_row = []
for i in row:
_row.append([i, i*i])
r.append(_row)
return np.array(r)
You could use np.dstack to stack both arrays depth wise:
np.dstack([a, a**2])
a = np.arange(15).reshape(5, 3)
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[12, 13, 14]])
np.dstack([a, a**2])
array([[[ 0, 0],
[ 1, 1],
[ 2, 4]],
[[ 3, 9],
[ 4, 16],
[ 5, 25]],
...
I'm having an issue using numpy in python3 at this instruction:
res = ( np.multiply(error, v_sigmop ))
I'm trying to multiply element-wise but I'm having this weird error :
res = ( np.multiply(error, v_sigmop ))
ValueError: operands could not be broadcast together with shapes (10,10000) (10000,10)
This operation isn't illegal since the amount of columns matches the amount of rows of the second array...
Any idea?
I think you are maybe trying to multiple 2 matrixes with shape (r1,c) and (c,r2)
You can use A.dot(B) for your problem, that will multiple 2 matrixes.
This is example:
>>> a = np.arange(12).reshape((3,4))
>>> b = np.arange(8).reshape((4,2))
>>> a
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
>>> b
array([[0, 1],
[2, 3],
[4, 5],
[6, 7]])
>>> a.dot(b)
array([[ 28, 34],
[ 76, 98],
[124, 162]])
Hope it will help you!
Edit
Because you don't want multiple 2 matrixes, you want to multiple as scalar, but multiple scalar is not your operation, that mean you can't multiple 2 matrixes such as:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
multiple with
array([[0, 1],
[2, 3],
[4, 5]])
That is not valid operation for multiple 2 scalar.
You only have operations:
>>> a = np.arange(12).reshape((3,4))
>>> a
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
# Multiple all elements with a scalar
>>> np.multiply(a,0)
array([[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]])
# Multiple each column with a column
>>> b = np.arange(3).reshape((3,1))
>>> b
array([[0],
[1],
[2]])
>>> np.multiply(a,b)
array([[ 0, 0, 0, 0],
[ 4, 5, 6, 7],
[16, 18, 20, 22]])
# Multiple each row with a row
>>> b = np.arange(4).reshape((1,4))
>>> b
array([[0, 1, 2, 3]])
>>> np.multiply(a,b)
array([[ 0, 1, 4, 9],
[ 0, 5, 12, 21],
[ 0, 9, 20, 33]])
# Multiple each element with the same shape
>>> b = np.arange(12).reshape((3,4))
>>> b
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
>>> np.multiply(a,b)
array([[ 0, 1, 4, 9],
[ 16, 25, 36, 49],
[ 64, 81, 100, 121]])
I've been going crazy trying to figure out what stupid thing I'm doing wrong here.
I'm using NumPy, and I have specific row indices and specific column indices that I want to select from. Here's the gist of my problem:
import numpy as np
a = np.arange(20).reshape((5,4))
# array([[ 0, 1, 2, 3],
# [ 4, 5, 6, 7],
# [ 8, 9, 10, 11],
# [12, 13, 14, 15],
# [16, 17, 18, 19]])
# If I select certain rows, it works
print a[[0, 1, 3], :]
# array([[ 0, 1, 2, 3],
# [ 4, 5, 6, 7],
# [12, 13, 14, 15]])
# If I select certain rows and a single column, it works
print a[[0, 1, 3], 2]
# array([ 2, 6, 14])
# But if I select certain rows AND certain columns, it fails
print a[[0,1,3], [0,2]]
# Traceback (most recent call last):
# File "<stdin>", line 1, in <module>
# ValueError: shape mismatch: objects cannot be broadcast to a single shape
Why is this happening? Surely I should be able to select the 1st, 2nd, and 4th rows, and 1st and 3rd columns? The result I'm expecting is:
a[[0,1,3], [0,2]] => [[0, 2],
[4, 6],
[12, 14]]
As Toan suggests, a simple hack would be to just select the rows first, and then select the columns over that.
>>> a[[0,1,3], :] # Returns the rows you want
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[12, 13, 14, 15]])
>>> a[[0,1,3], :][:, [0,2]] # Selects the columns you want as well
array([[ 0, 2],
[ 4, 6],
[12, 14]])
[Edit] The built-in method: np.ix_
I recently discovered that numpy gives you an in-built one-liner to doing exactly what #Jaime suggested, but without having to use broadcasting syntax (which suffers from lack of readability). From the docs:
Using ix_ one can quickly construct index arrays that will index the
cross product. a[np.ix_([1,3],[2,5])] returns the array [[a[1,2] a[1,5]], [a[3,2] a[3,5]]].
So you use it like this:
>>> a = np.arange(20).reshape((5,4))
>>> a[np.ix_([0,1,3], [0,2])]
array([[ 0, 2],
[ 4, 6],
[12, 14]])
And the way it works is that it takes care of aligning arrays the way Jaime suggested, so that broadcasting happens properly:
>>> np.ix_([0,1,3], [0,2])
(array([[0],
[1],
[3]]), array([[0, 2]]))
Also, as MikeC says in a comment, np.ix_ has the advantage of returning a view, which my first (pre-edit) answer did not. This means you can now assign to the indexed array:
>>> a[np.ix_([0,1,3], [0,2])] = -1
>>> a
array([[-1, 1, -1, 3],
[-1, 5, -1, 7],
[ 8, 9, 10, 11],
[-1, 13, -1, 15],
[16, 17, 18, 19]])
Fancy indexing requires you to provide all indices for each dimension. You are providing 3 indices for the first one, and only 2 for the second one, hence the error. You want to do something like this:
>>> a[[[0, 0], [1, 1], [3, 3]], [[0,2], [0,2], [0, 2]]]
array([[ 0, 2],
[ 4, 6],
[12, 14]])
That is of course a pain to write, so you can let broadcasting help you:
>>> a[[[0], [1], [3]], [0, 2]]
array([[ 0, 2],
[ 4, 6],
[12, 14]])
This is much simpler to do if you index with arrays, not lists:
>>> row_idx = np.array([0, 1, 3])
>>> col_idx = np.array([0, 2])
>>> a[row_idx[:, None], col_idx]
array([[ 0, 2],
[ 4, 6],
[12, 14]])
USE:
>>> a[[0,1,3]][:,[0,2]]
array([[ 0, 2],
[ 4, 6],
[12, 14]])
OR:
>>> a[[0,1,3],::2]
array([[ 0, 2],
[ 4, 6],
[12, 14]])
Using np.ix_ is the most convenient way to do it (as answered by others), but it also can be done as follows:
>>> rows = [0, 1, 3]
>>> cols = [0, 2]
>>> (a[rows].T)[cols].T
array([[ 0, 2],
[ 4, 6],
[12, 14]])
Given an array,
>>> n = 2
>>> a = numpy.array([[[1,1,1],[1,2,3],[1,3,4]]]*n)
>>> a
array([[[1, 1, 1],
[1, 2, 3],
[1, 3, 4]],
[[1, 1, 1],
[1, 2, 3],
[1, 3, 4]]])
I know that it's possible to replace values in it succinctly like so,
>>> a[a==2] = 0
>>> a
array([[[1, 1, 1],
[1, 0, 3],
[1, 3, 4]],
[[1, 1, 1],
[1, 0, 3],
[1, 3, 4]]])
Is it possible to do the same for an entire row (last axis) in the array? I know that a[a==[1,2,3]] = 11 will work and replace all the elements of the matching subarrays with 11, but I'd like to substitute a different subarray. My intuition tells me to write the following, but an error results,
>>> a[a==[1,2,3]] = [11,22,33]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: array is not broadcastable to correct shape
In summary, what I'd like to get is:
array([[[1, 1, 1],
[11, 22, 33],
[1, 3, 4]],
[[1, 1, 1],
[11, 22, 33],
[1, 3, 4]]])
... and n of course is, in general, a lot larger than 2, and the other axes are also larger than 3, so I don't want to loop over them if I don't need to.
Update: The [1,2,3] (or whatever else I'm looking for) is not always at index 1. An example:
a = numpy.array([[[1,1,1],[1,2,3],[1,3,4]], [[1,2,3],[1,1,1],[1,3,4]]])
You can achieve this with a much higher performance using np.all to check if all the columns have a True value for your comparison, then using the created mask to replace the values:
mask = np.all(a==[1,2,3], axis=2)
a[mask] = [11, 22, 23]
print(a)
#array([[[ 1, 1, 1],
# [11, 22, 33],
# [ 1, 3, 4]],
#
# [[ 1, 1, 1],
# [11, 22, 33],
# [ 1, 3, 4]]])
You have to do something a little more complicated to acheive what you want.
You can't select slices of arrays as such, but you can select all the specific indexes you want.
So first you need to construct an array that represents the rows you wish to select. ie.
data = numpy.array([[1,2,3],[55,56,57],[1,2,3]])
to_select = numpy.array([1,2,3]*3).reshape(3,3) # three rows of [1,2,3]
selected_indices = data == to_select
# array([[ True, True, True],
# [False, False, False],
# [ True, True, True]], dtype=bool)
data = numpy.where(selected_indices, [4,5,6], data)
# array([[4, 5, 6],
# [55, 56, 57],
# [4, 5, 6]])
# done in one step, but perhaps not very clear as to its intent
data = numpy.where(data == numpy.array([1,2,3]*3).reshape(3,3), [4,5,6], data)
numpy.where works by selecting from the second argument if true and the third argument if false.
You can use where to select from 3 different types of data. The first is an array that has the same shape as selected_indices, the second is just a value on its own (like 2 or 7). The first is most complicated as can be of shape that can be broadcast into the same shape as selected_indices. In this case we provided [1,2,3] which can be stacked together to get an array with shape 3x3.
Note sure if this is what you want, your code example does not create the array you say it does. But:
>>> a = np.array([[[1,1,1],[1,2,3],[1,3,4]], [[1,1,1],[1,2,3],[1,3,4]]])
>>> a
array([[[1, 1, 1],
[1, 2, 3],
[1, 3, 4]],
[[1, 1, 1],
[1, 2, 3],
[1, 3, 4]]])
>>> a[:,1,:] = [[8, 8, 8], [8,8,8]]
>>> a
array([[[1, 1, 1],
[8, 8, 8],
[1, 3, 4]],
[[1, 1, 1],
[8, 8, 8],
[1, 3, 4]]])
>>> a[:,1,:] = [88, 88, 88]
>>> a
array([[[ 1, 1, 1],
[88, 88, 88],
[ 1, 3, 4]],
[[ 1, 1, 1],
[88, 88, 88],
[ 1, 3, 4]]])