I have a row vector a and a column vector b in numpy. If I was to do matrix multiplication on the two vectors, I would obtain a matrix m where m[i,j] = a[i]b[j]. I was wondering if there was a simple way of performing this style of operation for addition - i.e, obtaining a matrix n where n[i,j] = a[i] + b[j]. Is there a built-in method for performing something like this?
I guess you mean np.add?
import numpy as np
x1 = np.arange(3).reshape((3, 1))
x2 = np.arange(3).reshape((1, 3))
result = np.add(x1, x2)
print(x1, '\n')
print(x2, '\n')
print(result)
Output:
[[0]
[1]
[2]]
[[0 1 2]]
[[0 1 2]
[1 2 3]
[2 3 4]]
A compact way of expanding a (n,) array to (n,1) is with the np.newaxis or None indexing:
In [30]: a = np.arange(1,5); b = np.arange(1,4)*10
In [31]: a,b
Out[31]: (array([1, 2, 3, 4]), array([10, 20, 30]))
In [32]: a[:,None]+b
Out[32]:
array([[11, 21, 31],
[12, 22, 32],
[13, 23, 33],
[14, 24, 34]])
where:
In [33]: a[:,None]
Out[33]:
array([[1],
[2],
[3],
[4]])
The broadcasting process is:
(m,1), (n,) => (m,1),(1,n) => (m,n)
The ufunc version of + is np.add, and as such it has an outer method:
In [35]: np.add.outer(a,b)
Out[35]:
array([[11, 21, 31],
[12, 22, 32],
[13, 23, 33],
[14, 24, 34]])
np.outer(a,b) and np.multiply.outer(a,b) and a[:,None]*b are equivalent outer product expressions.
Related
I have a dataframe like this,
pd.DataFrame({'a': [1,22,34],
'b': [3,49,65]})
and I want to add 1 to all arrays of this dataframe and store it in the 3rd dimension of a numpy array like the following figure. I want to do this in a for loop because my calculations is more than just adding one to arrays in reality. Any suggestion for a minimal implementation of this?
Another possible solution:
np.array([df.apply(lambda x: x+y) for y in np.arange(2)])
Output:
array([[[ 1, 3],
[22, 49],
[34, 65]],
[[ 2, 4],
[23, 50],
[35, 66]]])
df = pd.DataFrame({'a': [1,22,34],'b': [3,49,65]})
array_2d = df.values
array_3d = np.repeat(array_2d[np.newaxis, :, :], 2, axis=0)
# loop
for i in range(2):
array_3d[i] = array_3d[i] + i
array_3d
###
[[[ 1 3]
[22 49]
[34 65]]
[[ 2 4]
[23 50]
[35 66]]]
Here's #Michael Szczesny way,
(broadcasting)
you only have to choose how many layers you want,
for example, 3 layer
df.values + np.arange(3)[:,None,None]
###
array([[[ 1, 3],
[22, 49],
[34, 65]],
[[ 2, 4],
[23, 50],
[35, 66]],
[[ 3, 5],
[24, 51],
[36, 67]]])
suppose a 2D numpy array has to be created from two 1D numpy arrays x and y. The fortran pseudocode is as follows:
do i = 1, n
do j = 1, m
A[i, j] = x[i]*y[j]
enddo
enddo
what is the most numpythonic way to create this 2D array this without using loops?
The numpythonic way would be to use broadcasting.
You can do a broadcasting operation if the dimensions of your arrays are compatible based on two criteria. Starting from the rightmost dimensions:
they are equal
they are 1.
If you want there to be length-x number of rows, you could to give it an appropriate dimension, with 1, for it to be broadcast with the * operation:
>>> import numpy as np
>>> x = np.array([1,2,3,4])
>>> y = np.array([10, 11])
>>> print(x[:, None] , y, sep="\n"*2)
[[1]
[2]
[3]
[4]]
[10 11]
>>> x[:, None] * y
array([[10, 11],
[20, 22],
[30, 33],
[40, 44]])
Or using the other dimensions (corresponding to switching the orders of the loop):
>>> print(x , y[:, None], sep="\n"*2)
[1 2 3 4]
[[10]
[11]]
>>> x * y[:, None]
array([[10, 20, 30, 40],
[11, 22, 33, 44]])
This question already has answers here:
how does multiplication differ for NumPy Matrix vs Array classes?
(8 answers)
Closed 3 years ago.
What is the difference between the calculation in numpy.linalg.matrix_power and directly multiplying the matrix by itself those many times? This is what I observed and was confused about.
>> Matrix A:
[[2 4 5],
[4 4 5],
[8 3 1]]
>> numpy.linalg.matrix_power(A, 3)
[[556 501 530]
[676 579 600]
[708 500 471]]
>> (A * A) * A
[[556 501 530]
[676 579 600]
[708 500 471]]
But
>> A = normalize(A, axis=1, norm='l1')
[[0.18181818 0.36363636 0.45454545]
[0.30769231 0.30769231 0.38461538]
[0.66666667 0.25 0.08333333]]
>> numpy.linalg.matrix_power(A, 3)
[[0.34477471 0.31773179 0.3374935],
[0.36065187 0.31371769 0.32563044],
[0.42154896 0.2984543 0.27999674]]
>> (A * A) * A
[[0.00601052 0.04808415 0.09391435]
[0.02913063 0.02913063 0.05689577]
[0.2962963 0.015625 0.0005787 ]]
Why are the results different? Which one is the correct (expected) computation?
This is the simple code I am checking
import numpy as np
A = np.matrix([[2, 4, 5], [4, 4, 5], [8, 3, 1]])
#A = normalize(A, axis=1, norm='l1') #uncommented for the 2nd part
print(A)
print(np.linalg.matrix_power(A, 3))
print((A * A) * A)
In [1]: from sklearn.preprocessing import normalize
Start with a numpy array:
In [2]: A = np.array([[2,4,5],[4,4,5],[8,3,1]])
make a np.matrix from it:
In [3]: M = np.matrix(A)
In [4]: A
Out[4]:
array([[2, 4, 5],
[4, 4, 5],
[8, 3, 1]])
In [5]: M
Out[5]:
matrix([[2, 4, 5],
[4, 4, 5],
[8, 3, 1]])
* is element-wise for A:
In [6]: (A*A)
Out[6]:
array([[ 4, 16, 25],
[16, 16, 25],
[64, 9, 1]])
But matrix multiplication for M:
In [7]: M*M
Out[7]:
matrix([[60, 39, 35],
[64, 47, 45],
[36, 47, 56]])
In [8]: A#A # also np.dot(A,A)
Out[8]:
array([[60, 39, 35],
[64, 47, 45],
[36, 47, 56]])
Pass M through normalize:
In [9]: N = normalize(M)
In [10]: N
Out[10]:
array([[0.2981424 , 0.59628479, 0.74535599],
[0.52981294, 0.52981294, 0.66226618],
[0.92998111, 0.34874292, 0.11624764]])
The result is numpy array, not Matrix. So N*N will be elementwise.
Confusion like is part of why np.matrix is no longer recommended.
I have a np array of arrays of arrays:
arr1 = np.array([[1,2,3],[4,5,6],[7,8,9]])
arr2 = np.array([[10,20,30],[40,50,60],[70,80,90]])
arr3 = np.array([[15,25,35],[45,55,65],[75,85,95]])
list_arr = np.array([arr1,arr2,arr3])
and indices array:
indices_array = np.array([1,0,2])
I want to get the array at index 1 for the first (array of arrays), the array at
index 0 for the second (array of arrays) and the array at index 2 for the third (array of arrays)
expected output:
#[[ 4 5 6]
#[10 20 30]
#[75 85 95]]
I am looking for a numpy way to do it. As I have large arrays, I prefer not to use comprehension lists.
Basically, you are selecting the second axis elements with indices_array corresponding to each position along the first axis for all the elements along the third axis. As such, you can do -
list_arr[np.arange(list_arr.shape[0]),indices_array,:]
Sample run -
In [16]: list_arr
Out[16]:
array([[[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9]],
[[10, 20, 30],
[40, 50, 60],
[70, 80, 90]],
[[15, 25, 35],
[45, 55, 65],
[75, 85, 95]]])
In [17]: indices_array
Out[17]: array([1, 0, 2])
In [18]: list_arr[np.arange(list_arr.shape[0]),indices_array,:]
Out[18]:
array([[ 4, 5, 6],
[10, 20, 30],
[75, 85, 95]])
Just acces by linking postions to desired indexes (0-1, 1-0, 2-2) as follows:
desired_array = np.array([list_arrr[x][y] for x,y in enumerate([1,0,2])])
I am trying to improve my understanding of numpy functions. I understand the behaviour of numpy.dot. I'd like to understand the behaviour of numpy.outer in terms of numpy.dot.
Based on this Wikipedia article https://en.wikipedia.org/wiki/Outer_product I'd expect for array_equal to return True in the following code. However it does not.
X = np.matrix([
[1,5],
[5,9],
[4,1]
])
r1 = np.outer(X,X)
r2 = np.dot(X, X.T)
np.array_equal(r1, r2)
How can I assign r2 so that np.array_equal returns True? Also, why does numpy's implementation of np.outer not match the definition of outer multiplication on Wikipedia?
Using numpy 1.9.2
In [303]: X=np.array([[1,5],[5,9],[4,1]])
In [304]: X
Out[304]:
array([[1, 5],
[5, 9],
[4, 1]])
In [305]: np.inner(X,X)
Out[305]:
array([[ 26, 50, 9],
[ 50, 106, 29],
[ 9, 29, 17]])
In [306]: np.dot(X,X.T)
Out[306]:
array([[ 26, 50, 9],
[ 50, 106, 29],
[ 9, 29, 17]])
The Wiki outer link mostly talks about vectors, 1d arrays. Your X is 2d.
In [310]: x=np.arange(3)
In [311]: np.outer(x,x)
Out[311]:
array([[0, 0, 0],
[0, 1, 2],
[0, 2, 4]])
In [312]: np.inner(x,x)
Out[312]: 5
In [313]: np.dot(x,x) # same as inner
Out[313]: 5
In [314]: x[:,None]*x[None,:] # same as outer
Out[314]:
array([[0, 0, 0],
[0, 1, 2],
[0, 2, 4]])
Notice that the Wiki outer does not involve summation. Inner does, in this example 5 is the sum of the 3 diagonal values of the outer.
dot also involves summation - all the products followed summation along a specific axis.
Some of the wiki outer equations use explicit indices. The einsum function can implement these calculations.
In [325]: np.einsum('ij,kj->ik',X,X)
Out[325]:
array([[ 26, 50, 9],
[ 50, 106, 29],
[ 9, 29, 17]])
In [326]: np.einsum('ij,jk->ik',X,X.T)
Out[326]:
array([[ 26, 50, 9],
[ 50, 106, 29],
[ 9, 29, 17]])
In [327]: np.einsum('i,j->ij',x,x)
Out[327]:
array([[0, 0, 0],
[0, 1, 2],
[0, 2, 4]])
In [328]: np.einsum('i,i->',x,x)
Out[328]: 5
As mentioned in the comment, np.outer uses ravel, e.g.
return a.ravel()[:, newaxis]*b.ravel()[newaxis,:]
This the same broadcasted multiplication that I demonstrated earlier for x.
numpy.outer only works for 1-d vectors, not matrices. But for the case of 1-d vectors, there is a relation.
If
import numpy as np
A = np.array([1.0,2.0,3.0])
then this
np.matrix(A).T.dot(np.matrix(A))
should be the same as this
np.outer(A,A)
Another (clunky) version similar to a[:,None] * a[None,:]
a.reshape(a.size, 1) * a.reshape(1, a.size)