Develop Reference

Matrix multiplication-style addition in numpy

I have a row vector a and a column vector b in numpy. If I was to do matrix multiplication on the two vectors, I would obtain a matrix m where m[i,j] = a[i]b[j]. I was wondering if there was a simple way of performing this style of operation for addition - i.e, obtaining a matrix n where n[i,j] = a[i] + b[j]. Is there a built-in method for performing something like this?

I guess you mean np.add?
import numpy as np
x1 = np.arange(3).reshape((3, 1))
x2 = np.arange(3).reshape((1, 3))
result = np.add(x1, x2)
print(x1, '\n')
print(x2, '\n')
print(result)
Output:
[[0]
[1]
[2]]
[[0 1 2]]
[[0 1 2]
[1 2 3]
[2 3 4]]

A compact way of expanding a (n,) array to (n,1) is with the np.newaxis or None indexing:
In [30]: a = np.arange(1,5); b = np.arange(1,4)*10
In [31]: a,b
Out[31]: (array([1, 2, 3, 4]), array([10, 20, 30]))
In [32]: a[:,None]+b
Out[32]:
array([[11, 21, 31],
[12, 22, 32],
[13, 23, 33],
[14, 24, 34]])
where:
In [33]: a[:,None]
Out[33]:
array([[1],
[2],
[3],
[4]])
The broadcasting process is:
(m,1), (n,) => (m,1),(1,n) => (m,n)
The ufunc version of + is np.add, and as such it has an outer method:
In [35]: np.add.outer(a,b)
Out[35]:
array([[11, 21, 31],
[12, 22, 32],
[13, 23, 33],
[14, 24, 34]])
np.outer(a,b) and np.multiply.outer(a,b) and a[:,None]*b are equivalent outer product expressions.

Is there a way to create column vectors in numpy without having to create a list of lists? [duplicate]

This question already has an answer here:
numpy matrix vector multiplication [duplicate]
(1 answer)
Closed 4 years ago.
Say I want to multiply a matrix by a vector:
[1 2 3] [10]
[4 5 6] * [11]
[7 8 9] [12]
In Python using Numpy I would do it like this:
from numpy import *
A = matrix(
[[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
B = matrix(
[[10], [11], [12]])
print(A * B)
However as you can see, in order to define matrix B correctly, I have to type [[10], [11], [12]], and that is a bit tedious. Is there something that just constructs a vector, so I could type something like vector([10, 11, 12]) instead of matrix([[10], [11], [12]])?

You can create B as a numpy array and then use the method dot
B=np.array([10,11,12])
print(A.dot(B)) #Use A.dot(B).T if you need to keep the original dimension

You can just transpose the matrix:
In [1]: import numpy as np
In [2]: A = np.matrix(
...: [[1, 2, 3],
...: [4, 5, 6],
...: [7, 8, 9]])
...:
In [3]: B = np.matrix([10, 11, 12]).T
In [4]: print(A * B)
[[ 68]
[167]
[266]]

You can also use the c_ column assembly object:
>>> np.c_[[10,11,12]]
array([[10],
[11],
[12]])
or
>>> np.c_[10:13]
array([[10],
[11],
[12]])
or with linspace semantics
>>> np.c_[10:12:3j]
array([[10.],
[11.],
[12.]])

An easy option is to create a new axis of dimension None:
import numpy as np
A = np.array([
[1, 2, 3],
[4, 5, 6],
[7, 8, 9],
])
B = np.array([ 10, 11, 12 ])
print(A * B[:,None])

Slightly different than the other answers, but here are my 2 cents :)
Python's zip might be of help here!
In [13]: blist = [10, 11, 12]
In [14]: B = matrix(zip(blist))
In [15]: print B
[[10]
[11]
[12]]

Another option is np.reshape:
B = np.array([10, 11, 12]).reshape(-1, 1)
>>> array([[10],
[11],
[12]])

Convolution of a matrix and a vector i.e. inputs of different dimensions

That is a follow-up question of this one:
I do have Matlab code which I want to convert to Python which includes conv2. I can mimic its behavior in Python doing:
import numpy as np
from scipy import signal
def conv2(x, y, mode='same'):
return np.rot90(signal.convolve2d(np.rot90(x, 2), np.rot90(y, 2), mode=mode), 2)
If I then call
f = [[2, 3, 4], [1, 6, 7]]
g = [[9, 1, 0], [2, 5, 8], [1, 3, 3]]
print conv2(f, g)
print conv2(g, f)
it gives me the same output as Matlab's
conv2([[2,3,4];[1,6,7]], [[9,1,0];[2,5,8];[1,3,3]], 'same')
conv2([[9,1,0];[2,5,8];[1,3,3]], [[2,3,4];[1,6,7]], 'same')
However, Matlab's conv2 also works when one argument is a vector. For example,
conv2([[2,3,4];[1,6,7]], [9,1,0]', 'same')
gives:
11 57 67
1 6 7
I fail to get this output in Python though, as the standard functions usually require the same input dimensions. For example:
signal.convolve(f, [9, 1, 0])
yields
ValueError: in1 and in2 should have the same dimensionality
and
signal.convolve2d(f, [9, 1, 0])
ValueError: object of too small depth for desired array
How can I achieve the same output for inputs of different dimensions?

Just turn the 1d array into a 2d one.
In [71]: f = np.array([[2, 3, 4], [1, 6, 7]])
...: g = np.array([[9, 1, 0], [2, 5, 8], [1, 3, 3]])
In [72]: h = np.array([9,1,0])
In [73]: conv2(f,g)
Out[73]:
array([[ 71, 108, 51],
[ 26, 71, 104]])
In [74]: conv2(f, h[:,None])
Out[74]:
array([[11, 57, 67],
[ 1, 6, 7]])
In [75]: h[:,None]
Out[75]:
array([[9],
[1],
[0]])
In Octave your [9,1,0]' is a column matrix:
>> conv2([[2,3,4];[1,6,7]], [9,1,0]', 'same')
ans =
11 57 67
1 6 7
>> [9,1,0]'
ans =
9
1
0
In MATLAB everything is 2d (or higher). numpy allows 1d arrays.
np.rot90 applied to h is the same as h[None,:] or h created as a 2d array from [[9,1,0]]:
In [76]: np.rot90(h[:,None])
Out[76]: array([[9, 1, 0]])
In octave/matlab you could skip the transpose by making the column matrix right at the start:
conv2([[2,3,4];[1,6,7]], [9;1;0], 'same')
The numpy equivalent is:
conv2(f,[[9],[1],[0]])

Seems the easiest solution is just to use e.g. numpy.expand_dims and then pass a two dimensional input. For the example above it would be:
g2 = np.array([9, 1, 0])
g2 = np.expand_dims(g2, 0)
Then
conv2(f, g2.transpose())
gives me
array([[11, 57, 67],
[ 1, 6, 7]])
which is identical to the Matlab output.

Let's make a reference implementation of N-dimensional pixel binning/bucketing for python's numpy

I frequently want to pixel bin/pixel bucket a numpy array, meaning, replace groups of N consecutive pixels with a single pixel which is the sum of the N replaced pixels. For example, start with the values:
x = np.array([1, 3, 7, 3, 2, 9])
with a bucket size of 2, this transforms into:
bucket(x, bucket_size=2)
= [1+3, 7+3, 2+9]
= [4, 10, 11]
As far as I know, there's no numpy function that specifically does this (please correct me if I'm wrong!), so I frequently roll my own. For 1d numpy arrays, this isn't bad:
import numpy as np
def bucket(x, bucket_size):
return x.reshape(x.size // bucket_size, bucket_size).sum(axis=1)
bucket_me = np.array([3, 4, 5, 5, 1, 3, 2, 3])
print(bucket(bucket_me, bucket_size=2)) #[ 7 10 4 5]
...however, I get confused easily for the multidimensional case, and I end up rolling my own buggy, half-assed solution to this "easy" problem over and over again. I'd love it if we could establish a nice N-dimensional reference implementation.
Preferably the function call would allow different bin sizes along different axes (perhaps something like bucket(x, bucket_size=(2, 2, 3)))
Preferably the solution would be reasonably efficient (reshape and sum are fairly quick in numpy)
Bonus points for handling edge effects when the array doesn't divide nicely into an integer number of buckets.
Bonus points for allowing the user to choose the initial bin edge offset.
As suggested by Divakar, here's my desired behavior in a sample 2-D case:
x = np.array([[1, 2, 3, 4],
[2, 3, 7, 9],
[8, 9, 1, 0],
[0, 0, 3, 4]])
bucket(x, bucket_size=(2, 2))
= [[1 + 2 + 2 + 3, 3 + 4 + 7 + 9],
[8 + 9 + 0 + 0, 1 + 0 + 3 + 4]]
= [[8, 23],
[17, 8]]
...hopefully I did my arithmetic correctly ;)

I think you can do most of the fiddly work with skimage's view_as_blocks. This function is implemented using as_strided so it is very efficient (it just changes the stride information to reshape the array). Because it's written in Python/NumPy, you can always copy the code if you don't have skimage installed.
After applying that function, you just need to sum the N trailing axes of the reshaped array (where N is the length of the bucket_size tuple). Here's a new bucket() function:
from skimage.util import view_as_blocks
def bucket(x, bucket_size):
blocks = view_as_blocks(x, bucket_size)
tup = tuple(range(-len(bucket_size), 0))
return blocks.sum(axis=tup)
Then for example:
>>> x = np.array([1, 3, 7, 3, 2, 9])
>>> bucket(x, bucket_size=(2,))
array([ 4, 10, 11])
>>> x = np.array([[1, 2, 3, 4],
[2, 3, 7, 9],
[8, 9, 1, 0],
[0, 0, 3, 4]])
>>> bucket(x, bucket_size=(2, 2))
array([[ 8, 23],
[17, 8]])
>>> y = np.arange(6*6*6).reshape(6,6,6)
>>> bucket(y, bucket_size=(2, 2, 3))
array([[[ 264, 300],
[ 408, 444],
[ 552, 588]],
[[1128, 1164],
[1272, 1308],
[1416, 1452]],
[[1992, 2028],
[2136, 2172],
[2280, 2316]]])

Natively from as_strided :
x = array([[1, 2, 3, 4],
[2, 3, 7, 9],
[8, 9, 1, 0],
[0, 0, 3, 4]])
from numpy.lib.stride_tricks import as_strided
def bucket(x,bucket_size):
x=np.ascontiguousarray(x)
oldshape=array(x.shape)
newshape=concatenate((oldshape//bucket_size,bucket_size))
oldstrides=array(x.strides)
newstrides=concatenate((oldstrides*bucket_size,oldstrides))
axis=tuple(range(x.ndim,2*x.ndim))
return as_strided (x,newshape,newstrides).sum(axis)
if a dimension not divide evenly into the corresponding dimension of x, remaining elements are lost.
verification :
In [9]: bucket(x,(2,2))
Out[9]:
array([[ 8, 23],
[17, 8]])

To specify different bin sizes along each axis for ndarray cases, you can use iteratively use np.add.reduceat along each axis of it, like so -
def bucket(x, bin_size):
ndims = x.ndim
out = x.copy()
for i in range(ndims):
idx = np.append(0,np.cumsum(bin_size[i][:-1]))
out = np.add.reduceat(out,idx,axis=i)
return out
Sample run -
In [126]: x
Out[126]:
array([[165, 107, 133, 82, 199],
[ 35, 138, 91, 100, 207],
[ 75, 99, 40, 240, 208],
[166, 171, 78, 7, 141]])
In [127]: bucket(x, bin_size = [[2, 2],[3, 2]])
Out[127]:
array([[669, 588],
[629, 596]])
# [2, 2] are the bin sizes along axis=0
# [3, 2] are the bin sizes along axis=1
# array([[165, 107, 133, | 82, 199],
# [ 35, 138, 91, | 100, 207],
# -------------------------------------
# [ 75, 99, 40, | 240, 208],
# [166, 171, 78, | 7, 141]])
In [128]: x[:2,:3].sum()
Out[128]: 669
In [129]: x[:2,3:].sum()
Out[129]: 588
In [130]: x[2:,:3].sum()
Out[130]: 629
In [131]: x[2:,3:].sum()
Out[131]: 596

Can I produce the result of np.outer using np.dot?

I am trying to improve my understanding of numpy functions. I understand the behaviour of numpy.dot. I'd like to understand the behaviour of numpy.outer in terms of numpy.dot.
Based on this Wikipedia article https://en.wikipedia.org/wiki/Outer_product I'd expect for array_equal to return True in the following code. However it does not.
X = np.matrix([
[1,5],
[5,9],
[4,1]
])
r1 = np.outer(X,X)
r2 = np.dot(X, X.T)
np.array_equal(r1, r2)
How can I assign r2 so that np.array_equal returns True? Also, why does numpy's implementation of np.outer not match the definition of outer multiplication on Wikipedia?
Using numpy 1.9.2

In [303]: X=np.array([[1,5],[5,9],[4,1]])
In [304]: X
Out[304]:
array([[1, 5],
[5, 9],
[4, 1]])
In [305]: np.inner(X,X)
Out[305]:
array([[ 26, 50, 9],
[ 50, 106, 29],
[ 9, 29, 17]])
In [306]: np.dot(X,X.T)
Out[306]:
array([[ 26, 50, 9],
[ 50, 106, 29],
[ 9, 29, 17]])
The Wiki outer link mostly talks about vectors, 1d arrays. Your X is 2d.
In [310]: x=np.arange(3)
In [311]: np.outer(x,x)
Out[311]:
array([[0, 0, 0],
[0, 1, 2],
[0, 2, 4]])
In [312]: np.inner(x,x)
Out[312]: 5
In [313]: np.dot(x,x) # same as inner
Out[313]: 5
In [314]: x[:,None]*x[None,:] # same as outer
Out[314]:
array([[0, 0, 0],
[0, 1, 2],
[0, 2, 4]])
Notice that the Wiki outer does not involve summation. Inner does, in this example 5 is the sum of the 3 diagonal values of the outer.
dot also involves summation - all the products followed summation along a specific axis.
Some of the wiki outer equations use explicit indices. The einsum function can implement these calculations.
In [325]: np.einsum('ij,kj->ik',X,X)
Out[325]:
array([[ 26, 50, 9],
[ 50, 106, 29],
[ 9, 29, 17]])
In [326]: np.einsum('ij,jk->ik',X,X.T)
Out[326]:
array([[ 26, 50, 9],
[ 50, 106, 29],
[ 9, 29, 17]])
In [327]: np.einsum('i,j->ij',x,x)
Out[327]:
array([[0, 0, 0],
[0, 1, 2],
[0, 2, 4]])
In [328]: np.einsum('i,i->',x,x)
Out[328]: 5
As mentioned in the comment, np.outer uses ravel, e.g.
return a.ravel()[:, newaxis]*b.ravel()[newaxis,:]
This the same broadcasted multiplication that I demonstrated earlier for x.

numpy.outer only works for 1-d vectors, not matrices. But for the case of 1-d vectors, there is a relation.
If
import numpy as np
A = np.array([1.0,2.0,3.0])
then this
np.matrix(A).T.dot(np.matrix(A))
should be the same as this
np.outer(A,A)

Another (clunky) version similar to a[:,None] * a[None,:]
a.reshape(a.size, 1) * a.reshape(1, a.size)