How to compute a xi-xj matrix in numpy without loops (by api calls)?
Here's what to start with:
import numpy as np
x = np.random.rand(4)
xij = np.matrix([xi-xj for xj in x for xi in x]).reshape(4,4)
You can take advantage of broadcasting to subtract x as a column vector from x as a flat array and produce the matrix.
>>> x = np.random.rand(4)
Then:
>>> x - x[:,np.newaxis]
array([[ 0. , 0.89175647, 0.80930233, 0.37955823],
[-0.89175647, 0. , -0.08245415, -0.51219825],
[-0.80930233, 0.08245415, 0. , -0.4297441 ],
[-0.37955823, 0.51219825, 0.4297441 , 0. ]])
If you want a matrix object (and not the default array object) you could write:
np.matrix(x - x[:,np.newaxis])
By reshaping the array, you can use the minus operator to calculate what you want
import numpy as np
x = np.random.rand(4)
x = x.reshape(-1,1)
xij = np.matrix(x.T - x)
Another alternative is to use np.subtract.outer:
In [35]: x = np.random.rand(4)
In [36]: np.matrix([xi-xj for xj in x for xi in x]).reshape(4,4)
Out[36]:
matrix([[ 0. , 0.45365177, 0.07227472, -0.05824887],
[-0.45365177, 0. , -0.38137705, -0.51190064],
[-0.07227472, 0.38137705, 0. , -0.13052359],
[ 0.05824887, 0.51190064, 0.13052359, 0. ]])
In [37]: -np.subtract.outer(x, x)
Out[37]:
array([[-0. , 0.45365177, 0.07227472, -0.05824887],
[-0.45365177, -0. , -0.38137705, -0.51190064],
[-0.07227472, 0.38137705, -0. , -0.13052359],
[ 0.05824887, 0.51190064, 0.13052359, -0. ]])
(Note that the result is a numpy array, not a matrix.)
Related
I have the following code:
import numpy as np
epsilon = np.array([[0. , 0.00172667, 0.00071437, 0.00091779, 0.00154501],
[0.00128983, 0. , 0.00028139, 0.00215905, 0.00094862],
[0.00035811, 0.00018714, 0. , 0.00029365, 0.00036993],
[0.00035631, 0.00112175, 0.00022906, 0. , 0.00291149],
[0.00021527, 0.00017653, 0.00010341, 0.00104458, 0. ]])
Sii = np.array([19998169., 14998140., 9997923., 7798321., 2797958.])
n = len(Sii)
epsilonijSjj = np.zeros((n,n))
for i in range(n):
for j in range(n):
epsilonijSjj[i,j] = epsilon[i][j]*Sii[j]
print (epsilonijSjj)
How can I avoid the double for loop and write the code in a fast Pythonic way?
Thank you in advance
Numpy allow you to multiply 2 arrays directly.
So rather than define a 0 based array and populating it with the altered elements of the other array, you can simply create a copy of the other array and apply the multiplication directly like so:
import numpy as np
epsilon = np.array([[0. , 0.00172667, 0.00071437, 0.00091779, 0.00154501],
[0.00128983, 0. , 0.00028139, 0.00215905, 0.00094862],
[0.00035811, 0.00018714, 0. , 0.00029365, 0.00036993],
[0.00035631, 0.00112175, 0.00022906, 0. , 0.00291149],
[0.00021527, 0.00017653, 0.00010341, 0.00104458, 0. ]])
Sii = np.array([19998169., 14998140., 9997923., 7798321., 2797958.])
epsilonijSjj = epsilon.copy()
epsilonijSjj *= Sii
print(epsilonijSjj)
Output:
[[ 0. 25896.8383938 7142.21625351 7157.22103059
4322.87308958]
[25794.23832127 0. 2813.31555297 16836.96495505
2654.19891796]
[ 7161.54430059 2806.7519196 0. 2289.97696165
1035.04860294]
[ 7125.54759639 16824.163545 2290.12424238 0.
8146.22673742]
[ 4305.00584063 2647.6216542 1033.88521743 8145.97015018
0. ]]
Or, just do this, which is faster because it doesn't require creating a copy of an array:
import numpy as np
epsilon = np.array([[0. , 0.00172667, 0.00071437, 0.00091779, 0.00154501],
[0.00128983, 0. , 0.00028139, 0.00215905, 0.00094862],
[0.00035811, 0.00018714, 0. , 0.00029365, 0.00036993],
[0.00035631, 0.00112175, 0.00022906, 0. , 0.00291149],
[0.00021527, 0.00017653, 0.00010341, 0.00104458, 0. ]])
Sii = np.array([19998169., 14998140., 9997923., 7798321., 2797958.])
epsilonijSjj = epsilon * Sii
I have ndarray of eigenvalues and their multiplicities (for instance, np.array([(2.2, 2), (3, 3), (5, 1)])). I need to compute Jordan matrix for this eigenvalues without using Python cycles and iterables (list comprehensions, for loops etc.), only by using NumPy's functions.
I decided to build the matrix by this steps:
Create this blocks using np.vectorize and np.eye with np.fill_diagonal:
Combine blocks into one matrix using hstack and vstack.
But I've got two problems:
Here's snippet of my block creating code:
def eye(t):
eye = np.eye(t[1].astype(int),k=1)
return eye
def jordan_matrix(X: np.ndarray) -> np.ndarray:
dim = np.sum(X[:,1].astype(int))
eyes = np.vectorize(eye, signature='(x)->(n,m)')(X)
return eyes
And I'm getting error ValueError: could not broadcast input array from shape (3,3) into shape (2,2)
I need to create extra zero matrices to fill space which is not used by created blocks, but their sizes are variable and I can't figure out how to create them without using Python's for and its equivalents.
Am I on the right way? How can I get out of this problems?
np.vectorize would basically loop under the hoods. We could use NumPy funcs for actual vectorization at Python level. Here's one such way -
def blockwise_jordan(a):
r = a[:,1].astype(int)
v = np.repeat(a[:,0],r)
out = np.diag(v)
n = out.shape[1]
fillvals = np.ones(n, dtype=out.dtype)
fillvals[r[:-1].cumsum()-1] = 0
out.flat[1::out.shape[1]+1] = fillvals
return out
Sample run -
In [52]: X = np.array([(2.2, 2), (3, 3), (5, 1)])
In [53]: blockwise_jordan(X)
Out[53]:
array([[2.2, 1. , 0. , 0. , 0. , 0. ],
[0. , 2.2, 0. , 0. , 0. , 0. ],
[0. , 0. , 3. , 1. , 0. , 0. ],
[0. , 0. , 0. , 3. , 1. , 0. ],
[0. , 0. , 0. , 0. , 3. , 0. ],
[0. , 0. , 0. , 0. , 0. , 5. ]])
Optimization #1
We can replace the final three steps to perform the conditional assignment of 1s and 0s, like so -
out.flat[1::n+1] = 1
c = r[:-1].cumsum()-1
out[c,c+1] = 0
Here's my solution:
def jordan(a):
e = a[:,0] # eigenvalues
m = a[:,1].astype('int') # multiplicities
d = np.repeat(e, m) # main diagonal
ones = np.ones(d.size - 1)
ones[np.cumsum(m)[:-1] -1] = 0
j = np.diag(d) + np.diag(ones, k=1)
return j
Edit: just realized that my solution is almost the same as Divakar's.
I have an N-by-2 numpy array of 2d coordinates named coords, and another 2d numpy array named plane. What I want to do is like
for x,y in coords:
plane[x,y] = 0
but without for loop to improve efficiency. How to do this with vectorized code? Which function or method in numpy to use?
You can try plane[coords.T[0], coords.T[1]] = 0 Not sure this is what you want. For example:
Let,
plane = np.random.random((5,5))
coords = np.array([ [2,3], [1,2], [1,3] ])
Then,
plane[coords.T[0], coords.T[1]] = 0
will give:
array([[ 0.41981685, 0.4584495 , 0.47734686, 0.23959934, 0.82641475],
[ 0.64888387, 0.44788871, 0. , 0. , 0.298522 ],
[ 0.22764842, 0.06700281, 0.04856316, 0. , 0.70494825],
[ 0.18404081, 0.27090759, 0.23387404, 0.02314846, 0.3712009 ],
[ 0.28215705, 0.12886813, 0.62971 , 0.9059715 , 0.74247202]])
In general we could have matrices of arbitrary sizes. For my application it is necessary to have square matrix. Also the dummy entries should have a specified value. I am wondering if there is anything built in numpy?
Or the easiest way of doing it
EDIT :
The matrix X is already there and it is not squared. We want to pad the value to make it square. Pad it with the dummy given value. All the original values will stay the same.
Thanks a lot
Building upon the answer by LucasB here is a function which will pad an arbitrary matrix M with a given value val so that it becomes square:
def squarify(M,val):
(a,b)=M.shape
if a>b:
padding=((0,0),(0,a-b))
else:
padding=((0,b-a),(0,0))
return numpy.pad(M,padding,mode='constant',constant_values=val)
Since Numpy 1.7, there's the numpy.pad function. Here's an example:
>>> x = np.random.rand(2,3)
>>> np.pad(x, ((0,1), (0,0)), mode='constant', constant_values=42)
array([[ 0.20687158, 0.21241617, 0.91913572],
[ 0.35815412, 0.08503839, 0.51852029],
[ 42. , 42. , 42. ]])
For a 2D numpy array m it’s straightforward to do this by creating a max(m.shape) x max(m.shape) array of ones p and multiplying this by the desired padding value, before setting the slice of p corresponding to m (i.e. p[0:m.shape[0], 0:m.shape[1]]) to be equal to m.
This leads to the following function, where the first line deals with the possibility that the input has only one dimension (i.e. is an array rather than a matrix):
import numpy as np
def pad_to_square(a, pad_value=0):
m = a.reshape((a.shape[0], -1))
padded = pad_value * np.ones(2 * [max(m.shape)], dtype=m.dtype)
padded[0:m.shape[0], 0:m.shape[1]] = m
return padded
So, for example:
>>> r1 = np.random.rand(3, 5)
>>> r1
array([[ 0.85950957, 0.92468279, 0.93643261, 0.82723889, 0.54501699],
[ 0.05921614, 0.94946809, 0.26500925, 0.02287463, 0.04511802],
[ 0.99647148, 0.6926722 , 0.70148198, 0.39861487, 0.86772468]])
>>> pad_to_square(r1, 3)
array([[ 0.85950957, 0.92468279, 0.93643261, 0.82723889, 0.54501699],
[ 0.05921614, 0.94946809, 0.26500925, 0.02287463, 0.04511802],
[ 0.99647148, 0.6926722 , 0.70148198, 0.39861487, 0.86772468],
[ 3. , 3. , 3. , 3. , 3. ],
[ 3. , 3. , 3. , 3. , 3. ]])
or
>>> r2=np.random.rand(4)
>>> r2
array([ 0.10307689, 0.83912888, 0.13105124, 0.09897586])
>>> pad_to_square(r2, 0)
array([[ 0.10307689, 0. , 0. , 0. ],
[ 0.83912888, 0. , 0. , 0. ],
[ 0.13105124, 0. , 0. , 0. ],
[ 0.09897586, 0. , 0. , 0. ]])
etc.
I am a beginner at python and numpy and I need to compute the matrix logarithm for each "pixel" (i.e. x,y position) of a matrix-valued image of dimension NxMx3x3. 3x3 is the dimensions of the matrix at each pixel.
The function I have written so far is the following:
def logm_img(im):
from scipy import linalg
dimx = im.shape[0]
dimy = im.shape[1]
res = zeros_like(im)
for x in range(dimx):
for y in range(dimy):
res[x, y, :, :] = linalg.logm(asmatrix(im[x,y,:,:]))
return res
Is it ok?
Is there a way to avoid the two nested loops ?
Numpy can do that. Just call numpy.log:
>>> import numpy
>>> a = numpy.array(range(100)).reshape(10, 10)
>>> b = numpy.log(a)
__main__:1: RuntimeWarning: divide by zero encountered in log
>>> b
array([[ -inf, 0. , 0.69314718, 1.09861229, 1.38629436,
1.60943791, 1.79175947, 1.94591015, 2.07944154, 2.19722458],
[ 2.30258509, 2.39789527, 2.48490665, 2.56494936, 2.63905733,
2.7080502 , 2.77258872, 2.83321334, 2.89037176, 2.94443898],
[ 2.99573227, 3.04452244, 3.09104245, 3.13549422, 3.17805383,
3.21887582, 3.25809654, 3.29583687, 3.33220451, 3.36729583],
[ 3.40119738, 3.4339872 , 3.4657359 , 3.49650756, 3.52636052,
3.55534806, 3.58351894, 3.61091791, 3.63758616, 3.66356165],
[ 3.68887945, 3.71357207, 3.73766962, 3.76120012, 3.78418963,
3.80666249, 3.8286414 , 3.8501476 , 3.87120101, 3.8918203 ],
[ 3.91202301, 3.93182563, 3.95124372, 3.97029191, 3.98898405,
4.00733319, 4.02535169, 4.04305127, 4.06044301, 4.07753744],
[ 4.09434456, 4.11087386, 4.12713439, 4.14313473, 4.15888308,
4.17438727, 4.18965474, 4.20469262, 4.21950771, 4.2341065 ],
[ 4.24849524, 4.26267988, 4.27666612, 4.29045944, 4.30406509,
4.31748811, 4.33073334, 4.34380542, 4.35670883, 4.36944785],
[ 4.38202663, 4.39444915, 4.40671925, 4.41884061, 4.4308168 ,
4.44265126, 4.4543473 , 4.46590812, 4.47733681, 4.48863637],
[ 4.49980967, 4.51085951, 4.52178858, 4.53259949, 4.54329478,
4.55387689, 4.56434819, 4.57471098, 4.58496748, 4.59511985]])