I'm wondering that must be a way of doing the following without the for loop:
import numpy
from itertools import product as itprod
a = np.arange(120.).reshape(3,2,5,2,2)
fact = np.linspace(1,1.4,15).reshape((3,5))
for i,j in itprod(range(3),range(5)):
a[i,:,j]*= fact[i,j]
Any suggestions??
To take advantage of broadcasting, you have to insert new axes for fact at the right places:
a *= fact[:, np.newaxis, :, np.newaxis, np.newaxis]
Related
I have two numpy arrays, with just the 3-dimensional coordinates of two molecules.
I need to implement the following equation, and I'm having problems in the subtraction of each coordinate of one of the arrays by the second, and then square it.
I have tried the following, but since I'm still learning I feel that I am making some major mistake. The simple code I use is:
a = [math.sqrt(1/3*((i[:,0]-j[:,0])**2) + ((i[:,1] - j[:,1])**2) + ((i[:,2]-j[:,2])**2) for i, j in zip(coordenates_2, coordenates_1))]
It's numpy you can easily do it using the following example:
import numpy as np
x1 = np.random.randn(3,3,3)
x2 = np.random.randn(3,3,3)
res = np.sqrt(np.mean(np.power(x1-x2,2)))
I need to define matrix multiplication from scratch, as instead of multiplying each constant together, each constant is actually another array and any two arrays need to be "convolved" together (I don't think it's necessary to define what a convolution is here).
I have made a picture that hopefully explains what I'm trying to say better:
The code I have to do this with is this:
for row in range(arr1.shape[2]):
for column in range(arr2.shape[3]):
for index in range(arr2.shape[2]): # Could also be "arr1.shape[3]"
out[:, :, row, column] += convolve(
arr2[:, :, : , column][:, :, index],
arr1[:, :, row, : ][:, :, index]
)
However, this method had proved to be very slow for me, so I was wondering if there was a faster way to do this.
If the intermediate fits in memory the following should be reasonably efficient
import numpy as np
from scipy.signal import fftconvolve,convolve
# example
rng = np.random.default_rng()
A = rng.random((5,6,2,3))
B = rng.random((4,3,3,4))
# custom matmul
Ae,Be = A[...,None],B[:,:,None]
shsh = np.maximum(Ae.shape[2:],Be.shape[2:])
Ae = np.broadcast_to(Ae,(*Ae.shape[:2],*shsh))
Be = np.broadcast_to(Be,(*Be.shape[:2],*shsh))
C = fftconvolve(Ae,Be,axes=(0,1),mode='valid').sum(3)
# original loop for reference
out = np.zeros_like(C)
for row in range(A.shape[2]):
for column in range(B.shape[3]):
for index in range(B.shape[2]): # Could also be "A.shape[3]"
out[:, :, row, column] += convolve(
B[:, :, : , column][:, :, index],
A[:, :, row, : ][:, :, index],
mode='valid'
)
print(np.allclose(C,out))
# True
By doing the convolution in bulk we reduce the total number of fft's we have to do.
If need be this could be further optimized for both speed and memory by doing the sum reduction in Fourier space using einsum. This would require doing the fft convolution by hand, though.
let us say I have a numpy matrix A that is of size Nx2. What I am doing, is computing the 4-quadrant inverse tangent of the first column, and the second column, as so:
import math
for i in xrange(A.shape[0]):
phase[i] = math.atan2(A[i,0], A[i,1])
I would however like to do this in a vectorized manner. How can I do that? The math.atan2() function does not seem to support vectorization.
Thanks!
It looks to me like it should just be:
import numpy as np
phase = np.arctan2(A[:, 0], A[:, 1])
Or possibly (if phase is a different length than A for some odd reason):
phase[:len(A)] = np.arctan2(A[:, 0], A[:, 1])
In other words, don't use math.atan2, use numpy.arctan2 since numpy functions are generally vectorized versions of their math counterparts.
I Want to write the code below as Pythonic way, applying mean over two axis. What the best way to do this?
import numpy as np
m = np.random.rand(30, 10, 10)
m_mean = np.zeros((30, 1))
for j in range(30):
m_mean[j, 0] = m[j, :, :].mean()
If you have a sufficiently recent NumPy, you can do
m_mean = m.mean(axis=(1, 2))
I believe this was introduced in 1.7, though I'm not sure. The documentation was only updated to reflect this in 1.10, but it worked earlier than that.
If your NumPy is too old, you can take the mean a bit more manually:
m_mean = m.sum(axis=2).sum(axis=1) / np.prod(m.shape[1:3])
These will both produce 1-dimensional results. If you really want that extra length-1 axis, you can do something like m_mean = m_mean[:, np.newaxis] to put the extra axis there.
You can also use the numpy.mean() ufunc and pass the output array as an argument to out= as in:
np.mean(m, axis=(1, 2), out=m_mean)
I am a python beginner and I am trying to average two NumPy 2D arrays with shape of (1024,1024). Doing it like this is quite fast:
newImage = (image1 + image2) / 2
But now the images have a "mask" that invalidate certain elements if set to zero. That means if one of the elements is zero, the resulting element should also be zero. My trivial solution is:
newImage = numpy.zeros( (1024,1024) , dtype=numpy.int16 )
for y in xrange(newImage.shape[0]):
for x in xrange(newImage.shape[1]):
val1 = image1[y][x]
val2 = image2[y][x]
if val1!=0 and val2!=0:
newImage[y][x] = (val1 + val2) / 2
But this is really slow. I did not time it, but it seems to be slower by a factor of 100.
I also tried using a lambda operator and "map", but this does not return a NumPy array.
Try this:
newImage = numpy.where(np.logical_and(image1, image2), (image1 + image2) / 2, 0)
Where none of image1 and image2 equals zero, take their mean, otherwise zero.
Looping with native Python code is generally much slower than using
built-in tools that use fast C loops. I'm not familiar with NumPy; can
you use map() to do a transformation from your two input arrays to
the output? If so, that should be faster.
Explicit for loops are very inefficient in Python in general, not only for numpy operations. Fortunately, there are faster ways to solve our problem. If memory is not an issue, this solution is quite good:
import numpy as np
new_image = np.zeros((1024, 1024), dtype=np.int16)
valid = (image1!=0) & (image2!=0)
new_image[valid] = (image1+image2)[valid]
Another solution using masked arrays, which do not create copies of the arrays (they represent views of the original image1/2:
m1 = np.ma.masked_equal(image1, 0)
m2 = np.ma.masked_equal(image2, 0)
new_image = (m1+m2).filled(0)
Update: The first solution seems to be 3 times faster than the second for arrays with about 1000 non-zero entries.
numpy array access operation seems slow at best. I can't see any reason for it. You can clearly see it by constructing a simple example:
import numpy
# numpy version
def at(s,n):
t1=time.time()
a=numpy.zeros(s,dtype=numpy.int32)
for i in range(n):
a[i%s]=n
t2=time.time()
return t2-t1
# native version
def an(s,n):
t1=time.time()
a=[(i) for i in range(s)]
for i in range(n):
a[i%s]=n
t2=time.time()
return t2-t1
# test
[at(100000,1000000),an(100000,1000000)]
Result: [0.21972250938415527, 0.15950298309326172]