I'm trying to convert this version of interp2 from Matlab to Python.
In Matlab is used as
Vq = interp2(V,k)
Which perform interpolation over a matrix V where each original interval has been recursively subdivided k times. Adding a total of 2^k-1 elements to each division.
However I haven't found a Python alternative to this function. I tried with scipy.interpolation.interp2 but only works with three matrix.
I found this alternative in a Forum, looks like an email transcription, but I´ll paste the answer here anyways.
import numpy as np
def interp2d_interleave(z,n):
'''performs linear interpolation on a grid
all points are interpolated in one step not recursively
Parameters
----------
z : 2d array (M,N)
n : int
number of points interpolated
Returns
-------
zi : 2d array ((M-1)*n+M, (N-1)*n+N)
original and linear interpolated values
'''
frac = np.atleast_2d(np.arange(0,n+1)/(1.0+n)).T
zi1 = np.kron(z[:,:-1],np.ones(len(frac))) + np.kron(np.diff(z),frac.T)
zi1 = np.hstack((zi1,z[:,-1:]))
zi2 = np.kron(zi1.T[:,:-1],np.ones(len(frac))) + np.kron(np.diff(zi1.T),frac.T)
zi2 = np.hstack((zi2,zi1.T[:,-1:]))
return zi2.T
def interp2d_interleave_recursive(z,n):
'''interpolates by recursively interleaving n times
'''
zi = z.copy()
for ii in range(1,n+1):
zi = interp2d_interleave(zi,1)
return zi
This should be used as follows
xyz = np.zeros((2, 2))
xyz = interp2d_interleave_recursive(xyz, 1)
And the result would be:
Related
I'm facing a problem with vectorizing a function so that it applies efficiently on a numpy array.
My program entries :
A pos_part 2D Array of Nb_particles lines, 3 columns (basicaly x,y,z coordinates, only z is relevant for the part that bothers me) Nb_particles can up to several hundreds of thousands.
An prop_part 1D array with Nb_particles values. This part I got covered, creation is made with some nice numpy functions ; I just put here a basic distribution that ressembles real values.
A z_distances 1D Array, a simple np.arange betwwen z=0 and z=z_max.
Then come the calculation that takes time, because where I can't find a way to do things properply with only numpy operation of arrays. What i want to do is :
For all distances z_i in z_distances, sum all values from prop_part if corresponding particle coordinate z_particle < z_i. This would return a 1D array the same length as z_distances.
My ideas so far :
Version 0, for loop, enumerate and np.where do retrieve the index of values that I need to sum. Obviously quite long.
Version 1, using a mask on a new array (combination of z coordinates and particle properties), and sum on the masked array. Seems better than v0
Version 2, another mask and a np.vectorize, but i understand it's not efficient as vectorize is basicaly a for loop. Still seems better than v0
Version 3, I'm trying to use mask on a function that can I directly apply to z_distances, but it's not working so far.
So, here I am. There is maybe something to do with a sort and a cumulative sum, but I don't know how to do this, so any help would be greatly appreciated. Please find below the code to make things clearer
Thanks in advance.
import numpy as np
import time
import matplotlib.pyplot as plt
# Creation of particles' positions
Nb_part = 150_000
pos_part = 10*np.random.rand(Nb_part,3)
pos_part[:,0] = pos_part[:,1] = 0
#usefull property creation
beta = 1/1.5
prop_part = (1/beta)*np.exp(-pos_part[:,2]/beta)
z_distances = np.arange(0,10,0.1)
#my version 0
t0=time.time()
result = np.empty(len(z_distances))
for index_dist, val_dist in enumerate(z_distances):
positions = np.where(pos_part[:,2]<val_dist)[0]
result[index_dist] = sum(prop_part[i] for i in positions)
print("v0 :",time.time()-t0)
#A graph to help understand
plt.figure()
plt.plot(z_distances,result, c="red")
plt.ylabel("Sum of particles' usefull property for particles with z-pos<d")
plt.xlabel("d")
#version 1 ??
t1=time.time()
combi = np.column_stack((pos_part[:,2],prop_part))
result2 = np.empty(len(z_distances))
for index_dist, val_dist in enumerate(z_distances):
mask = (combi[:,0]<val_dist)
result2[index_dist]=sum(combi[:,1][mask])
print("v1 :",time.time()-t1)
plt.plot(z_distances,result2, c="blue")
#version 2
t2=time.time()
def themask(a):
mask = (combi[:,0]<a)
return sum(combi[:,1][mask])
thefunc = np.vectorize(themask)
result3 = thefunc(z_distances)
print("v2 :",time.time()-t2)
plt.plot(z_distances,result3, c="green")
### This does not work so far
# version 3
# =============================
# t3=time.time()
# def thesum(a):
# mask = combi[combi[:,0]<a]
# return sum(mask[:,1])
# result4 = thesum(z_distances)
# print("v3 :",time.time()-t3)
# =============================
You can get a lot more performance by writing your first version completely in numpy. Replace pythons sum with np.sum. Instead of the for i in positions list comprehension, simply pass the positions mask you are creating anyways.
Indeed, the np.where is not necessary and my best version looks like:
#my version 0
t0=time.time()
result = np.empty(len(z_distances))
for index_dist, val_dist in enumerate(z_distances):
positions = pos_part[:, 2] < val_dist
result[index_dist] = np.sum(prop_part[positions])
print("v0 :",time.time()-t0)
# out: v0 : 0.06322097778320312
You can get a bit faster if z_distances is very long by using numba.
Running calc for the first time usually creates some overhead which we can get rid of by running the function for some small set of `z_distances.
The below code achieves roughly a factor of two speedup over pure numpy on my laptop.
import numba as nb
#nb.njit(parallel=True)
def calc(result, z_distances):
n = z_distances.shape[0]
for ii in nb.prange(n):
pos = pos_part[:, 2] < z_distances[ii]
result[ii] = np.sum(prop_part[pos])
return result
result4 = np.zeros_like(result)
# _t = time.time()
# calc(result4, z_distances[:10])
# print(time.time()-_t)
t3 = time.time()
result4 = calc(result4, z_distances)
print("v3 :", time.time()-t3)
plt.plot(z_distances, result4)
I need to generate a Healpyx map (using Healpy) from random $a_{\ell m}$, for a spin-2 function.
Schematically, this should look like that:
import healpy as hp
nside = 16 # for example
for el in range(1, L+1): #loop over ell mode
for m in range(-el,el): #for each ell mode loop over m
ind = hp.sphtfunc.Alm.getidx(nside, el, m)
if m == 0:
a_lm[ind] = np.random.randn()
else:
a_lm[ind] = np.random.randn() + 1j * np.random.randn()
a_tmp = hp.sphtfunc.alm2map(a_lm, nside, pol=True)
My two questions are:
1) how do I initialise a_lm ? Specifically, what would be its dimension, using
a_lm = np.zeros(???)
2) if I understood correctly, the output a_tmp is a 1 dimensional list. How do I reshape it into a two-dimensional list (the map) for plotting?
1) What properties do you want your alm to have? You could also just assume a certain power spectrum (C_ell) and use hp.synalm() or hp.synfast().
For the initialization, you've already implemented that m goes from -ell to +ell, so you have a one-dimensional array of length sum_0^ell [2ell+1]. Doing the math should give you the length you need.
2) For the plotting, you could just directly generate a random map and then use e.g. hp.mollview(), which takes the 1-dimensional HEALPix map.
Alternatively, you can use hp.alm2map() to convert your alm to a map.
I also suggest you check out the tutorial for the plotting.
Usually we can follow the following steps to get the length of a_lm.
import healpy as hp
inside = 16
# Get the maximum multipole with the current nside
lmax = 3*nside - 1 #This can vary according to the use. In cosmology, the common value is 2*nside
alm_len = hp.Alm.getsize(lmax)
a_lm = np.empty(alm_len)
I think the tutorial linked in #Daniel's answer is a good resource for plotting Healpix maps.
I have a simple 2x2 transformation matrix, s, which encodes some liner transformation of coordinates such that X' = sX.
I have generated a set of uniformley distributed coordinates on a grid using the np.meshgrid() function and at the moment I traverse each coordinate and apply the transformation at a coordinate by coordinate level. Unfortunately, this very slow for large arrays. Are there any fast ways of doing this? Thanks!
import numpy as np
image_dimension = 1024
image_index = np.arange(0,image_dimension,1)
xx, yy = np.meshgrid(image_index,image_index)
# Pre-calculated Transformation Matrix.
s = np.array([[ -2.45963439e+04, -2.54997726e-01], [ 3.55680731e-02, -2.48005486e+04]])
xx_f = xx.flatten()
yy_f = yy.flatten()
for x_t in range(0, image_dimension*image_dimension):
# Get the current (x,y) coordinate.
x_y_in = np.matrix([[xx_f[x_t]],[yy_f[x_t]]])
# Perform the transformation with x.
optout = s * x_y_in
# Store the new coordinate.
xx_f[x_t] = np.array(optout)[0][0]
yy_f[x_t] = np.array(optout)[1][0]
# Reshape Output
xx_t = xx_f.reshape((image_dimension, image_dimension))
yy_t = yy_f.reshape((image_dimension, image_dimension))
You can use the numpy dot function to get the dot product of your matices as:
xx_tn,yy_tn = np.dot(s,[xx.flatten(),yy.flatten()])
xx_t = xx_tn.reshape((image_dimension, image_dimension))
yy_t = yy_tn.reshape((image_dimension, image_dimension))
Which is much faster
Loops are slow in Python. It is better to use vectorization.
In a nutshell, the idea is to let numpy do the loops in C, which is much faster.
You can express your problem as matrix multiplications X' = sX, where you put all the points in X and transform them all with just one call to numpy's dot product:
xy = np.vstack([xx.ravel(), yy.ravel()])
xy_t = np.dot(s, xy)
xx_t, yy_t = xy_t.reshape((2, image_dimension, image_dimension))
I'm working on an image processing program with OpenCV and numpy. For most pixel operations, I'm able to avoid nested for loops by using np.vectorize(), but one of the functions I need to implement requires as a parameter the 'distance from center', or basically the coordinates of the point being processed.
Pseudoexample :
myArr = [[0,1,2]
[3,4,5]]
def myFunc(val,row,col):
return [row,col]
f = np.vectorize(myFunc)
myResult = f(myArr,row,col)
I obviously can't get elemX and elemY from the vectorized array, but is there another numpy function I could use in this situation or do I have to use for loops?, Is there a way to do it using openCV?
The function I need to put each pixel through is :
f(i, j) = 1/(1 + d(i, j)/L) , d(i,j) being the euclidean distance of the point from the center of the image.
You can get an array of distance from the center using the following lines (which is an example, there are a lot of ways to do this):
import numpy as np
myArr = np.array([[0,1,2], [3,4,5]])
nx, ny = myArr.shape
x = np.arange(nx) - (nx-1)/2. # x an y so they are distance from center, assuming array is "nx" long (as opposed to 1. which is the other common choice)
y = np.arange(ny) - (ny-1)/2.
X, Y = np.meshgrid(x, y)
d = np.sqrt(X**2 + Y**2)
# d =
# [[ 1.11803399 1.11803399]
# [ 0.5 0.5 ]
# [ 1.11803399 1.11803399]]
Then you can calculate f(i, j) by:
f = 1/(1 + d/L)
As an aside, your heavy use of np.vectorize() is a bit dubious. Are you sure it's doing what you want, and did you note the statement from the documentation:
The vectorize function is provided primarily for convenience, not for performance. The implementation is essentially a for loop.
It's generally better to just write you code in vectorized form (like my line for f above which will work whether L is an array or a scaler), and not use numpy.vectorize(), and these are different things.
np.vectorize don't accelerate the code, you can vectorize it this way, `
# This compute distance between all points of MyArray and the center
dist_vector= np.sqrt(np.sum(np.power(center-MyArray,2),axis=1))
# F will contain the target value for each point
F = 1./(1 + 1. * dist_vector/L)
I have a pair of 1D arrays (of different lengths) like the following:
data1 = [0,0,0,1,1,1,0,1,0,0,1]
data2 = [0,1,1,0,1,0,0,1]
I would like to get the max cross correlation of the 2 series in python. In matlab, the xcorr() function will return it OK
I have tried the following 2 methods:
numpy.correlate(data1, data2)
signal.fftconvolve(data2, data1[::-1], mode='full')
Both methods give me the same values, but the values I get from python are different from what comes out of matlab. Python gives me integers values > 1, whereas matlab gives actual correlation values between 0 and 1.
I have tried normalizing the 2 arrays first (value-mean/SD), but the cross correlation values I get are in the thousands which doesnt seem correct.
Matlab will also give you a lag value at which the cross correlation is the greatest. I assume it is easy to do this using indices but whats the most appropriate way of doing this if my arrays contain 10's of thousands of values?
I would like to mimic the xcorr() function that matlab has, any thoughts on how I would do that in python?
numpy.correlate(arr1,arr2,"full")
gave me same output as
xcorr(arr1,arr2)
gives in matlab
Implementation of MATLAB xcorr(x,y) and comparision of result with example.
import scipy.signal as signal
def xcorr(x,y):
"""
Perform Cross-Correlation on x and y
x : 1st signal
y : 2nd signal
returns
lags : lags of correlation
corr : coefficients of correlation
"""
corr = signal.correlate(x, y, mode="full")
lags = signal.correlation_lags(len(x), len(y), mode="full")
return lags, corr
n = np.array([i for i in range(0,15)])
x = 0.84**n
y = np.roll(x,5);
lags,c = xcorr(x,y);
plt.figure()
plt.stem(lags,c)
plt.show()
This code will help in finding the delay between two channels in audio file
xin, fs = sf.read('recording1.wav')
frame_len = int(fs*5*1e-3)
dim_x =xin.shape
M = dim_x[0] # No. of rows
N= dim_x[1] # No. of col
sample_lim = frame_len*100
tau = [0]
M_lim = 20000 # for testing as processing takes time
for i in range(1,N):
c = np.correlate(xin[0:M_lim,0],xin[0:M_lim,i],"full")
maxlags = M_lim-1
c = c[M_lim -1 -maxlags: M_lim + maxlags]
Rmax_pos = np.argmax(c)
pos = Rmax_pos-M_lim+1
tau.append(pos)
print(tau)