Develop Reference

Using Rectbivariate over 2 matrices of coordinates

I am using scipy.interp2d to interpolate over a bunch of coordinates of points:
from scipy.interpolate import interp2d
import numpy as np
grid_x = np.linspace(0, 1, 10)
grid_y = np.linspace(0, 1, 10)
grid_z = np.zeros((grid_size_kc,grid_size_kp))*5
function = RectBivariateSpline(x = grid_x, y = grid_y, z = grid_z)
I would like to interpolate over two matrices of coordinates
x = np.array([[1,2],[3,4]])
y = np.array([[5,6],[7,8]])
So that it would return the interpolated values at x=1,y=5, x=2, y=6, x=3, y=7, etc. Right now, I am simply looping over all potential values but this slows down my code quite a bit and I am trying to use vectorized operations to make things quicker.
Ideally, this would return an array of size 10x10 with all the interpolated values.
Thank you for your help!

I found an easy answer to this, when you call your 'function', juste write
function(x, y, grid = False)
Hope this will be useful for somebody.

Python scipy.sparse: how to efficiently set a set of entries to 0?

Let a be a big scipy.sparse matrix and IJ={(i0,j0),(i1,j1),...} a set of positions. How can I efficiently set all the entries in a in positions IJ to 0? Something like a[IJ]=0.
In Mathematica, I would create a new sparse matrix b with background value 1 (instead of 0) and all entries in IJ. Then, I would use a=a*b (entry-wise multiplication). That does not seem to be an option here.
A toy example:
import scipy.sparse as sp
import numpy as np
np.set_printoptions(linewidth=200,edgeitems=5,precision=4)
m=n=10**1;
a=sp.random(m,n,4/m,format='csr'); print(a.toarray())
IJ=np.array([range(0,n,2),range(0,n,2)]); print(IJ) #every second diagonal

You are almost there. To go by your definitions, all you'd need to do is:
a[IJ[0],IJ[1]] = 0
Note that scipy will warn you:
SparseEfficiencyWarning: Changing the sparsity structure of a csr_matrix is expensive. lil_matrix is more efficient.
You can read more about that here.

The scipy sparse matrices can't have a non-zero background value. While it it possible to make a "sparse" matrix with lots of non-zero value, the performance (speed & memory) would be far worse than dense matrix multiplication.
A possible work-around is to rewrite every sparse matrix to have a default value of zero. For example, if matrix Y' contains mostly 1, I can replace Y' by I - Y where Y = I - Y' and I is the identity matrix.
import scipy.sparse as sp
import numpy as np
size = (100, 100)
x = np.random.uniform(-1, 1, size=size)
y = sp.random(*size, 0.001, format='csr')
# Z = (I - Y)X = X - YX
z = x - y.multiply(x)
# A = X(I - Y) = X - XY = X - transpose(YX)
a = x - y.multiply(x).T

Numpy array using expression

how can I create an np array using expression y1 = x, when x array is already defined
x = [1,2,5,7]
from this array x , I would like to create another array y1 using the expression
y1 = x
using numpy

If you want a copy of the array it would be
import numpy as np
y1 = np.array(x)
Currently you just assign the list from x to y1. With this you create a new numpy array with the values from x.

"Invalid dimensions for image data" error when creating a density plot of a function [duplicate]

This question already has answers here:
How to plot 3D function as 2D colormap in python?
(4 answers)
Closed 4 years ago.
I keep receiving:
TypeError: Invalid dimensions for image data
with a blank graph that has axis going from 0.0 to 1.0 on both sides
import numpy as np
import matplotlib.pyplot as plt
from numpy import sin,cos,sqrt,arctan2
from pylab import plot,show,xlabel,ylabel,linspace,ylim
D = 50
x = linspace(0,D,1000)
y = linspace(0,D,1000)
r1 = sqrt((x**2)+(y*2))
def function1():
f1 = sin(r1)
return f1
function1()
plt.imshow(r1,origin='lower',extent=[0,10,0,5])
plt.colorbar()
plt.show()

plt.imshow() is expecting to receive a two-dimensional array (i.e. one with an x and y axis), but r1 is one-dimensional (i.e. it is just a flat list of numbers). You can fix this with the reshape method of numpy arrays:
r1 = r1.reshape(10,100)
This will convert your list of 1000 numbers into 10 rows of 100 numbers. I arbitrarily chose 10 and 100 as a pair of numbers that multiply out to 1000. You should replace them with whatever you want the actual x and y dimensions of the image to be.

Sampling from bivariate normal in python

I'm trying to create two random variables which are correlated with one another, and I believe the best way is to draw from a bivariate normal distribution with given parameters (open to other ideas). The uncorrelated version looks like this:
import numpy as np
sigma = np.random.uniform(.2, .3, 80)
theta = np.random.uniform( 0, .5, 80)
However, for each one of the 80 draws, I want the sigma value to be related to the theta value. Any thoughts?

Use the built-in: http://docs.scipy.org/doc/numpy/reference/generated/numpy.random.multivariate_normal.html
>>> import numpy as np
>>> mymeans = [13,5]
>>> # stdevs = sqrt(5),sqrt(2)
>>> # corr = .3 / (sqrt(5)*sqrt(2) = .134
>>> mycov = [[5,.3], [.3,2]]
>>> np.cov(np.random.multivariate_normal(mymeans,mycov,500000).T)
array([[ 4.99449936, 0.30506976],
[ 0.30506976, 2.00213264]])
>>> np.corrcoef(np.random.multivariate_normal(mymeans,mycov,500000).T)
array([[ 1. , 0.09629313],
[ 0.09629313, 1. ]])
As shown, things get a little hairier if you have to adjust for not-unit variances)
more reference: http://www.riskglossary.com/link/correlation.htm
To be real-world meaningful, the covariance matrix must be symmetric and must also be positive definite or positive semidefinite (it must be invertable). Particular anti-correlation structures might not be possible.

import multivariate_normal from scipy can be used. Suppose we create random variables x and y:
from scipy.stats import multivariate_normal
rv_mean = [0, 1] # mean of x and y
rv_cov = [[1.0,0.5], [0.5,2.0]] # covariance matrix of x and y
rv = multivariate_normal.rvs(rv_mean, rv_cov, size=10000)
You have x from rv[:,0] and y from rv[:,1]. Correlation coefficients can be obtained from
import numpy as np
np.corrcoef(rv.T)