Develop Reference

curve_fit multivariable arrays non-linear regression

I am trying to fit the coefficients of a multivariate function with curve_fit. All variables are arrays of the following shape : (1000,) Manually I can fit the curves as follows. First I define my function where the variables = [dphi1,dphi2,phi1,phi2,M] and the coefficients = [c1,c2,c3,c4,c5,c6,c7,c8,c9]:
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
import math
# Function definition
def ddphi1(dphi1,dphi2,phi1,phi2,M,c1,c2,c3,c4,c5,c6,c7,c8,c9):
return (-(c1*np.sin(phi1-phi2)*np.cos(phi1-phi2)*dphi1**2)-(c2*np.sin(phi1-phi2)*dphi2**2)+(c3*np.cos(phi1-phi2)*np.sin(phi2))+(c4*np.cos(phi1-phi2)*(dphi2-dphi1))-(c5*np.sin(phi1))+c6*M-c7*dphi1)/(c8-(c9*np.cos(phi1-phi2)*np.cos(phi1-phi2)))
my first prediction of the coefficients:
p = [0.5625, 0.375, 27.590625000000003, 0.09375, 55.18125, 62.5, 0.425, 1, 0.5625]
I calculate the values of the function iteratively. I take the length of any of the variables already have the same size:
n = len(time1)
y = np.empty(n)
for i in range(n):
y[i] = ddphi1(dphi11[i],dphi22[i],phi11[i],phi22[i],M[i],p[0],p[1],p[2],p[3],p[4],p[5],p[6],p[7],p[8])
plt.plot(time1, ddphi11)
plt.plot(time1, y, 'r')
Predicted Vs real data
Now the idea is to calculate the coefficients automatically with curve_fit as follows: ** ddphi1 ist my Callback function and ddphi11 my data of shape (1000,) as well as the other variables
from scipy.optimize import curve_fit
g = [0.56, 0.37, 27.63, 0.094, 55.18, 62.5, 0.625, 1, 0.56]
c,cov =curve_fit(ddphi1,(dphi1,dphi2,phi1,phi2,M),ddphi11,g)
print(c)
and I receive this error
--------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-158-e8e42e7b1216> in <module>()
1 from scipy.optimize import curve_fit
2
----> 3 c,cov =curve_fit(ddphi1,(dphi1,dphi2,phi1,phi2,M),ddphi11,g=[0.56, 0.37, 27.63, 0.094, 55.18, 62.5, 0.625, 1, 0.56])
4 print(c)
1 frames
/usr/local/lib/python3.7/dist-packages/scipy/optimize/minpack.py in curve_fit(f, xdata, ydata, p0, sigma, absolute_sigma, check_finite, bounds, method, jac, **kwargs)
719 # non-array_like `xdata`.
720 if check_finite:
--> 721 xdata = np.asarray_chkfinite(xdata, float)
722 else:
723 xdata = np.asarray(xdata, float)
/usr/local/lib/python3.7/dist-packages/numpy/lib/function_base.py in asarray_chkfinite(a, dtype, order)
484
485 """
--> 486 a = asarray(a, dtype=dtype, order=order)
487 if a.dtype.char in typecodes['AllFloat'] and not np.isfinite(a).all():
488 raise ValueError(
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (5,) + inhomogeneous part.
I have seen that most of the data that goes into the curve_fir is in the form list. Maybe there is a solution when dealing with arrays? espero I hope you can help me as I am new to Python.

I was finally able to solve it. only the arrays should have been concatenated in a global variable
X=np.column_stack([dphi11,dphi22,phi11,phi22,M])
Then describe the model in based the global variable
def model(X,c1,c2,c3,c4,c5,c6,c7,c8,c9):
dphi1 = X[:,0]
dphi2 = X[:,1]
phi1 = X[:,2]
phi2 = X[:,3]
M = X[:,4]
f = (-(c1*np.sin(phi1-phi2)*np.cos(phi1-phi2)*dphi1**2)-(c2*np.sin(phi1-phi2)*dphi2**2)+(c3*np.cos(phi1-phi2)*np.sin(phi2))+(c4*np.cos(phi1-phi2)*(dphi2-dphi1))-(c5*np.sin(phi1))+c6*M-c7*dphi1)/(c8-(c9*np.cos(phi1-phi2)*np.cos(phi1-phi2)))
return f
and the magic begins
guesses = [0.56, 0.37, 27.63, 0.094, 55.18, 62.5, 0.625, 1, 0.56]
from scipy.optimize import curve_fit
popt, pcov = curve_fit(model, X, ddphi11, guesses)
print(popt)

skcuda.linalg.PCA's fit_transform throws error

I am trying to run PCA (Principal component analysis) on GPU. I am using skcuda.linalg.PCA for that purpose, but it's not working. From their tutorial (https://scikit-cuda.readthedocs.io/en/latest/generated/skcuda.linalg.PCA.html):
import pycuda.autoinit
import pycuda.gpuarray as gpuarray
import numpy as np
import skcuda.linalg as linalg
from skcuda.linalg import PCA as cuPCA
pca = cuPCA(n_components=4) # map the data to 4 dimensions
X = np.random.rand(1000,100) # 1000 samples of 100-dimensional data vectors
X_gpu = gpuarray.GPUArray((1000,100), np.float64, order="F") # note that order="F" or a transpose is necessary. fit_transform requires row-major matrices, and column-major is the default
X_gpu.set(X) # copy data to gpu
T_gpu = pca.fit_transform(X_gpu) # calculate the principal components
When I run it I get the following error:
cublasInternalError Traceback (most recent call last)
<ipython-input-31-02aaf0fa19e4> in <module>
8 X_gpu = gpuarray.GPUArray((1000,100), np.float64, order="F") # note that order="F" or a transpose is necessary. fit_transform requires row-major matrices, and column-major is the default
9 X_gpu.set(X) # copy data to gpu
---> 10 T_gpu = pca.fit_transform(X_gpu)
/opt/conda/lib/python3.7/site-packages/skcuda/linalg.py in fit_transform(self, X_gpu)
204 cuGemv (self.h, 'n', p, k, -1.0, P_gpu.gpudata, p, U_gpu.gpudata, 1, 1.0, P_gpu[:,k].gpudata, 1)
205
--> 206 l2 = cuNrm2(self.h, p, P_gpu[:,k].gpudata, 1)
207 cuScal(self.h, p, 1.0/l2, P_gpu[:,k].gpudata, 1)
208 cuGemv(self.h, 'n', n, p, 1.0, R_gpu.gpudata, n, P_gpu[:,k].gpudata, 1, 0.0, T_gpu[:,k].gpudata, 1)
/opt/conda/lib/python3.7/site-packages/skcuda/cublas.py in cublasDnrm2(handle, n, x, incx)
1295 n, int(x), incx,
1296 ctypes.byref(result))
-> 1297 cublasCheckStatus(status)
1298 return np.float64(result.value)
1299
/opt/conda/lib/python3.7/site-packages/skcuda/cublas.py in cublasCheckStatus(status)
177 raise cublasError
178 else:
--> 179 raise e
180
181 # Helper functions:
cublasInternalError
Initially, I was running on my own data and I got this error. Then I decided to run the example and I got exactly the same error. Does any1 know what's the problem here? I am using Kaggle notebook with Tesla T4 GPU. Thanks.

3D graph error: "The truth value of an array with more than one element is ambiguous"

I am trying to plot a 3D graph, using a re-existing function to generate the Z values. However, this is yielding the error "The truth value of an array with more than one element is ambiguous". This seems strange, as I am able to generate a list of Z values using the same function and y,x values, but once I include the 3D graphing code the error occurs.
My graphing code is:
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d.axes3d import Axes3D
from matplotlib import cm
def f(tau,tau_b): #re-use society welfare function of tau & tau_b, using corr=0.6
Z = society_welfare2 (0.6, tau, tau_b)
return Z
xgrid=np.linspace(1e-5, 1-1e-5,100) #tau grid
ygrid=np.linspace(1e-5, 1-1e-5,100) #tau_b grid
tau,tau_b=np.meshgrid(xgrid,ygrid)
fig=plt.figure(figsize=(8,6))
ax=fig.add_subplot(111,projection='3d')
ax.plot_surface(tau,
tau_b,
f(tau,tau_b),
rstride=2,cstride=2,
cmap=cm.jet,
alpha=0.7,
linewidth=0.25)
ax.set_zlim(-0.5,1.0)
plt.show()
My society_welfare2 function code:
def society_welfare2 (corr, tau, tau_b):
cov = [[1,corr], [corr,1]] #covariance
epsilon_start,b_start = np.random.multivariate_normal(mean, cov, sample_N).T
epsilon = np.exp(epsilon_start) #to ensure epsilon positive
b = np.exp(b_start) #to ensure b positive
indv_welfares = []
def GBC (t_o):
taxes_paid = []
for i in range(sample_N): #loop over all agents to find their C1,C2,L
def consumption_functions(Lguess,epsilon=epsilon,b=b):
C2 = (((1-tau)*epsilon[i]*w*Lguess) +(1-tau_b)*b[i] + ((t_o)/(1+r)))/((1/((beta**(1/gamma))*((1+r)**(1/gamma)))) + (1/(1+r)))
C1 = C2 /((beta**(1/gamma))*(1+r)**(1/gamma))
return -Utility(C1,C2,Lguess)
result = minimize_scalar(consumption_functions,bounds=(0,1),method='bounded', args=(epsilon, b))
opt_L = result.x
opt_C1=(((1-tau)*(epsilon[i])*w)/(opt_L**sigma))**(1/gamma)
opt_C2=(opt_C1)*((beta**(1/gamma))*(1+r)**(1/gamma))
income_tax = tau*(epsilon[i])*w*opt_L
bequest_tax = tau_b*(b[i])
taxes_paid.append(income_tax)
taxes_paid.append(bequest_tax)
welfare_func = opt_C1**(1-gamma)/(1-gamma)-opt_L**(1+sigma)/(1+sigma) + beta*(opt_C2**(1-gamma)/(1-gamma))
indv_welfares.append(welfare_func)
total_tax_revenue = sum(taxes_paid)
return total_tax_revenue - (10000*t_o)
result1 = minimize_scalar(GBC,bounds=(1e-5, 100000),method='bounded')
opt_t_o = result1.x
total_welfare = sum(indv_welfares)
return total_welfare
The full traceback error code:
ValueError Traceback (most recent call last)
<ipython-input-19-3633f4a9db76> in <module>
18 ax.plot_surface(tau,
19 tau_b,
---> 20 f(tau,tau_b),
21 rstride=2,cstride=2,
22 cmap=cm.jet,
<ipython-input-19-3633f4a9db76> in f(tau, tau_b)
7
8 def f(tau,tau_b): #re-use society welfare function of tau & tau_b, using corr=0.6
----> 9 Z = society_welfare2 (0.6, tau, tau_b)
10 return Z
11
<ipython-input-17-321a709b9684> in society_welfare2(corr, tau, tau_b)
61 return total_tax_revenue - (10000*t_o)
62
---> 63 result1 = minimize_scalar(GBC,bounds=(1e-5, 100000),method='bounded')
64
65 opt_t_o = result1.x
/opt/anaconda3/lib/python3.8/site-packages/scipy/optimize/_minimize.py in minimize_scalar(fun, bracket, bounds, args, method, tol, options)
798 if isinstance(disp, bool):
799 options['disp'] = 2 * int(disp)
--> 800 return _minimize_scalar_bounded(fun, bounds, args, **options)
801 elif meth == 'golden':
802 return _minimize_scalar_golden(fun, bracket, args, **options)
/opt/anaconda3/lib/python3.8/site-packages/scipy/optimize/optimize.py in _minimize_scalar_bounded(func, bounds, args, xatol, maxiter, disp, **unknown_options)
1956 rat = e = 0.0
1957 x = xf
-> 1958 fx = func(x, *args)
1959 num = 1
1960 fmin_data = (1, xf, fx)
<ipython-input-17-321a709b9684> in GBC(t_o)
41 return -Utility(C1,C2,Lguess)
42
---> 43 result = minimize_scalar(consumption_functions,bounds=(0,1),method='bounded', args=(epsilon, b))
44
45 opt_L = result.x
/opt/anaconda3/lib/python3.8/site-packages/scipy/optimize/_minimize.py in minimize_scalar(fun, bracket, bounds, args, method, tol, options)
798 if isinstance(disp, bool):
799 options['disp'] = 2 * int(disp)
--> 800 return _minimize_scalar_bounded(fun, bounds, args, **options)
801 elif meth == 'golden':
802 return _minimize_scalar_golden(fun, bracket, args, **options)
/opt/anaconda3/lib/python3.8/site-packages/scipy/optimize/optimize.py in _minimize_scalar_bounded(func, bounds, args, xatol, maxiter, disp, **unknown_options)
2015 print("%5.0f %12.6g %12.6g %s" % (fmin_data + (step,)))
2016
-> 2017 if fu <= fx:
2018 if x >= xf:
2019 a = xf
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

The lowest point the trace back is
if fu <= fx:
That's comparing two variables in an if. That will work if fu and fx are scalars, or single value arrays. But if either is a multivalue array if will raise this error.
Our task at this point is to trace those variables back to your code. I suspect you are providing arrays for some parameter, where they/is should be a scalar.
Looking at the top. It occurs when you ask for the plot, but a parameter is a function call:
f(tau,tau_b)
and on through a function calls to the minimize on the GBC function. I think that GBC is the func in:
fx = func(x, *args)
Which raises the question, what exactly does GBC return? It's being used in a _minimize_scalar, so it should return exactly one value.
What is its return expression?
return total_tax_revenue - (10000*t_o)
Do you think you can take the analysis from there?
Now do you see why we insist on seeing the traceback. The error is in your code, but the sequence getting there is long, and not obvious from simply reading the code.
edit
Oops, I see another level of minimize, one that uses
consumption_functions
It has several parameters, epsilon and b. I suppose we can deduce what those are. But what is
Utility
The fu <= fx appears to be testing the fx return value against a bound fu. Assuming the bound is scalar, then the value fx must be an array. Is it???

hierarchical clustering in scipy - memory error

Here is my code:
import numpy as np
from scipy.cluster.hierarchy import fclusterdata
def mydist(p1,p2):
return 1
Y = np.random.randn(100000,2)
fclust1 = fclusterdata(Y, 1.0, metric=mydist)
It produces the following error:
MemoryError Traceback (most recent call last)
<ipython-input-52-818db8791e96> in <module>()
----> 1 fclust1 = fclusterdata(Y, 1.0, metric=mydist)
C:\Anaconda3\lib\site-packages\scipy\cluster\hierarchy.py in fclusterdata(X, t, criterion, metric, depth, method, R)
1682 'array.')
1683
-> 1684 Y = distance.pdist(X, metric=metric)
1685 Z = linkage(Y, method=method)
1686 if R is None:
C:\Anaconda3\lib\site-packages\scipy\spatial\distance.py in pdist(X, metric, p, w, V, VI)
1218
1219 m, n = s
-> 1220 dm = np.zeros((m * (m - 1)) // 2, dtype=np.double)
1221
1222 wmink_names = ['wminkowski', 'wmi', 'wm', 'wpnorm']
MemoryError:
So I am guessing my vector is too large. I am a bit surprised, since my distance function is trivial. What is max size vector that fclusterdata can accept?

Hierarchical clustering usually requires a pairwise distance matrix.
That means you need O(n^2) memory. And it does not 'see' that your distance is constant (and it doesn't make sense to optimize for this either).
It's not a very scalable algorithm.

Problems interpolating and evaluating numpy array at arbitrary points with Scipy

I am trying to replicate some of the functionality of Matlab's interp2. I know somewhat similar questions have been asked before, but none apply to my specific case.
I have a distance map (available at this Google drive location):
https://drive.google.com/open?id=0B6acq_amk5e3X0Q5UG1ya1VhSlE&authuser=0
Values are normalized in the range 0-1. Size is 200 rows by 300 columns.
I can load it up with this code snippet:
import numpy as np
dstnc1=np.load('dstnc.npy')
Coordinates are defined by the next snippet:
xmin = 0.
xmax = 9000.
ymin = 0.
ymax = 6000.
r1,c1 = dstnc1.shape
x = np.linspace(xmin,xmax,c1)
y = np.linspace(ymin, ymax,r1)
I have three map points defined by vectors xnew1, ynew1 with this snippet:
xnew1=[3700.540199,3845.940199,3983.240199]
ynew1=[1782.8611,1769.862,1694.862]
I check their location with respect to the distance map with this:
import matplotlib.pyplot as plt
fig = plt.figure(figsize=(20, 16))
ax = fig.add_subplot(1, 1, 1)
plt.imshow(dstnc1, cmap=my_cmap_r,vmin=0,vmax=0.3,
extent=[0, 9000, 0, 6000], origin='upper')
plt.scatter(xnew1, ynew1, s=50, linewidths=0.15)
plt.show()
They plot in the correct location. Now I would like to extract the
distance value at those three points. I tried first interp2d.
from scipy.interpolate import interp2d
x1 = np.linspace(xmin,xmax,c1)
y1 = np.linspace(ymin,ymax,r1)
f = interp2d(x1, y1, dstnc1, kind='cubic')
but when I try to evaluate with:
test=f(xnew1,ynew1)
I get this error:
--------------------
ValueError Traceback (most recent call last)
<ipython-input-299-d0f42e609b23> in <module>()
----> 1 test=f(xnew1,ynew1)
C:\...\AppData\Local\Continuum\Anaconda\lib\site-packages\scipy\interpolate\interpolate.pyc
in __call__(self, x, y, dx, dy)
270 (self.y_min, self.y_max)))
271
--> 272 z = fitpack.bisplev(x, y, self.tck, dx, dy)
273 z = atleast_2d(z)
274 z = transpose(z)
C:\...\AppData\Local\Continuum\Anaconda\lib\site-packages\scipy\interpolate\fitpack.pyc
in bisplev(x, y, tck, dx, dy)
1027 z,ier = _fitpack._bispev(tx,ty,c,kx,ky,x,y,dx,dy)
1028 if ier == 10:
-> 1029 raise ValueError("Invalid input data")
1030 if ier:
1031 raise TypeError("An error occurred")
ValueError: Invalid input data
If I try RectBivariateSpline:
from scipy.interpolate import RectBivariateSpline
x2 = np.linspace(xmin,xmax,r1)
y2 = np.linspace(ymin,ymax,c1)
f = RectBivariateSpline(x2, y2, dstnc1)
I get this error:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-302-d0f42e609b23> in <module>()
----> 1 test=f(xnew1,ynew1)
C:\...\AppData\Local\Continuum\Anaconda\lib\site-packages\scipy\interpolate\fitpack2.pyc
in __call__(self, x, y, mth, dx, dy, grid)
643 z,ier = dfitpack.bispev(tx,ty,c,kx,ky,x,y)
644 if not ier == 0:
--> 645 raise ValueError("Error code returned by
bispev: %s" % ier)
646 else:
647 # standard Numpy broadcasting
ValueError: Error code returned by bispev: 10
Any suggestion as to whether I am using the wrong functions or the right
function with wrong syntax, and how I may fix it is appreciated. Thank you.
UPDATE
I am running Python 2.7.9 and Scipy 0.14.0 (on Continuum Anaconda)
As posted on the Scipy mailing list here the documentation seems confusing, being a mix of Scipy 0.14.0, and the next version. Can anybody suggest a workaround or the correct syntax for version 0.14.0.
UPDATE 2
tried
xnew1=np.array([3700.540199,3845.940199,3983.240199])
ynew1=np.array([1782.8611,1769.862,1694.862])
as suggested inj a comment but the error remains.

This syntax worked with RectBivariateSpline
x2 = np.linspace(xmin,xmax,c1)
y2 = np.linspace(ymin,ymax,r1)
f2 = sp.interpolate.RectBivariateSpline(x2, y2, dstnc1.T,kx=1, ky=1)
I can then evaluate at new points with this:
out2 = f2.ev(xnew1,ynew1)
For interp2d I am stuck as I am not able to bypass firewall at my office to update Anaconda (Windows). I may be able at home on a Mac installation, in which case, if I get the syntax right, I will add to thsi answer.