Develop Reference

3D graph error: "The truth value of an array with more than one element is ambiguous"

I am trying to plot a 3D graph, using a re-existing function to generate the Z values. However, this is yielding the error "The truth value of an array with more than one element is ambiguous". This seems strange, as I am able to generate a list of Z values using the same function and y,x values, but once I include the 3D graphing code the error occurs.
My graphing code is:
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d.axes3d import Axes3D
from matplotlib import cm
def f(tau,tau_b): #re-use society welfare function of tau & tau_b, using corr=0.6
Z = society_welfare2 (0.6, tau, tau_b)
return Z
xgrid=np.linspace(1e-5, 1-1e-5,100) #tau grid
ygrid=np.linspace(1e-5, 1-1e-5,100) #tau_b grid
tau,tau_b=np.meshgrid(xgrid,ygrid)
fig=plt.figure(figsize=(8,6))
ax=fig.add_subplot(111,projection='3d')
ax.plot_surface(tau,
tau_b,
f(tau,tau_b),
rstride=2,cstride=2,
cmap=cm.jet,
alpha=0.7,
linewidth=0.25)
ax.set_zlim(-0.5,1.0)
plt.show()
My society_welfare2 function code:
def society_welfare2 (corr, tau, tau_b):
cov = [[1,corr], [corr,1]] #covariance
epsilon_start,b_start = np.random.multivariate_normal(mean, cov, sample_N).T
epsilon = np.exp(epsilon_start) #to ensure epsilon positive
b = np.exp(b_start) #to ensure b positive
indv_welfares = []
def GBC (t_o):
taxes_paid = []
for i in range(sample_N): #loop over all agents to find their C1,C2,L
def consumption_functions(Lguess,epsilon=epsilon,b=b):
C2 = (((1-tau)*epsilon[i]*w*Lguess) +(1-tau_b)*b[i] + ((t_o)/(1+r)))/((1/((beta**(1/gamma))*((1+r)**(1/gamma)))) + (1/(1+r)))
C1 = C2 /((beta**(1/gamma))*(1+r)**(1/gamma))
return -Utility(C1,C2,Lguess)
result = minimize_scalar(consumption_functions,bounds=(0,1),method='bounded', args=(epsilon, b))
opt_L = result.x
opt_C1=(((1-tau)*(epsilon[i])*w)/(opt_L**sigma))**(1/gamma)
opt_C2=(opt_C1)*((beta**(1/gamma))*(1+r)**(1/gamma))
income_tax = tau*(epsilon[i])*w*opt_L
bequest_tax = tau_b*(b[i])
taxes_paid.append(income_tax)
taxes_paid.append(bequest_tax)
welfare_func = opt_C1**(1-gamma)/(1-gamma)-opt_L**(1+sigma)/(1+sigma) + beta*(opt_C2**(1-gamma)/(1-gamma))
indv_welfares.append(welfare_func)
total_tax_revenue = sum(taxes_paid)
return total_tax_revenue - (10000*t_o)
result1 = minimize_scalar(GBC,bounds=(1e-5, 100000),method='bounded')
opt_t_o = result1.x
total_welfare = sum(indv_welfares)
return total_welfare
The full traceback error code:
ValueError Traceback (most recent call last)
<ipython-input-19-3633f4a9db76> in <module>
18 ax.plot_surface(tau,
19 tau_b,
---> 20 f(tau,tau_b),
21 rstride=2,cstride=2,
22 cmap=cm.jet,
<ipython-input-19-3633f4a9db76> in f(tau, tau_b)
7
8 def f(tau,tau_b): #re-use society welfare function of tau & tau_b, using corr=0.6
----> 9 Z = society_welfare2 (0.6, tau, tau_b)
10 return Z
11
<ipython-input-17-321a709b9684> in society_welfare2(corr, tau, tau_b)
61 return total_tax_revenue - (10000*t_o)
62
---> 63 result1 = minimize_scalar(GBC,bounds=(1e-5, 100000),method='bounded')
64
65 opt_t_o = result1.x
/opt/anaconda3/lib/python3.8/site-packages/scipy/optimize/_minimize.py in minimize_scalar(fun, bracket, bounds, args, method, tol, options)
798 if isinstance(disp, bool):
799 options['disp'] = 2 * int(disp)
--> 800 return _minimize_scalar_bounded(fun, bounds, args, **options)
801 elif meth == 'golden':
802 return _minimize_scalar_golden(fun, bracket, args, **options)
/opt/anaconda3/lib/python3.8/site-packages/scipy/optimize/optimize.py in _minimize_scalar_bounded(func, bounds, args, xatol, maxiter, disp, **unknown_options)
1956 rat = e = 0.0
1957 x = xf
-> 1958 fx = func(x, *args)
1959 num = 1
1960 fmin_data = (1, xf, fx)
<ipython-input-17-321a709b9684> in GBC(t_o)
41 return -Utility(C1,C2,Lguess)
42
---> 43 result = minimize_scalar(consumption_functions,bounds=(0,1),method='bounded', args=(epsilon, b))
44
45 opt_L = result.x
/opt/anaconda3/lib/python3.8/site-packages/scipy/optimize/_minimize.py in minimize_scalar(fun, bracket, bounds, args, method, tol, options)
798 if isinstance(disp, bool):
799 options['disp'] = 2 * int(disp)
--> 800 return _minimize_scalar_bounded(fun, bounds, args, **options)
801 elif meth == 'golden':
802 return _minimize_scalar_golden(fun, bracket, args, **options)
/opt/anaconda3/lib/python3.8/site-packages/scipy/optimize/optimize.py in _minimize_scalar_bounded(func, bounds, args, xatol, maxiter, disp, **unknown_options)
2015 print("%5.0f %12.6g %12.6g %s" % (fmin_data + (step,)))
2016
-> 2017 if fu <= fx:
2018 if x >= xf:
2019 a = xf
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

The lowest point the trace back is
if fu <= fx:
That's comparing two variables in an if. That will work if fu and fx are scalars, or single value arrays. But if either is a multivalue array if will raise this error.
Our task at this point is to trace those variables back to your code. I suspect you are providing arrays for some parameter, where they/is should be a scalar.
Looking at the top. It occurs when you ask for the plot, but a parameter is a function call:
f(tau,tau_b)
and on through a function calls to the minimize on the GBC function. I think that GBC is the func in:
fx = func(x, *args)
Which raises the question, what exactly does GBC return? It's being used in a _minimize_scalar, so it should return exactly one value.
What is its return expression?
return total_tax_revenue - (10000*t_o)
Do you think you can take the analysis from there?
Now do you see why we insist on seeing the traceback. The error is in your code, but the sequence getting there is long, and not obvious from simply reading the code.
edit
Oops, I see another level of minimize, one that uses
consumption_functions
It has several parameters, epsilon and b. I suppose we can deduce what those are. But what is
Utility
The fu <= fx appears to be testing the fx return value against a bound fu. Assuming the bound is scalar, then the value fx must be an array. Is it???

PYTHON : IndexError: index 2 is out of bounds for axis 0 with size 2

This was my piece of code initially :
Here X is the array of data points with dimensions (m x n) where m is number of data points to predict, and n is number of features without the bias term.
y is the data labels with shape (m,)
lambda_ is the regularization term.
from scipy import optimize
def oneVsAll(X,y,num_labels,lambda_):
#used to find the optimal parametrs theta for each label against the others
#X (m,n)
#y (m,)
#num_labels : possible number of labels
#lambda_ : regularization param
#all_theta : trained param for logistic reg for each class
#hence (k,n+1) where k is #labels and n+1 is #features with bias
m,n = X.shape
all_theta = np.array((num_labels,n+1))
X = np.concatenate([np.ones((m,1)),X],axis = 1)
for k in np.arange(num_labels):
#y == k will generate a list with shape of y,but 1 only for index with value same as k and rest with 0
initial_theta = np.zeros(n+1)
options = {"maxiter" : 50}
res = optimize.minimize(lrCostFunction,
initial_theta,args = (X,y==k,lambda_),
jac = True,method = 'CG',
options = options)
all_theta[k] = res.x
return all_theta
lambda_ = 0.1
all_theta = oneVsAll(X,y,num_labels,lambda_)
The error I got was :
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-45-f9501694361e> in <module>()
1 lambda_ = 0.1
----> 2 all_theta = oneVsAll(X,y,num_labels,lambda_)
<ipython-input-44-05a9b582ccaf> in oneVsAll(X, y, num_labels, lambda_)
20 jac = True,method = 'CG',
21 options = options)
---> 22 all_theta[k] = res.x
23 return all_theta
ValueError: setting an array element with a sequence.
Then after debugging, I changed the code to :
from scipy import optimize
def oneVsAll(X,y,num_labels,lambda_):
#used to find the optimal parametrs theta for each label against the others
#X (m,n)
#y (m,)
#num_labels : possible number of labels
#lambda_ : regularization param
#all_theta : trained param for logistic reg for each class
#hence (k,n+1) where k is #labels and n+1 is #features with bias
m,n = X.shape
all_theta = np.array((num_labels,n+1),dtype = "object")
X = np.concatenate([np.ones((m,1)),X],axis = 1)
for k in np.arange(num_labels):
#y == k will generate a list with shape of y,but 1 only for index with value same as k and rest with 0
initial_theta = np.zeros(n+1)
options = {"maxiter" : 50}
res = optimize.minimize(lrCostFunction,
initial_theta,args = (X,y==k,lambda_),
jac = True,method = 'CG',
options = options)
all_theta[k] = res.x
return all_theta
Now the error I am getting is :
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-47-f9501694361e> in <module>()
1 lambda_ = 0.1
----> 2 all_theta = oneVsAll(X,y,num_labels,lambda_)
<ipython-input-46-383fc22e26cc> in oneVsAll(X, y, num_labels, lambda_)
20 jac = True,method = 'CG',
21 options = options)
---> 22 all_theta[k] = res.x
23 return all_theta
IndexError: index 2 is out of bounds for axis 0 with size 2
How can I correct this?

You create all_theta running:
all_theta = np.array((num_labels,n+1),dtype = "object")
This instruction actually creates an array containig just 2 elements
(the shape is (2,)), containing two passed values, whereas you probably
intend to pass the shape of the array to be created.
Change this instruction to:
all_theta = np.empty((num_labels,n+1))
Specification of dtype (in my opinion) is not necessary.

Can't differentiate wrt numpy arrays of dtype int64?

I am a newbie to numpy. Today when I use it to work with linear regression, it shows as below:
KeyError Traceback (most recent call
last)
~/anaconda3/lib/python3.6/site-packages/autograd/numpy/numpy_extra.py
in new_array_node(value, tapes)
84 try:
---> 85 return array_dtype_mappings[value.dtype](value, tapes)
86 except KeyError:
KeyError: dtype('int64')
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call
last)
<ipython-input-4-aebe8f7987b0> in <module>()
24 return cost/float(np.size(y))
25
---> 26 weight_h, cost_h = gradient_descent(least_squares, alpha,
max_its, w)
27
28 # a)
<ipython-input-2-1b74c4f818f4> in gradient_descent(g, alpha, max_its,
w)
12 for k in range(max_its):
13 # evaluate the gradient
---> 14 grad_eval = gradient(w)
15
16 # take gradient descent step
~/anaconda3/lib/python3.6/site-packages/autograd/core.py in
gradfun(*args, **kwargs)
19 #attach_name_and_doc(fun, argnum, 'Gradient')
20 def gradfun(*args,**kwargs):
---> 21 return
backward_pass(*forward_pass(fun,args,kwargs,argnum))
22 return gradfun
23
~/anaconda3/lib/python3.6/site-packages/autograd/core.py in
forward_pass(fun, args, kwargs, argnum)
57 tape = CalculationTape()
58 arg_wrt = args[argnum]
---> 59 start_node = new_node(safe_type(getval(arg_wrt)),
[tape])
60 args = list(args)
61 args[argnum] = merge_tapes(start_node, arg_wrt)
~/anaconda3/lib/python3.6/site-packages/autograd/core.py in
new_node(value, tapes)
185 def new_node(value, tapes=[]):
186 try:
--> 187 return Node.type_mappings[type(value)](value, tapes)
188 except KeyError:
189 return NoDerivativeNode(value, tapes)
~/anaconda3/lib/python3.6/site-packages/autograd/numpy/numpy_extra.py
in new_array_node(value, tapes)
85 return array_dtype_mappings[value.dtype](value, tapes)
86 except KeyError:
---> 87 raise TypeError("Can't differentiate wrt numpy arrays
of dtype {0}".format(value.dtype))
88 Node.type_mappings[anp.ndarray] = new_array_node
89
TypeError: Can't differentiate wrt numpy arrays of dtype int64
I really have no idea about what is happened. I guess it might be related to the structure of array in numpy. Or did I forget to download any packages? Below is my original codes.
# import statements
datapath = 'datasets/'
from autograd import numpy as np
# import automatic differentiator to compute gradient module
from autograd import grad
# gradient descent function
def gradient_descent(g,alpha,max_its,w):
# compute gradient module using autograd
gradient = grad(g)
# run the gradient descent loop
weight_history = [w] # weight history container
cost_history = [g(w)] # cost function history container
for k in range(max_its):
# evaluate the gradient
grad_eval = gradient(w)
# take gradient descent step
w = w - alpha*grad_eval
# record weight and cost
weight_history.append(w)
cost_history.append(g(w))
return weight_history,cost_history
# load in dataset
csvname = datapath + 'kleibers_law_data.csv'
data = np.loadtxt(csvname,delimiter=',')
# get input and output of dataset
x = data[:-1,:]
y = data[-1:,:]
x = np.log(x)
y = np.log(y)
#Data Initiation
alpha = 0.01
max_its = 1000
w = np.array([0,0])
#linear model
def model(x, w):
a = w[0] + np.dot(x.T, w[1:])
return a.T
def least_squares(w):
cost = np.sum((model(x,w)-y)**2)
return cost/float(np.size(y))
weight_h, cost_h = gradient_descent(least_squares, alpha, max_its, w)
# a)
k = np.linspace(-5.5, 7.5, 250)
y = weight_h[max_its][0] + k*weight_h[max_its][1]
plt.figure()
plt.plot(x, y, label='Linear Line', color='g')
plt.xlabel('log of mass')
plt.ylabel('log of metabolic rate')
plt.title("Answer Of a")
plt.legend()
plt.show()
# b)
w0 = weight_h[max_its][0]
w1 = weight_h[max_its][1]
print("Nonlinear relationship between the body mass x and the metabolic
rate y is " /
+ str(w0) + " + " + "log(xp)" + str(w1) + " = " + "log(yp)")
# c)
x2 = np.log(10)
Kj = np.exp(w0 + w1*x2)*1000/4.18
print("It needs " + str(Kj) + " calories")
Could someone help me to figure it out? Thanks a lot.

Here's the important parts of your error:
---> 14 grad_eval = gradient(w)
...
Type Error: Can't differentiate wrt numpy arrays of dtype int64
Your gradient function is saying it doesn't like to differentiate arrays of ints, which makes some sense, since it probably wants more precision than an int can give. You probably need them to be doubles or floats. For a simple solution to this, I believe you can just change your initializer from:
w = np.array([0,0])
which is going to automatically cast those 0s as ints, to:
w = np.array([0.0,0.0])
Those decimals after the 0 will let it know you want floats. There's other ways to go about telling it what kind of array you want (https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.array.html), but this is a simple way.

reshape fails with total size of new array must be unchanged even for correct dimenions

Here is an example code that I am trying
def normalizeImages(data):
x = np.shape(data)[0]
y = np.shape(data)[1]
z = np.shape(data)[2]
w = np.shape(data)[3]
darray = np.array(data)
dflat = darray.flatten()
mindflat = min(dflat)
maxdflat = max(dflat)
for x in range(0, len(dflat)):
dflat[x] = (dflat[x] - mindflat)/(maxdflat - mindflat)* 255
arraynd = np.reshape(dflat, (x, y, z, w))
return arraynd.tolist();
X_valid = normalizeImages(X_valid)
it fails with
ValueErrorTraceback (most recent call last) <ipython-input-33-eb3d96398060> in <module>()
19 return arraynd.tolist();
20
---> 21 X_valid = normalizeImages(X_valid)
22 #X_train = normalizeImages(X_train)
23 #X_test = normalizeImages(X_test)
<ipython-input-33-eb3d96398060> in normalizeImages(data)
16 for x in range(0, len(dflat)):
17 dflat[x] = (dflat[x] - mindflat)/(maxdflat - mindflat)* 255
---> 18 arraynd = np.reshape(dflat, (x, y, z, w))
19 return arraynd.tolist();
20
C:\ProgramData\Miniconda3\envs\carnd-term1\lib\site-packages\numpy\core\fromnumeric.py in reshape(a, newshape, order)
222 except AttributeError:
223 return _wrapit(a, 'reshape', newshape, order=order)
--> 224 return reshape(newshape, order=order)
225
226
ValueError: total size of new array must be unchanged
The input X_valid is a 4 dimensional list of shape (4410, 32, 32, 3). Can anyone explain why this fails and how to fix?

You are changing the value of variable x in your loop. Do this instead; you don't even need a loop for this
def normalizeImages(data):
darray = np.array(data)
x, y, z, w = np.shape(darray)
dflat = darray.flatten()
mindflat = min(dflat)
maxdflat = max(dflat)
dflat = (dflat - mindflat)/(maxdflat - mindflat) * 255
return dflat.reshape(x, y, z, w).tolist()
You don't even need to reshape your data
def normalizeImages(data):
darray = np.array(data)
mind, maxd = darray.min(), darray.max()
darray = (darray - mind)/(maxd - mind) * 255
return darray.tolist()

Converting a mixture of gaussians to PyMC3

I am trying to learn PyMC3, I want to make a simple mixture of gaussians example. I found this example and want to convert it to pymc3 but I'm currently getting an error when trying to plot the traceplot.
n1 = 500
n2 = 200
n = n1+n2
mean1 = 21.8
mean2 = 42.0
precision = 0.1
sigma = np.sqrt(1 / precision)
# precision = 1/sigma^2
print "sigma1: %s" % sigma1
print "sigma2: %s" % sigma2
data1 = np.random.normal(mean1,sigma,n1)
data2 = np.random.normal(mean2,sigma,n2)
data = np.concatenate([data1 , data2])
#np.random.shuffle(data)
fig = plt.figure(figsize=(7, 7))
ax = fig.add_subplot(111, xlabel='x', ylabel='y', title='mixture of 2 guassians')
ax.plot(range(0,n1+n2), data, 'x', label='data')
plt.legend(loc=0)
with pm.Model() as model:
#priors
p = pm.Uniform( "p", 0 , 1) #this is the fraction that come from mean1 vs mean2
ber = pm.Bernoulli( "ber", p = p) # produces 1 with proportion p.
precision = pm.Gamma('precision', alpha=0.1, beta=0.1)
mean1 = pm.Normal( "mean1", 0, 0.01 ) #better to use normals versus Uniforms (unless you are certain the value is truncated at 0 and 200
mean2 = pm.Normal( "mean2", 0, 0.01 )
mean = pm.Deterministic('mean', ber*mean1 + (1-ber)*mean2)
process = pm.Normal('process', mu=mean, tau=precision, observed=data)
# inference
step = pm.Metropolis()
trace = pm.sample(10000, step)
pm.traceplot(trace)
Error:
sigma1: 3.16227766017
sigma2: 1.69030850946
[-----------------100%-----------------] 10000 of 10000 complete in 4.4 sec
---------------------------------------------------------------------------
LinAlgError Traceback (most recent call last)
<ipython-input-10-eb728824de83> in <module>()
44 step = pm.Metropolis()
45 trace = pm.sample(10000, step)
---> 46 pm.traceplot(trace)
/usr/lib/python2.7/site-packages/pymc-3.0-py2.7.egg/pymc/plots.pyc in traceplot(trace, vars, figsize, lines, combined, grid)
70 ax[i, 0].set_xlim(mind - .5, maxd + .5)
71 else:
---> 72 kdeplot_op(ax[i, 0], d)
73 ax[i, 0].set_title(str(v))
74 ax[i, 0].grid(grid)
/usr/lib/python2.7/site-packages/pymc-3.0-py2.7.egg/pymc/plots.pyc in kdeplot_op(ax, data)
94 for i in range(data.shape[1]):
95 d = data[:, i]
---> 96 density = kde.gaussian_kde(d)
97 l = np.min(d)
98 u = np.max(d)
/usr/lib64/python2.7/site-packages/scipy/stats/kde.pyc in __init__(self, dataset, bw_method)
186
187 self.d, self.n = self.dataset.shape
--> 188 self.set_bandwidth(bw_method=bw_method)
189
190 def evaluate(self, points):
/usr/lib64/python2.7/site-packages/scipy/stats/kde.pyc in set_bandwidth(self, bw_method)
496 raise ValueError(msg)
497
--> 498 self._compute_covariance()
499
500 def _compute_covariance(self):
/usr/lib64/python2.7/site-packages/scipy/stats/kde.pyc in _compute_covariance(self)
507 self._data_covariance = atleast_2d(np.cov(self.dataset, rowvar=1,
508 bias=False))
--> 509 self._data_inv_cov = linalg.inv(self._data_covariance)
510
511 self.covariance = self._data_covariance * self.factor**2
/usr/lib64/python2.7/site-packages/scipy/linalg/basic.pyc in inv(a, overwrite_a, check_finite)
381 inv_a, info = getri(lu, piv, lwork=lwork, overwrite_lu=1)
382 if info > 0:
--> 383 raise LinAlgError("singular matrix")
384 if info < 0:
385 raise ValueError('illegal value in %d-th argument of internal '
LinAlgError: singular matrix

Thanks to Fonnesbeck for answering this on the github issue tracker:
https://github.com/pymc-devs/pymc3/issues/452
here is the updated code:
with pm.Model() as model:
#priors
p = pm.Uniform( "p", 0 , 1) #this is the fraction that come from mean1 vs mean2
ber = pm.Bernoulli( "ber", p = p, shape=len(data)) # produces 1 with proportion p.
sigma = pm.Uniform('sigma', 0, 100)
precision = sigma**-2
mean = pm.Normal( "mean", 0, 0.01, shape=2 )
mu = pm.Deterministic('mu', mean[ber])
process = pm.Normal('process', mu=mu, tau=precision, observed=data)
with model:
step1 = pm.Metropolis([p, sigma, mean])
step2 = pm.BinaryMetropolis([ber])
trace = pm.sample(10000, [step1, step2])
You need to use BinaryMetropolis when inferring a Bernoulli random variable

And an even simpler and quicker version is as follows:
with pm.Model() as model2:
p = pm.Beta( "p", 1., 1.)
means = pm.Uniform('mean', 15, 60, shape=2)
sigma = pm.Uniform('sigma', 0, 20, testval=5)
process = pm.NormalMixture('obs', tt.stack([p, 1-p]), means, sd=sigma, observed=data)
with model2:
step = pm.Metropolis()
trace = pm.sample(10000, step=step)

I know this issue is old, but I am trying differente examples of PyMC3 usages to get used to modeling in PyMC3. The answer as given above does not work in current version 1.0 of PyMC3 (It does not distringuish the two means correctly). The minimum changes I had to do in order to make it work were the following:
1)
# mean = pm.Normal("mean", 0, 0.01, shape=2 )
mean = pm.Uniform('mean', 15, 60, shape=2)
2)
# step2 = pm.BinaryMetropolis([ber])
step2 = pm.ElemwiseCategorical(vars=[ber], values=[0, 1])
Just in case anybody else is having a similar problem.