I'm trying to optimize a function with scipy.optimize.minimize but I can't figure out what goes where, alternatively getting the error messages "ValueError: setting an array element with a sequence" or "TypeError: llf() takes 1 positional argument but 2 were given"
My code is what follows:
import numpy as np
import pandas as pd
u = np.random.normal(0, 1, 50)
t = 25
x = t*u/(1-u)
x = np.sort(x, axis=0)
theta = list(range(1, 1001, 1))
theta = np.divide(theta, 10)
xv, tv = np.meshgrid(x, theta)
xt_sum = xv + tv # Each *theta* has been added to all values of *x*
xt_sum_inv = 1/xt_sum
xt_sum_n = np.sum(xt_sum_inv, axis=1) # This is a length 1000 vector where each entry is equal to sum(1/(theta + x))
def llf(arg):
return (-1 * (50/arg - 2 * xt_sum_n))
res = scipy.optimize.minimize(llf, theta, method='BFGS')
theta is what I am trying to optimize for.
I feel I might have either my positional arguments wrong, or my variables or function output is the wrong data structure. Any help would be much appreciated.
From the documentation,
scipy.optimize.minimize
Minimization of scalar function of one or more variables
The keyword above is scalar. Your function does not return a single value (a scalar), but many, i.e. it returns a vector.
Whatever you are trying to achieve, you are using the wrong numerical function, or you are defining the wrong target function, i.e. your llf().
Related
I am trying to solve an ODE in python using the Euler method but I am getting the
"TypeError: 'float' object is not subscriptable" error when I call the function. Here's my code:
##Parameters
g = 9.8 #in m/s^2
l = 0.5 #in m
omega0 = np.sqrt(g/l)
def euler(theta0, w0, deltat, t_end):
t0 = 0 #in s
##Constructing the arrays
t_arr = np.arange(t0, t_end + deltat, deltat)
w = np.zeros(len(t_arr)) #angular velocity in rad/s
theta = np.zeros(len(t_arr))
##Setting up our initial conditions
w[0] = w0
theta = theta0
##Performing the Euler method for both small and large angles
for i in range(len(t_arr) -1):
w[i + 1] = w[i] - ((omega0)**2)*np.sin(theta[i])*deltat
theta[i + 1] = theta[i] + w[i]*deltat
return theta
euler(0.07, 0, 0.05, 5)
Output:
TypeError: 'float' object is not subscriptable
What am I doing wrong? Please help, your help will be much appreciated.
The two lines
theta = np.zeros(len(t_arr))
...
theta = theta0
close to each other do not make sense and indicate a misunderstanding in their meaning because both put a value in the variable theta. I.e. that the first one is without any effect. The second line is the only one which is relevant.
I assume that the variable theta should hold a numpy array. But what you pass in the call of the last line as the value for the parameter theta0 is a mere float value, so that contradicts.
I guess that you might want to put the theta0 into the first field of the just created array theta:
theta[0] = theta0
Probably then it works.
You meant to set theta[0] = theta0 when you set the initial conditions (line 16), but instead you redefined theta to be a float by just writing theta = theta0, instead of the np.zeros array you had previously made it in line 12. Change that and it will work.
The 'not subscriptable' TypeError is not a particularly easy-to-understand way of explaining the error. Read this question if you want more details about what that means. The most up-voted answer is:
It basically means that the object implements the getitem() method. In other words, it describes objects that are "containers", meaning they contain other objects. This includes strings, lists, tuples, and dictionaries.
I am trying to build a linear regression model and find optimal values using fmin_cg optimizer.
I have two functions for this job. First linear_reg_cost which is cost function and second linear_reg_grad which is gradient of cost function. This functions both have same argument.
def hypothesis(x,theta):
return np.dot(x,theta)
Cost function:
def linear_reg_cost(x_flatten, y, theta_flatten, lambda_, num_of_features,num_of_samples):
x = x_flatten.reshape(num_of_samples, num_of_features)
theta = theta_flatten.reshape(n,1)
loss = hypothesis(x,theta)-y
regularizer = lambda_*np.sum(theta[1:,:]**2)/(2*m)
j = np.sum(loss ** 2)/(2*m)
return j
Gradient function:
def linear_reg_grad(x_flatten, y, theta_flatten, lambda_, num_of_features,num_of_samples):
x = x_flatten.reshape(num_of_samples, num_of_features)
m,n = x.shape
theta = theta_flatten.reshape(n,1)
new_theta = np.zeros(shape=(theta.shape))
loss = hypothesis(x,theta)-y
gradient = np.dot(x.T,loss)
new_theta[0:,:] = gradient/m
new_theta[1:,:] = gradient[1:,:]/m + lambda_*(theta[1:,]/m)
return new_theta
and fmin_cg:
theta = np.ones(n)
from scipy.optimize import fmin_cg
new_theta = fmin_cg(f=linear_reg_cost, x0=theta, fprime=linear_reg_grad,args=(x.flatten(), y, lambda_, m,n))
Note: I flatten x as input and retrieve in the cost and gradient function as matrix.
the output error:
<ipython-input-98-b29c1b8f6e58> in linear_reg_grad(x_flatten, y, theta_flatten, lambda_, num_of_features, num_of_samples)
1 def linear_reg_grad(x_flatten, y, theta_flatten, lambda_,num_of_features, num_of_samples):
----> 2 x = x_flatten.reshape(num_of_samples, num_of_features)
3 m,n = x.shape
4 theta = theta_flatten.reshape(n,1)
5 new_theta = np.zeros(shape=(theta.shape))
ValueError: cannot reshape array of size 2 into shape (2,12)
Note: x.shape = (12,2), y.shape = (12,1) ,theta.shape = (2,). So num_of_features =2 and num_of_samples=12. But error shows that my input x is parsing instead of theta. Why this happening even when I explicitly assigned args in fmin_cg? And how I should solve this problem?
Thanks for any advice
All of your implementations are correct but you have a little mistake.
Be inform to pass arguments in order for both of your functions.
Your problem is the order of num_of_feature and num_of_samples. You can replace their position with each other in linear_reg_grad or linear_reg_cost. Of course you should change this order in scipy.optimize.fmin_cg, args argument.
Second important thing is, x as first argument in fmin_cg is the variable you want to update each time and find the optimal one. So in your solution, x in fmin_cg must be theta not your x which is your input.
need help to calculate this:
so, the total number of y is equal to number of x, and each y is calculated with one x and several a.
My code list below, it gives the correct results for a0. what is a simple way to calculate this? maybe a different version can also verify the results.
Thanks a lot.
import numpy as np
import matplotlib.pyplot as plt
a = np.array([1,2,3,4],float) # here we can give several a
b = np.asarray(list(enumerate(a)))
x = np.linspace(0.0,1.0,10)
y1 = []
for r in x:
y1.append(np.exp(np.sum((1-r)**2*a*((2*b[:,0]+1)*r-1+r)*(r-1+r)**(b[:,0]-1))))
y1=np.asarray(y1)
You can write almost literally the same in numpy:
def f(x, a):
x, a = np.asanyarray(x), np.asanyarray(a)
x = x[:, None] # create new dimension to sum along
i = np.arange(len(a)) # create counter
return np.sum((1-x)**2 * a * ((2*i + 1) * x - (1-x)) * (x - (1-x))**(i-1), axis=-1)
As a side note: there are obvious algebraic simplifications you could take advantage of.
I am trying to define a function of n variables to fit to a data set. The function looks like this.
Kelly Function
I then want to find the optimal ai's and bj's to fit my data set using scipy.optimize.leastsq
Here's my code so far.
from scipy.optimize import leastsq
import numpy as np
def kellyFunc(a, b, x): #Function to fit.
top = 0
bot = 0
a = [a]
b = [b]
for i in range(len(a)):
top = top + a[i]*x**(2*i)
bot = bot + b[i]*x**(2*i)
return(top/bot)
def fitKelly(x, y, n):
line = lambda params, x : kellyFunc(params[0,:], params[1,:], x) #Lambda Function to minimize
error = lambda params, x, y : line(params, x) - y #Kelly - dataset
paramsInit = [[1 for x in range(n)] for y in range(2)] #define all ai and bi = 1 for initial guess
paramsFin, success = leastsq(error, paramsInit, args = (x,y)) #run leastsq optimization
#line of best fit
xx = np.linspace(x.min(), x.max(), 100)
yy = line(paramsFin, xx)
return(paramsFin, xx, yy)
At the moment it's giving me the error:
"IndexError: too many indices" because of the way I've defined my initial lambda function with params[0,:] and params[1,:].
There are a few problems with your approach that makes me write a full answer.
As for your specific question: leastsq doesn't really expect multidimensional arrays as parameter input. The documentation doesn't make this clear, but parameter inputs are flattened when passed to the objective function. You can verify this by using full functions instead of lambdas:
from scipy.optimize import leastsq
import numpy as np
def kellyFunc(a, b, x): #Function to fit.
top = 0
bot = 0
for i in range(len(a)):
top = top + a[i]*x**(2*i)
bot = bot + b[i]*x**(2*i)
return(top/bot)
def line(params,x):
print(repr(params)) # params is 1d!
params = params.reshape(2,-1) # need to reshape back
return kellyFunc(params[0,:], params[1,:], x)
def error(params,x,y):
print(repr(params)) # params is 1d!
return line(params, x) - y # pass it on, reshape in line()
def fitKelly(x, y, n):
#paramsInit = [[1 for x in range(n)] for y in range(2)] #define all ai and bi = 1 for initial guess
paramsInit = np.ones((n,2)) #better
paramsFin, success = leastsq(error, paramsInit, args = (x,y)) #run leastsq optimization
#line of best fit
xx = np.linspace(x.min(), x.max(), 100)
yy = line(paramsFin, xx)
return(paramsFin, xx, yy)
Now, as you see, the shape of the params array is (2*n,) instead of (2,n). By doing the re-reshape ourselves, your code (almost) works. Of course the print calls are only there to show you this fact; they are not needed for the code to run (and will produce bunch of needless output in each iteration).
See my other changes, related to other errors: you had a=[a] and b=[b] in your kellyFunc, for no good reason. This turned the input arrays into lists containing arrays, which made the next loop do something very different from what you intended.
Finally, the sneakiest error: you have input variables named x, y in fitKelly, then you use x and y is loop variables in a list comprehension. Please be aware that this only works as you expect it to in python 3; in python 2 the internal variables of list comprehensions actually leak outside the outer scope, overwriting your input variables named x and y.
I'm running a bit of code whose purpose is to take a list/array of floats and an associated list/array of the same length acting as an "error" and shuffle the first list around according to a Gaussian distribution.
This is a MWE of the code:
import random
import numpy as np
def random_data(N, a, b):
# Generate some random data.
return np.random.uniform(a, b, N).tolist()
# Obtain values for x.
x = random_data(100, 0., 1.)
# Obtain error/sigma values for x.
x_sigma = random_data(100, 0., 0.2)
# Generate new x values shuffling each float around a
# Gaussian distribution with a given sigma.
x_gauss = random.gauss(np.array(x), np.array(x_sigma))
print x-x_gauss
What I find is that the result of doing x-x_gauss is a list of floats that is always either positive or negative. This means the random.gauss call is always assigning either a larger new value for each float in x or a smaller one for all values in x.
I would expect the random.gauss call to shuffle the floats in x around its values both to the right and to the left, since this is a random process.
Why is this not happening? Am I understanding something wrong about the process?
This is the definition of random.gauss:
def gauss(self, mu, sigma):
random = self.random
z = self.gauss_next
self.gauss_next = None
if z is None:
x2pi = random() * TWOPI
g2rad = _sqrt(-2.0 * _log(1.0 - random()))
z = _cos(x2pi) * g2rad
self.gauss_next = _sin(x2pi) * g2rad
return mu + z*sigma
Notice that is is generating one value for z, and returning mu + z*sigma.
Since mu and sigma are numpy arrays, this calculation is being done element-wise. Since sigma is positive, the shift z*sigma is either always positive or negative, depending on the sign of z
If you are using NumPy, unless there is a specific reason to do otherwise, I would use the np.random module to generate these values. It would be quicker than using a Python loop with calls to random.gauss:
import numpy as np
N = 100
x = np.random.uniform(0., 1., size=N)
x_sigma = np.random.uniform(0., 0.2, size=N)
z = np.random.normal(0, 1, size=N)
x_gauss = x + z*x_sigma
print x-x_gauss